Feature request: Bib sync

[Dellu]

One of the features I liked in Mendeley is the feature called Bib sync. There is no need to export the Mendeley library for having Bib files. It automatically syncs the internal library to an outside Bib file. Every time I modified my Mendeley library, the Bib file stored in the Finder is automatically updated. I use that file to work on my TEX files.

A similar feature would make Bookends much more Latex friendly. Now, every time I add a new reference, I have to export and cite that reference. The process is a bit convoluted and tiring.

is it a complex process to implement such a feature?

[Jon]

There are multiple problems with this, primarily that using a format to output the BibTeX file simply takes too long to run automatically. I might be able write a dedicated BibTeX export feature that is faster to do this, but it wouldn't be editable by users, and BibTeX is so lenient (of perhaps incomplete, for want of a better word) about what can be included no doubt you would find that unsatisfactory. And then there's the problem of how often to write it (and when to write it). Bookends updates its database every time a field is edited, do you really want a 20 MB file (for example) exported each time? Given that this is a niche feature and there is a perfectly workable alternative (you don't need to update the BibTeX file every time you add a reference of course), I'm not included to pursue it.

BTW, have you considered writing an AppleScript do this for you? It would only run on demand, but could take care of any steps in the path I think.

Jon
Sonny Software

[joao]

Just a note on this.

I would be interested in this functionality, but with a twist.
I see no need to have the entire bookends library sync with a bibtex file. But would be very interested in having Bookends groups sync with bibtex files.

As I see it, the bookends library is a repository for all research, organised by groups and whatnot. A synced bibtex file would only be used to build the final bibliography, etc for individual Latex projects. I would not be organising this bibtex with groups and whatnot in Jabref or Bibdesk (Bookends does that for me). In this sense, automatic sync does not have to be that slow, as it would only happen on a subset of the references (rather than the entire library). The best use case would be to allow local or smart groups to have individual bibtex files attached to them which would be always up to date with the references in the respective groups. As long as the user does not have a huge amount of these groups (or bookends allows us to switch sync on and off on a per-group basis), I don't imagine syncing would be that slow.
Does this make sense?

I realise this may be a more complex feature request.

[Jon]

It makes sense. And yes, it is more complex.

What's the problem with an AppleScript to do this (e.g., select a group and and return the references in BibTeX format)? It can even be called from the dock.

Jon
Sonny Software

[Dellu]

The AppleScript solution might work. But, I do not think it is a much of a progress from what we have now. Personally, the main reason I like the sync method is because it avoids Duplicates in the external library. You see, what I am doing right now is managing references both inside Bookends and Jabref(BibDesk). So, every time I have big updates in Bookends, I have to export to the BibDesk library. Since I have to use the Modified date filter to export for updates (and, new references, of course), I end up having a lot of duplicates in the BibDesk library.
i have good tools to cleanup the duplicates. But, the process just become too much; felt giving up in one of the two systems.

Had I sync feature in Bookends, I would have eliminated BibDesk (Jabref) from my life. If I have to export, there is no way to eliminate BibDesk because I have to cleanup the duplicates (remove the older copies of the newly exported references).

Anyways, if the Syncing feature is complex to implement, I won't bother about the AppleScript. Probably, I have to give up with the whole idea of exporting from Bookends; and get the references to BibDesk in a separate process (I have to import the references from Google Scholar twice in that case; which is unfortunate); while using Bookends only as PDF manager.

[Dellu]

I might be able write a dedicated BibTeX export feature that is faster to do this, but it wouldn't be editable by users, and BibTeX is so lenient (of perhaps incomplete, for want of a better word) about what can be included no doubt you would find that unsatisfactory. And then there's the problem of how often to write it (and when to write it). Bookends updates its database every time a field is edited, do you really want a 20 MB file (for example) exported each time?

Yes, the user cannot edit that file. That is understandable. If the user has to make changes, she has to change it inside Bookends. That is not problem at all. And, as to the interval of updates (exports), you can use some adhoc means to determine. Assume, if more than 5 references have been changed; or, every 10 minutes after the library is updated etc. Even better, you can set a small menu item for the user to update the Sync Bib file.The user can hit it every time she needs her changes reflected in the Bib file.

this is a niche feature

I agree, this is the thing you have to decide yourself. It is up to you if you consider the feature a valuable one; which could be of boon for the development of Bookends; or not. Bookends is really a great software. Having this feature or not is not going to make a fundamental shift. But, my impression is the feature is one progress forward for the users who are into the academics because, most academics use BibLatex (and, my guess is; majority of the users of Bookends are in the academics--hard to imagine a novelist using BE; I could be wrong). A syc with this BibTEx file would make Bookends more attractive; more useful for us (the academics).

You see, in the academics, there is a big chasm between the two systems. Some journals ask publications to be submitted in Word format; others ask in Latex format. Some bibliography managers do best in one side of the system; but terribly fail on the other side. Jabref and BibDesk are born for the Latex, they can help nothing when the journals ask us to submit in the Word format. Others like Zetoro, and Mendeley try to do both; Papers is better on the Word side. Had other internal weaknesses not debilitated it, Mendeley's Sync system came very close to close the chasm. Bookends, on the other hand, is a very capable reference manager; much much better than Mendeley. It still has little tendency to be on the Word side. I see this feature as a means to close this gap. It will make Bookends more complete system; much more Latex friendly.

[Jon]

If you agree that having an option to sync BibTeX at the user's request is useful, then I've told you it exists right now. Create an AppleScript to do this. The AppleEvents that Bookends responds to, with examples, are given in the User Guide (Help menu), starting on p. 22.

Jon
Sonny Software

[Dellu]

If you agree that having an option to sync BibTeX at the user's request is useful, then I've told you it exists right now. Create an AppleScript to do this. The AppleEvents that Bookends responds to, with examples, are given in the User Guide (Help menu), starting on p. 22.

Jon
Sonny Software

is it possible to avoid the duplication using AppleScript?

[Jon]

I'm not sure what duplication you refer to. Please explain.

Jon
Sonny Software

P.S. To respond to your point, there is certainly a hard-core group of BibTeX users among Bookends' academic users (my son is one), but it most certainly is not in wide-spread use. If one considers researchers as a whole to be "academics" (and I do), the number of Bookends users involved in biomedical research dwarfs the numbers involved in math/physics/astronomy/chemistry/etc. And if you mention BibTex to anyone in biomedical research you'll get a quizzical look and a "huh?". :-)

[iandol]

Yes why are you exporting to Bibdesk at all (which is where you say your duplicates are being created), once you have a .BIB file exported from Bookends, you don't need Bibdesk?

[joao]

What's the problem with an AppleScript to do this?

Probably no problem at all. I for one am not a fan of applescript (especially when they get big), but I imagine that in this case one would just have to call Bookends' export function and replace the existing bibtex file, so a pretty simple applescript it would be.

I don't understand the duplication issue either. Unless this is due to JabRef or Bibdesk imposing changes on the bibtex keys.
I still think it would be a nice addition to Bookends, but other things are more important (a csl style importer perhaps....).

Joao

[Dellu]

Yes why are you exporting to Bibdesk at all (which is where you say your duplicates are being created), once you have a .BIB file exported from Bookends, you don't need Bibdesk?

Because I have to add new references in the course of composing my paper. Assume you have read 20 articles in the preparation for writing a paper. They all are organised inside Bookends. You exported them. Now, you are writing the paper inside TexStudio (Texshop). You find another paper to cite; which is not in your 20 reference list. Now, you have two choices:

1. you go back to Bookends, add the new reference (from google scholar or manually); and then, re-export the reference (the references)
2. get the reference from google scholar and add it directly to your list of exported references using BibDesk (or Texshop)

The first process is impossible to live with because,if you have 50 papers to add in the course of writing the article, that means, you have to export 50 times out of Bookends. That is not acceptable.

That is why, I am maintaining a separate library inside BibDesk which keeps all the exported references from Bookends; plus other references. Had we have a syncing system, I would have added the new reference into Bookends b/c I don't have to export. It will show up in my Bib library automatically ( won't need BibDesk in that case).

[iandol]

Dellu: OK, I suspect you are visualising your PDF constantly as you write, so you need your .BIB file to be up-to-date continuously. I think the Applescript route is the only way to go, it seems much less trouble than dealing with duplicates. Don't forget you can use many different apps to assign key commands or any other triggers to an applescript (Quicksilver, Alfred, Keyboard Maestro etc etc), so it would become a very simple workflow...

[Nhaps]

I use LaTeX and BE, and would second this request. What I do is drag the reference to a static folder, then use an applescript to export and update the bib file. My request is for automatic export to a default folder each time a reference is added to the static folder. This would eliminate the need for applescript

[Jon]

I've explained multiple times why automatically generating a BibTeX file of all references with each reference addition/edit is a bad idea, and have explained how to do this on demand. I have no more to add to this thread.

Jon
Sonny Software