Script to create and update BibJSON file
Posted: Tue May 14, 2019 6:17 pm
If you've seen the previous script I posted to continuously update a .bib file library, you'll know that one of my strategies for this is to use bib2json and jq to return a list of citekeys from my file against which to compare current Bookends references.
Now, I only ever create a .bib file for the purpose of converting it to a .json file anyway. Why? Because JSON also works with Pandoc (indeed, it works quicker with Pandoc than a .bib file) and is more easily parsable in general. There are also a myriad of tools, both on macOS and iOS, for working with JSON and relatively few for doing so with BiBTeX.
Because of this, I've asked myself why I bother at all with the BibTeX intermediary step. Why not just export my citations directly to a JSON file and continuously update this file? Due to the greater parsability, I also reckoned that, were I to use JSON, I'd be able to not only add and subtract references, but also to change recently modified references.
So here is my first attempt at doing this:
https://gist.github.com/zverhope/1b3906 ... b99088a257
This script will create a new JSON file from scratch, which can take a bit of time if you have a large library. The initial run of my 3500 item library took about 90 minutes. Updates, however, will be much, much faster. An 80 reference test of an update, which modified 10 refs and added 70, took a couple of minutes. The resulting JSON was perfectly formatted, easily parsible, and worked great with Pandoc.
Two things: you need somewhere to store the date you last updated, which is the parameter that the script uses to perform an SQL search via Bookends in order to retrieve the references that have been modified since you last ran the script. As you'll see, the script gets this variable from Keyboard Maestro when it starts to run and then fills this variable with the current date when it finishes running. If you don't have Keyboard Maestro or something similar, you could save the date in a text file that you then read and write to when running the script, or even in a given Bookends field if you prefer.
Also, the script requires the "JSON Helper" application, which I use to convert JSON to AppleScript objects (and then back again). You can download this for free through the Mac App Store.
This is only a first version, but I just wanted to put it out there incase it was useful for anyone.
Now, I only ever create a .bib file for the purpose of converting it to a .json file anyway. Why? Because JSON also works with Pandoc (indeed, it works quicker with Pandoc than a .bib file) and is more easily parsable in general. There are also a myriad of tools, both on macOS and iOS, for working with JSON and relatively few for doing so with BiBTeX.
Because of this, I've asked myself why I bother at all with the BibTeX intermediary step. Why not just export my citations directly to a JSON file and continuously update this file? Due to the greater parsability, I also reckoned that, were I to use JSON, I'd be able to not only add and subtract references, but also to change recently modified references.
So here is my first attempt at doing this:
https://gist.github.com/zverhope/1b3906 ... b99088a257
This script will create a new JSON file from scratch, which can take a bit of time if you have a large library. The initial run of my 3500 item library took about 90 minutes. Updates, however, will be much, much faster. An 80 reference test of an update, which modified 10 refs and added 70, took a couple of minutes. The resulting JSON was perfectly formatted, easily parsible, and worked great with Pandoc.
Two things: you need somewhere to store the date you last updated, which is the parameter that the script uses to perform an SQL search via Bookends in order to retrieve the references that have been modified since you last ran the script. As you'll see, the script gets this variable from Keyboard Maestro when it starts to run and then fills this variable with the current date when it finishes running. If you don't have Keyboard Maestro or something similar, you could save the date in a text file that you then read and write to when running the script, or even in a given Bookends field if you prefer.
Also, the script requires the "JSON Helper" application, which I use to convert JSON to AppleScript objects (and then back again). You can download this for free through the Mac App Store.
This is only a first version, but I just wanted to put it out there incase it was useful for anyone.