Creating BibTex bibliography

Users asking other users for AppleScripts that work with Bookends.
Post Reply
kseggleton
Posts: 43
Joined: Fri Jan 01, 2016 6:47 am

Creating BibTex bibliography

Post by kseggleton »

I thought that I would post a script here, that I use for quickly creating a BibTex bibliography, for anyone else who might find it useful. My workflow is that I am writing using Pandoc and pandoc-citeproc, both on my Mac and on iOS using Editorial. A number of other people have posted on how they use Pandoc and Markdown for writing academic texts (see http://programminghistorian.org/lessons ... d-markdown). However, BibTex files are required for this workflow and BibDesk is 1. not the greatest piece of software and 2. I would not want to leave Bookends. Bookends can, however, generate BibTex files although there are two issues in relation to Pandoc use. The first is that it is a little time consuming generating an export every time you make changes to your references. The second issue is that Pandoc doesn't recognise proper nouns in titles - these have to be enclosed in curly brackets in the BibTex file. While this is possible it is pretty time consuming to do manually and defeats the purpose of writing in Pandoc and Markdown - which is speed and simplicity. The answer, that I found is to create an Applescript that generates a BibTex bibliography from a static group in Bookends and then encloses the titles of each article in curly brackets to force the capitalisation of proper nouns in titles.

My workflow is therefore to invoke the script, using Alfred (although you could put the script in the Bookends script folder and select it from the script menu). The script will ask for the name of the static group that I want to generate a bibliography for. This will then place a BibTex formatted bibliography in my dropbox folder that is accessible by both Pandoc (via the command line or via Marked http://marked2app.com with a custom processor command) and Editorial (using the Pandoc workflow).

If you do use the script below you need to change "/your/path" to the path where you will store your BibTex bibliography.

Another alternative to manually triggering the script is to make it run automatically by using a tool like Lingon - (although if I was doing this I would modify the script to generate a particular bibliography that I always wanted to keep up-to-date)

Code: Select all

--Applescript for generating a BibTex bibliography from Bookends
--Created by kseggleton
--Change /your/path/ to the name of the path to your BibTex files

display dialog "Enter name of group" default answer ""
set refGroup to text returned of the result

set permBibPath to "/your/path/" & refGroup & "-bibliography.bib"
set tempBibPath to "/your/path/" & refGroup & "-bibliography.bib.tmp"

tell application "Bookends"
	set refList to («event ToySGUID» («event ToySRUID» refGroup) given «class RRTF»:"false", string:"BibTex") as string
end tell

do shell script "echo '" & refList & "' | tr '\\r' '\\n' >" & permBibPath

do shell script "sed -E 's/(title = )({.*})/\\1{\\2}/' " & permBibPath & " > " & tempBibPath & " && mv " & tempBibPath & " " & permBibPath
iandol
Posts: 465
Joined: Fri Jan 25, 2008 2:31 pm

Re: Creating BibTex bibliography

Post by iandol »

Hi, the sed regex here does not work for me, it is greedy so matches beyond the title (.*) and because you don't use '/g' flag only modifies the first match. I use a modified version of Naupaka's script (I have >3000 refs to export which crashes applescript unless you batch them up), but have updated that to do the title case protection like so:

Code: Select all

set permBibPath to quoted form of permBibPath
set tempBibPath to quoted form of tempBibPath
set cmd to "sed -E 's/(title = )({[^}]*})/\\1{\\2}/g' " & permBibPath & " > " & tempBibPath & " && mv " & tempBibPath & " " & permBibPath
do shell script cmd
I'm using a [^}] not curly bracket to stop it being greedy, there are probably better ways to do it but BSD sed is a bit cranky. I prefer to do this with ruby -e as its regex engine is much better, but doing it with a big file is slower than sed...
kseggleton
Posts: 43
Joined: Fri Jan 01, 2016 6:47 am

Re: Creating BibTex bibliography

Post by kseggleton »

I have modified my original script recently after I realised that a better way to generate a BibTex bibliography, with preservation of the case of the title entered in Bookends, was to simply create a modified BibTex format rather than going through the process of doing a sed regex. The bibliography format that I am using in Bookends (labelled 'BibTex modified' in the AppleScript) has the following change in the format manager that has been propagated to all types of references: title = $`{{`t`}}`, $ This places a double pair of curly brackets around the title. The AppleScript for generating the BibTex bibliography is as follows:

Code: Select all

set permBibPath to "/Users/path/to/bibliography/file"
set tempBibPath to permBibPath & ".tmp"

tell application "Bookends"
	set refList to («event ToySGUID» («event ToySSQLS» "id > 0") given «class RRTF»:"false", string:"BibTex modified") as string
end tell

set theBib to (open for access tempBibPath with write permission)
set eof of theBib to 0
write refList to theBib as «class utf8»
close access theBib

do shell script "cat '" & tempBibPath & "' | tr '\\r' '\\n' > '" & permBibPath & "' && rm " & tempBibPath
Another change to my original script is that I am exporting all the references in my Bookends database with the command "id > 0". I run this script as an automated script at night (can either use launchd, Lingon or Hazel).
iandol
Posts: 465
Joined: Fri Jan 25, 2008 2:31 pm

Re: Creating BibTex bibliography

Post by iandol »

FYI: for title case preservation, I prefer a more specific solution (have a word list for preserve case and another for force case), and have a ruby script to handle this without protecting the whole title using {}

https://github.com/iandol/dotfiles/blob ... fixCase.rb

This can be called from the applescript, and can protect case for either bib or json files (json is faster if you use citeproc, so I always convert my bibtex to json for Pandoc)...
kseggleton
Posts: 43
Joined: Fri Jan 01, 2016 6:47 am

Re: Creating BibTex bibliography

Post by kseggleton »

iandol wrote:FYI: for title case preservation, I prefer a more specific solution (have a word list for preserve case and another for force case), and have a ruby script to handle this without protecting the whole title using {}
That is a nice approach @iandol. The ruby script unfortunately fails for me when I have two words adjacent to each other that both need capitalising. For example with “New Zealand” - “New” will be enclosed in curly brackets but “Zealand” will not.
iandol
Posts: 465
Joined: Fri Jan 25, 2008 2:31 pm

Re: Creating BibTex bibliography

Post by iandol »

OK, I'll fix the regex to deal with spaces tomorrow...
iandol
Posts: 465
Joined: Fri Jan 25, 2008 2:31 pm

Re: Creating BibTex bibliography

Post by iandol »

Hm, I cannot reproduce this problem, the pattern match seems to handle replacements like "New Zealand", see the regular expression test here: http://rubular.com/r/ApkLV3WZaq
Dellu
Posts: 268
Joined: Sun Mar 27, 2016 5:30 am

Re: Creating BibTex bibliography

Post by Dellu »

I was thinking to use betobibtex as a means to sync my BE library with my Bib library managed by Jabref/Bibdesk. The idea was to have a system similar to the Better BibTex plugin in Zotero: https://github.com/retorquere/zotero-better-bibtex,
https://retorque.re/zotero-better-bibtex/push-and-pull/
When exporting using Better BibTex you will be offered a new export option: Keep updated. Checking this option registers the export for automation; any changes to the collection after you’ve completed the current export will trigger an automatic re-export to update the bib file.
The problem with the current state of bebtobibtex is that it seem to overwrite the old Bib database (or write completely new file). The problem with this approach is I find the same reference being exported twice/duplicate (the newly exported ones are already part of the old/main bib database that contains all the references). What we wanted to have is a system that compares (searches) the new updates across the old database and updates the changes.

is it very hard to do a comparison?

Dear iandol, can you take this post as a request to improve (if possible) bebtobibtex so that it can occasionally update the main bib database without overwriting (new bib file) it?
iandol
Posts: 465
Joined: Fri Jan 25, 2008 2:31 pm

Re: Creating BibTex bibliography

Post by iandol »

I don't have time to work on this, as much as I'd also like to have something that is more elegant in terms of keeping the external bib file up-to-date. I just use brute force replacement for the moment...

Because a user may delete as well as add any reference, keeping track of all the refs in external BIB file and then doing a comparison is not so easy, but it would be possible. You'd ideally need to keep a separate "database" of the UUIDs for the BIB, then query BE and find the unique UUIDs to then add. So this is tractable and doable, for someone with some time on their hands... :(
Dellu
Posts: 268
Joined: Sun Mar 27, 2016 5:30 am

Re: Creating BibTex bibliography

Post by Dellu »

iandol wrote: Thu Mar 21, 2019 9:45 am You'd ideally need to keep a separate "database" of the UUIDs for the BIB, then query BE and find the unique UUIDs to then add. So this is tractable and doable, for someone with some time on their hands... :(
Exactly. That is what I have been thinking. It is even possible to query into a specific group where the group keeps lists of recently updated references.


Thank you for the reply. I am totally incapable of editing these scripts :oops: . I hope sb will come up with a solution.
Post Reply