How do you avoid duplicate CiteKeys across libraries?

A place for users to ask each other questions, make suggestions, and discuss Bookends.
Post Reply
Dellu
Posts: 268
Joined: Sun Mar 27, 2016 5:30 am

How do you avoid duplicate CiteKeys across libraries?

Post by Dellu »

I know many people in here export their library to bibtex format. I have been doing the same for a long time.

There is one issue that I have been suffering from. Many citekeys exported from my two libraries are identical.
- I have a library for my books
- I have a library for my articles

When the references are exported to bibtex format, many of the citekeys from the article library are identical to that of the book.

I have been suggested before to merge my libraries; but after a long time contemplation, I find the solution impractical for my setup


(One strategy I used was to insert [-a] as a suffix on CiteKeys of my articles using "change field" command. That again created a lot of duplicates within the article library itself).

I am really tired of this.
I spent many hours trying to cleanup these duplicated keys.

Any help would be really appreciated.
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: How do you avoid duplicate CiteKeys across libraries?

Post by Jon »

Bookends only knows the citekeys in the library being used, not others. But here are two options I can think of:

After merging libraries you can select all references and generate fresh citekeys. That will only change the citekeys of any references where there are collisions, which will be only a subset of the total.

Alternatively, you can make the citekeys of a library unique before exporting by doing a Global Change -> Change Field and putting an identifier for the library in front, or behind, the citekey. Making sure there are no spaces.

Jon
Sonny Software
zvh
Posts: 32
Joined: Sun Aug 27, 2017 12:47 am

Re: How do you avoid duplicate CiteKeys across libraries?

Post by zvh »

Just another suggestion: I agree that the best course would be to merge your libraries if possible. If not, however, one thing you could do is take the beginning of my script that updates your Bibfile by making a list of its current BibKeys. With that list, you could almost certainly find a handler AppleScript function via Google (I've seen them before, I think) that would check for duplicates within the list. You would then take the new list of duplicates and, repeating with each BibKey, would get the publication item corresponding to that BibKey. You could then add a suffix to the end of the BibKey of that item that would make it a unique record. You would then have to either reproduce your Bibfile from scratch (if many items were altered) or else could manually alter the BibKey in your Bibfile.

I don't have two libraries, so can't test. I also don't have much time right now, so can't be of further help. But I'm almost certain the above should work.
Dellu
Posts: 268
Joined: Sun Mar 27, 2016 5:30 am

Re: How do you avoid duplicate CiteKeys across libraries?

Post by Dellu »

zvh wrote: Thu May 02, 2019 12:45 pm Just another suggestion: I agree that the best course would be to merge your libraries if possible. If not, however, one thing you could do is take the beginning of my script that updates your Bibfile by making a list of its current BibKeys. With that list, you could almost certainly find a handler AppleScript function via Google (I've seen them before, I think) that would check for duplicates within the list. You would then take the new list of duplicates and, repeating with each BibKey, would get the publication item corresponding to that BibKey. You could then add a suffix to the end of the BibKey of that item that would make it a unique record. You would then have to either reproduce your Bibfile from scratch (if many items were altered) or else could manually alter the BibKey in your Bibfile.

I don't have two libraries, so can't test. I also don't have much time right now, so can't be of further help. But I'm almost certain the above should work.
Thanks for the suggestion. Both Jabref and Bibdesk can easily identify the duplicates. But, I don't want the duplicates in BE library in the first place.
Dellu
Posts: 268
Joined: Sun Mar 27, 2016 5:30 am

Re: How do you avoid duplicate CiteKeys across libraries?

Post by Dellu »

Jon wrote: Thu May 02, 2019 7:01 am
Alternatively, you can make the citekeys of a library unique before exporting by doing a Global Change -> Change Field and putting an identifier for the library in front, or behind, the citekey. Making sure there are no spaces.

Jon
Sonny Software
I have been doing this for some time. The problem with this manually modifying the CiteKey is it also creates duplicates within the single library.
I have been suffixing this[-a] to all the CiteKeys in my article library.
After some time, many references start to have a lot of duplicated CiteKeys.

If there is an author known as Anderson, his 2008 publication will have Anderson2008-a key. If he has a couple of other articles in the same year, the others will have the same key because I cannot search every entry when I manually insert the suffix.

A real solution would have been to move the citation key generation template from the universal setting into the library internal setting like the Watch folder setting. But, that might need a lot of work.

A simpler (partially manual) solution might be to add the CiteKey (user1) to the fields identified by the duplicate remover.

Can you add the CiteKey to the Duplicate remover?
Image
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: How do you avoid duplicate CiteKeys across libraries?

Post by Jon »

You don't have automatic disambiguation turned on for citekey generation in Preferences, BibTeX tab? If Key field is not unique: warn or append lowercase letter? With automatic append you can't have identical citekeys.

Jon
Sonny Software
Dellu
Posts: 268
Joined: Sun Mar 27, 2016 5:30 am

Re: How do you avoid duplicate CiteKeys across libraries?

Post by Dellu »

Jon wrote: Thu May 02, 2019 6:30 pm You don't have automatic disambiguation turned on for citekey generation in Preferences, BibTeX tab?
This is my setting:
Image
If Key field is not unique: warn or append lowercase letter? With automatic append you can't have identical citekeys.
Yes, I know. The problem is when I manually (via the Global change) add suffixes to the CiteKeys. Note: I am adding a suffix [-a] to my @articles library to avoid duplicates with the @Book library.
With the manual addition of the suffix, now the duplicates are created within the @article library.
Yesterday, I have an article with author Anderson, and year 2008. I added The suffix: [Anderson2008-a] became the Key. Today, I have another reference with the same author and year, again, when I add the suffix, I get [Anderson2008-a] key. That is duplicate.
Note that I am not adding suffixes to each reference; I am adding on batches. Therefore, the library internal duplicates are unavoidable with the manual manipulation of the keys.

with two libraries:
=>if I let BE to manage CiteKey creation by itself, it will protect library internal duplicates, but create duplicates across libraries
=>If I start to manually modify the keys by suffixing a marker using Global Change, I will protect the cross-library duplicates, but I will get library-internal duplicates.

Duplicates are unavoidable in both cases.
Nhaps
Posts: 244
Joined: Mon Sep 26, 2011 10:05 pm

Re: How do you avoid duplicate CiteKeys across libraries?

Post by Nhaps »

I use one library for everything so I don't face that problem. But if I were you I would distinguish articles with a "dash." For example Joe2000a is a book, and Joe2000-a is an article. The problem is that BE does not automatically add "dash" to your articles library.
Post Reply