Slow (10 seconds) opening of references from Endnote Import

[macgarvin]

I've imported 45 references into a trial Bookend Library, leaving the references within Bookends in their original Endnote folder structure, as I evaluate moving from Endnote.

Unfortunately it takes up to 10 seconds to shift from one reference to another. This seems to be because Endnote puts pdfs associated with each reference into a separate sub-folder within the Endnotes reference folder, and Bookends is slowed down by this.

Unfortunately it isn't an easy option just to migrate them all in one folder (which is my ideal anyway), because the location of the Endnote pdfs are hyperlinked in other software, including DevonThink Pro and Scrivener, and there are almost 20,000 references.These links would be broken, and I am not aware of quick fixes in either DevonThink or Scrivener.

Can anybody say whether my diagnosis is correct (putting pdfs into subfolders slows down Bookend), and/or can anyone suggest a workaround or solution?

As Endnote for mac seems increasingly like abandonware, I'm sure I am not the only Endnote user looking for a new home. Unfortunately, this problem would be a bit of a showstopper if it is unresolvable.

[Jon]

Having them is subfolders does slow things down a bit, but I doubt that's the source of the problem. Having 20K PDFs, though, exacerbates everything. I'd suggest trying two things:

1. Rebuild your library (it can't hurt).
2. Make sure that the top folder that contains all your PDFs is set as the Bookends default attachment folder. Then click "Remove Paths to Arbitrary Attachment folders).

If still slow,

3. Close the lower PDF view pane and the right edit pane. Now that the PDFs aren't being loaded, is switching between references instantaneous?

Jon
Sonny Software

[macgarvin]

Thanks Jon

Following your comments I've rebuilt the library, confirmed that the folder containing all my pdfs (in subfolders) is set as the Bookends default attachment folder, and clicked 'Remove Paths...'. The problem persists for anything involving the display of a pdf, but not otherwise.

I'm using the trial version of Bookends so I had only imported a 45 file selection from the 20,000 odd references into Bookends (I've been using Endnote for however long since it and pdfs appeared on the scene). My Endnote files are stored on Dropbox.

So I've tried the following trouble shoot:

1) Created an Endnote Library with only 45 refs including pdfs.
2) stored that locally, ie not in a dropbox folder.
3) Opened the new Endnote library with the 45 refs and then imported them into a new Bookends library (stored locally) and reset the Bookends default attachment folder to the new Endnote pdf storage folder - result = fast loading of pdfs
4) Then repeated steps 1-3, except this time putting the new Endnote library inc pdf folder onto Dropbox, again keeping the new Bookends library locally, and let it sync - result = again, fast loading of pdfs.

Puzzled.

[If it helps troubleshoot, I can send you a private note of the dropbox link to the 20,000 reference Endnote library!]

[Jon]

So when the attachment folder contains 45 PDFs -> fast

When it contains 20K PDFs -> slow

Is that right (the number of references in the Bookends library is irrelevant, it could be 2, which would allow you to switch back and forth)?

Jon
Sonny Software

[macgarvin]

Yes that is right - when there are 20,000 in the reference folder (in individual subfolders) it is slow, even though Bookends only knows about 45 of the pdfs.

[Jon]

Bookends doesn't store the pathnames of attachments, it stores the pathnames of folders that contain attachments. When you want to find the PDF, Bookends searches the folder(s) for the matching name. In your case, you have 20K PDFs, each in a subfolder (or a sub-subfolder?). Bookends has to search those individually, too, until it finds the name. I think what's happening here is that this search, which is now through many tens of thousands of folders/subfolders is what is taking the time. The time to find the file will reflect its position in the hierarchy (which is sorted by the Finder alphabetically). I'm sorry, but since you can't change the folder hierarchy because of the links to DTP and Scrivener, this will almost certainly preclude your use of Bookends.

Jon
Sonny Software

[macgarvin]

Thanks Jon

Could there an alternative route, using aliases in a new folder? One that would subsequently be used for pdfs new references by Bookends in the conventional way. It is quite easy to create aliases for all pdfs contained within a folder and subfolders, and to shift these en masse to a new location. If I place these aliases in the dedicated Bookends folder, Bookends doesn't immediately pick up on these (could it do?), but opening the inspector, clicking on 'click here to reattach' manually restores the link, and the original file in the original Endnote folder then opens quickly each time. If this could be automated, then the original files could be left in their original locations. Because Endnote allows you to create long file names (eg author-date-title) there is no problem of masses of duplicates.

Meanwhile I have contacted DevonThink Pro Support to see if there is a quick way of editing the links to the files en-masse and so solve the problem by another route (my Scrivener links are via DevonThink Pro, not Endnote).

[Jon]

I just checked -- if you place an alias of a PDF in the default attachment folder, Bookends will resolve it to the original. BUT, the file name must be identical -- i.e. it can't have the word "alias" added to it, as the Finder does. Does that help?

Jon
Sonny Software

[macgarvin]

Thanks - yes it does - in Finder I can do a right click to batch change al the the alias file names, replacing ' alias' with nothing.

That turns a showstopper into a minor inconvenience - at worst I have to relink the pdf with a couple of clicks if if I want to view it in Bookends. There are some puzzles which may be sortable - I don't understand why in the filename in Bookends inspector is appearing with a space and three digits before the .pdf, filename (which don't appear in the amended alias name and I think prevents automatic relinking in my case); external to this and I think that for longer filenames (articles with long titles) the creation of the alias which adds 'pdf alias' is truncating the 'real' filename.

But this is better than being stuck in Endnote - I'll buy a copy. Perhaps some of these steps could be automated at a future date.

[Jon]

All attachment names must be unique, or Bookends would just show you the first one it found, which might not be the one you wanted. To ensure that each name is unique, when attaching a file with a name matches an attachment already in the library, Bookends will add a random number. That's what you're seeing.

Jon
Sonny Software