How to migrate from Papers3 to Bookends

A place for users to ask each other questions, make suggestions, and discuss Bookends.
JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

How to migrate from Papers3 to Bookends

Post by JamesAFoster » Wed Aug 30, 2017 1:57 pm

ok, I've finally had it with Papers3. Why I switched from Bookends in the first place I'll never know. So, how does one do this migration?

Following the instructions in the manual and on the webpage uses the Papers3 files for attachments and notes, using a URL for the notes. But the Papers3 files do not have the annotations from Papers3. And the URL for notes doesn't show me the notes, and requires that I use Papers3 to view them.

It seems that the xml file one exports from Papers3 and imports into Bookends contains only meta-information using Papers3 files.

what I want is to have my annotated Papers3 files (with highlighting and notes) as my Bookends Attachments.

Is it possible?
james a. foster

Jon
Site Admin
Posts: 8315
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: How to migrate from Papers3 to Bookends

Post by Jon » Wed Aug 30, 2017 3:38 pm

We're going to streamline importing from Paper 3 in the next major update (upcoming changes in in Papers 3 are not going to please everyone). But that won't help with PDF annotations. IIRC Papers uses proprietary markup, not PDFKit, for annotations. Therefore they cannot be read by other apps that use PDFKit (e.g. Preview, Adobe Acrobat, and of course Bookends). I've tried highlighting PDF text in Papers 3 -- the highlights don't show up in Preview.

You can try this yourself. If you find that PDF annotations you made in Papers 3 do show up in Preview, please send me an example PDF and I'll take a look.

As for any metadata -- export a reference from Papers 3 as xml and open it in TextEdit. If you see information in that xml file that is not imported into Bookends, send me and example and I'll take a look at that, too.

Jon
Sonny Software

JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

Re: How to migrate from Papers3 to Bookends

Post by JamesAFoster » Wed Aug 30, 2017 3:52 pm

Hi, Jon. I noticed that when I EXPORT pdfs from Papers 3 to Bookends/Attachments, the results are annotated. But Bookends is still using the Papers3 directory as the source for the pdf, so it is picking up the unannotated version.

Is there a way to get Bookends to use the Bookends directory (which is what I set in my preferences) rather than the old Papers3 directory?
james a. foster

Jon
Site Admin
Posts: 8315
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: How to migrate from Papers3 to Bookends

Post by Jon » Wed Aug 30, 2017 3:57 pm

It sounds like Paper 3 is converting their own annotations to PDFKit during the export. The question is, then, which you can export PDF in batch from Paper 3. If so there should be no problem. Do that to the Bookends default attachments folder and, in Bookends Preferences, click the button Remove Arbitrary Attachments Folders. When you do that Bookends will "forget" any PDFs in the old Papers locations and find only those in the Bookends default attachments folder.

Jon
Sonny Software

Edit: Good news, you can export "PDF Files and Media" in batch from Papers 3.

JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

Re: How to migrate from Papers3 to Bookends

Post by JamesAFoster » Wed Aug 30, 2017 4:12 pm

(yes, that's how I exported the PDFs)

OK, I did Remove... Now the attachments tab in Bookends says it can't find the attachments, and gives the Papers3 name. It looks like bookends isn't using the default Bookends attachments directory.
james a. foster

Jon
Site Admin
Posts: 8315
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: How to migrate from Papers3 to Bookends

Post by Jon » Wed Aug 30, 2017 4:16 pm

Did you set it to the default in Bookends preferences?

Jon
Sonny Software

JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

Re: How to migrate from Papers3 to Bookends

Post by JamesAFoster » Wed Aug 30, 2017 4:24 pm

I think so. Here is a screen shot:
[img]
Screen Shot 2017-08-30 at 1.35.37 PM.png
Screen Shot 2017-08-30 at 1.35.37 PM.png (100.11 KiB) Viewed 3696 times
[/img]

I went to that directory and verified that it DOES have the annotated pdfs
james a. foster

Jon
Site Admin
Posts: 8315
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: How to migrate from Papers3 to Bookends

Post by Jon » Wed Aug 30, 2017 4:29 pm

Locate one of the PDFs in Bookends (where Bookends says it can't be found) and see if the attachment name exactly matches the name of the PDF in the Finder. This might go faster if you contact tech support directly (support@sonysoftware.com).

Jon
Sonny Software

JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

Re: How to migrate from Papers3 to Bookends

Post by JamesAFoster » Wed Aug 30, 2017 4:37 pm

I did that, and it does. I'll contact support. I'm sure it's just a silly thing I'm doing wrong.
james a. foster

JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

Re: How to migrate from Papers3 to Bookends

Post by JamesAFoster » Wed Aug 30, 2017 6:59 pm

Let's start over on this thread.

In Papers3, list all papers, highlight them all (cmd-A), click on File/Export/PDFs. Specify a directory (I used ~/Documents/Bookends/Attachments).

Now, how to I import all those PDFs into Bookends? Is there a way to create a new library with all those pdfs, and with the metadata filled in?
james a. foster

Jon
Site Admin
Posts: 8315
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: How to migrate from Papers3 to Bookends

Post by Jon » Wed Aug 30, 2017 7:13 pm

You have to export the Papers library as an XML file, then import that into Bookends. That tells Bookends what the PDF names are. It will look for them in the default attachment folder.

Please read these instructions for importing from various other reference managers:

http://www.sonnysoftware.com/bookends/t ... mport.html

Jon
Sonny Software

JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

Re: How to migrate from Papers3 to Bookends

Post by JamesAFoster » Wed Aug 30, 2017 7:20 pm

Oh well. That doesn't work, as the above discussion documents. That links you to the pdfs maintained in your Papers3 library, without any annotation. It does NOT look in the default directory, probably because the xml file says to look in the Papers3 directory. And when I do the steps described above, I end up with broken links to the pdfs, so I have no attachments at all. I was hoping to find a way to make Bookends use the pdfs themselves to build the library.
james a. foster

Jon
Site Admin
Posts: 8315
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: How to migrate from Papers3 to Bookends

Post by Jon » Wed Aug 30, 2017 9:14 pm

Please take this off-forum and contact tech support. There is no reason this shouldn't work, of course Bookends looks in the default attachment folder, there are no "links" to PDFs at all. This issues needs to be looked at in real time, which cannot be done with forum posts. If you want it resolved contact tech support.

Jon
Sonny Software

JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

Re: How to migrate from Papers3 to Bookends

Post by JamesAFoster » Thu Aug 31, 2017 1:25 pm

I have been working with Jon offline (great support, Jon) and here is a summary of what we have found.

The online instructions for this migration (export Papers3 repository as XML, import into Bookends, move Papers3 files into Bookends folders) will only give you the unannoted pdfs from Papers3. This is because Papers3 does uses an proprietary format for their pdfs. This limitation is documented in the Bookends manual.

You can export annotated pdfs from Papers3 using their File/Export/Export PDF function. But the pdfs you get from this process are not the names you would get if exporting to xml format. The only way to use these annotated files in Bookends curently is to reattach them to your Bookends entries manually (right click on your Bookends entry and select the annotated files). Unfortunately, this only works one entry at a time and isn't very helpful if you are migrating a large Papers3 library with lots of annotated files.

Jon and I have discussed possible workarounds. But the bottom line right now is that if you migrate from Papers3 to Bookends, you have to either lose your Papers3 annotations, or do a lot of entry-by-entry processing in Bookends. If I find a better way, I will post it here.
james a. foster

JamesAFoster
Posts: 30
Joined: Thu Jan 05, 2006 7:03 pm
Location: Idaho
Contact:

Re: How to migrate from Papers3 to Bookends

Post by JamesAFoster » Fri Sep 01, 2017 1:06 pm

Here is a summary of migrating from Papers3 to Bookends as of 2017-09-01.

First, this is a good opportunity to clean up your library. If you clean up in Papers3, then you will simplify the transition. here is the process I used (but see the next post as well):

P2B

1. create new Bookends library
2. export PDFs (good ones) from Papers3 to a temporary directory such as ~/Desktop/P3files
3. drag and drop those pdfs into Bookends, adding attachment when there is metadata AND use option to MOVE pdf files (so that the ones remaining are problems to be handled later). Press enter to skip those with corrupted pdfs. Best to do this in chunks. Be sure you have a good internet connection.
4. export Papers3 xml format (because I had about 1,000 papers3 entries without any pdfs at all)
5. import xml into bookends
6. in preferences: remove arbitrary attachments (probably not necessary)
7. Refs/Global Change/Remove Orphaned Attachments, in entire library
8. Remove Duplicates (Bookends chooses duplicates without attachments, which were added in 5; it also has other smart choices such as removing entries with missing data when other entries exist with those data)

Also consider cleaning things up with Refs/Global changes. This can fix duplicates with different case styles, or missing fields.

Files remaining in the directory from 2 need to be handled manually.
james a. foster

Post Reply