Advice needed: refugee from Papers

A place for users to ask each other questions, make suggestions, and discuss Bookends.
Post Reply
J_G
Posts: 5
Joined: Fri Jul 09, 2021 6:13 pm

Advice needed: refugee from Papers

Post by J_G »

I am thinking yet again of escaping from the black hole of Readcube-Papers. Some testing shows that Bookends autorecognition is *much* better, the interface simplify-able, and of course: it's not Springer.

I've missed them if there instructions online, but it looks like I might be able to:
- Rename my Papers PDF collection so that every file has a unique name and is all in one folder.
- Export the Papers metadata in RIS format
- Import the RIS file into Bookends
- Drag the PDFs into Bookends? Maybe in batches of 100 or so (I likely have about 10K)?

Any advice? On this approach or on the transfer in general? I haven't used Papers for PDF annotation, so I'm not worried about losing them, and I can reconstruct my groups. But I'm worried about losing metadata (including tags) and definitely about losing reference-PDF links.

I'm also wondering whether Bookends can:
- automatically place PDFs into subfolders by authorname--e.g., all "Smith" publications in one subfolder of the attachments folder. Because Papers is so bad, I've gotten used to quickly finding references through author folders.
- rename files author-date-title

Thanks for any pointers!
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Advice needed: refugee from Papers

Post by Jon »

Moving Detailed steps for moving from Papers 3 to Bookends are given here:

https://www.sonnysoftware.com/bookends/ ... mport.html

(Moving from ReadCube isn't as smooth, simply because they don't make it easy.)

As for your specific questions:

1. Bookends doesn't particularly like attachment subfolders (although it will use ones that exist) and won't organize PDFs for you. You can assign one subfolder per library, but that's not what you want (and in any case, we recommend one library).

2. You can have Bookends rename attachments any way you like. There are some built in option (like author-date), but you can also create a format for this purpose and use any element in the metadata to name your attachments.

Jon
Sonny Software
DrJJWMac
Posts: 345
Joined: Sat Jun 22, 2019 8:04 am
Location: Alabama USA

Re: Advice needed: refugee from Papers

Post by DrJJWMac »

> I am thinking yet again of escaping from the black hole of Readcube-Papers.

The four primary choices on macOS are Bookends, Mendeley, Papers, and Zotero. I am currently running all four in various stages of exploration. Another option is JabRef. I've substituted BibDesk in its place for my needs (LaTeX output).

> and of course: it's not Springer.

FWIW, the Mac Power Users forum has a scattering of discussion threads from folks using bibliography/citation management apps. The threads can sometimes reveal significant insights about the trust that users place in the developers of each of the above four apps.

> I've missed them if there instructions online, but it looks like I might be able to:
> - Rename my Papers PDF collection so that every file has a unique name and is all in one folder.

As Jon says ...

> 1. Bookends doesn't particularly like attachment subfolders (although it will use ones that exist) and won't organize PDFs for you.
> You can assign one subfolder per library, but that's not what you want (and in any case, we recommend one library).

The easiest starting point therefore is to put all attachments in one folder without sub-folders. Continuing further ...

> 2. You can have Bookends rename attachments any way you like. There are some built in option (like author-date), but you can also
> create a format for this purpose and use any element in the metadata to name your attachments.

The automatic renaming happens only at import *and only if you select it properly*. After that, renaming becomes a manual task.

> I'm also wondering whether Bookends can:
> - automatically place PDFs into subfolders by authorname--e.g., all "Smith" publications in one subfolder of the attachments folder.
> Because Papers is so > bad, I've gotten used to quickly finding references through author folders.
> - rename files author-date-title

Each of the four apps automatically moves and/or renames attachments to various degrees. All have internal mechanisms that are set at the preference level. None of the four have no *case-by-case* settings to override the global preferences per attachment. The Zotfile plug-in to Zotero will move/copy sets of attachments to a specific "tablet" folder if desired. Bookends can be set up to use a specific sub-folder for attachments to a given library and does have a dialog option to move an attachment to that sub-folder. Again, this happens only at import; after that, moving becomes a manual task.

Finally, since you are considering whether/how to use your own sub-folders, I will make two comments. First, do not go down this road in Bookends without first fully understanding all of the dangers. Jon's predicate to his first comment is to be taken seriously ... ** Bookends DOESN'T PARTICULARLY LIKE attachments subfolders ***. Secondly, you can use sub-folders in one of two ways. The most reliable way is to start at Finder level. Set up sub-folders within the main Attachments folder to Bookends. Move your attachments into the sub-folders either immediately when you import them or after they are imported to your Bookends library. Bookends will NOT move (and rename) attachments ** after ** they are imported to the library.

As a side note, if you ever do want to go down the path where you move/copy attachments into sub-folders after they have already been imported, you should follow up on the AppleScripts thread to the forum. I've posted a script that will do this.
--
JJW
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Advice needed: refugee from Papers

Post by Jon »

DrJJWMac wrote: Sat Jul 10, 2021 9:37 am The automatic renaming happens only at import *and only if you select it properly*. After that, renaming becomes a manual task.
That is not correct. You can rename attachments at any time. For example.

1. Right-click on a reference and if there are attachments, you'll see the option to Rename Attachments.
2. When a PDF is showing in the display pane, click the Action popup and select Rename.
3. In Attachment Inspector, right click and choose Rename
4. Global Change -> Rename Attachments (can be applied to the selected ref, the Hits, or the entire library at once).

Jon
Sonny Software
DrJJWMac
Posts: 345
Joined: Sat Jun 22, 2019 8:04 am
Location: Alabama USA

Re: Advice needed: refugee from Papers

Post by DrJJWMac »

I was not aware of option 4 as the automatic option.
--
JJW
J_G
Posts: 5
Joined: Fri Jul 09, 2021 6:13 pm

Re: Advice needed: refugee from Papers

Post by J_G »

Thanks! Unfortunately, Readcube-Papers (4) has eliminated almost all export options, so the Papers 3 process doesn't seem possible. I wonder whether that's intentional? :x

Here's what does mostly work:

Papers 4 only exports RIS and BibTex. Both import references into Bookends fine. I copy the PDFs into the Bookends attachments folder, and then use Refs>Get PDF>Find & Attach Local PDFs on the imported references. That had an 80-90% success rate of matching reference to PDF.

So, two new questions:

1. Can I do anything to increase that success rate? The filenames are author-date-title, but Bookends kept asking me to identify reference Author, Title on a list that including a file with that exact name. The problem may have been multiple PDFs with the same author name? Is there a naming convention I can use before importing to make the match go more smoothly?

(I also tried editing the .bib and .ris files before importing to Bookends to replace the old, Papers filepaths with the new filepaths in the Bookends attachment folder, but that didn't seem to work.)

2. Multiple keywords in Papers exports are formatted "keyword1,keyword2" which Bookends reads as a single keyword. Any work-around for that?

Finally, in case other Papers wafflers are reading this in the future: don't learn the wrong lesson from my experience! Getting out of Readcube Papers is indeed hard. That's why you should do it as soon as possible!
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Advice needed: refugee from Papers

Post by Jon »

1. If the RIS files have the file path you're part way there. Bookends wants the file name, NOT the filepath. So if it it just contains My File and the RIS filter imports it to the Attachments field, Bookends will find the PDF named My File in the default attachment folder.

FYI, Find and attach local PDFs doesn't rely on the file name, it relies on the file content (behind the scenes Bookends uses the Spotlight APIs). 80-90% is quite good, given that Bookends creates what it thinks is a good search, based on the metadata, but without knowing how Spotlight actually indexes one just has to do trial and error (not you, me). Too few words and you get lots of possible matches, too many and you get no matches. So the short answer is no, you can't affect the searches. You can do Spotlight searches yourself, of course, and tailor the searches until you get the right match. But that would be one-by-one. There is another option if your references have PMIDs or DOIs, and that's to select the ones with no attachments and try Refs -> Get PDF From Internet. It's also not going to work for all, but you might get another 50% of the remainder.

2. Once imported, do a global change -> find/replace in the Keywords field. Replace , with ¬ (¬ is made with Option-L, and it stands for the Return character).

If you want Bookends to do this for you on import, edit the RIS import filter (right side of the General tab) and tell Bookends that keywords are separated by commas (select , from the popup menu).

Jon
Sonny Software
J_G
Posts: 5
Joined: Fri Jul 09, 2021 6:13 pm

Re: Advice needed: refugee from Papers

Post by J_G »

Victory! :lol: :lol: :lol:

While I agree that 80-90% is excellent for artificial "intelligence," it turns out that 100% isn't hard--it just took some fiddling. Thanks for your help!

How to export from Readcube-Papers 4 to Bookends:

Set-up
1. Recommended: turn off wifi and make an archive of your Papers library, just in case.
2. In Papers preferences, un-check folder organization. It will make things slightly easier also if you make sure filenames don't use "space."
3. In Bookends, File>Import Manager: edit the RIS filter assigning tag "L1" to "Attachments." That will get the Papers filename into the right slot in Bookends.

Execution
1. Select some references in Papers and export them as RIS. (I moved them to a temp list to make this easier.)
2. Open the just-created export.ris file in a text editor (I used BBedit). Find the file path in one of the references (file://localhost/Users/yourname/....../) and then find and replace all with ""<--nothing. Also search for %20 and replace it with " "<--a space.
3. Copy the PDFs of those references from Papers Library to Bookends Attachments folder.
4. In Bookends, File>Import References>From File or Clipboard> using RIS.fltr, import the export.ris file into Bookends. Importing into Hits List will make things easier.
5. Clean up (e.g., batch rename the new Bookends attachments, separate keywords, clear Hits List, trash export.ris file--and whatever else I haven't discovered yet).
6. Repeat Execution until done!
DrJJWMac
Posts: 345
Joined: Sat Jun 22, 2019 8:04 am
Location: Alabama USA

Re: Advice needed: refugee from Papers

Post by DrJJWMac »

> Victory!

Glad you found a successful path and thanks for the details. I wonder whether automatic batch conversion of the RIS is possible either through BBEdit or with a new tool (that I have yet to explore) called Easy Data Transform.

I was also going to retry the method that I had used. It did not involve editing the RIS but did involve some after-the-fact semi-manual clean up. I will report back at some point later when I go to pull some other attachments out of Papers to create a new library in Bookends. This may be a while.

As for the question on "automatic" renaming, following Jon's comment ...

> That is not correct. You can rename attachments at any time. For example.

What I should say is that Bookends (and the other apps) will notrename attachments that are already imported without at least one manual invocation step. Or, if you want, the better statement might be that Bookends will allow you to rename an attachment when you first import it or, if the file is already imported, if you follow one of the methods that Jon presented. If you want to call part of all of the renaming process "automatic", there you go.
--
JJW
Post Reply