Importing LibraryThing into Bookends

Users asking other users for import filters.
Post Reply
jeremydouglass
Posts: 33
Joined: Wed Feb 15, 2006 10:06 am

Importing LibraryThing into Bookends

Post by jeremydouglass »

For anyone interested, I've written an import filter for the purpose of taking LibraryThing.com data and importing it into Bookends. (This example uses Excel, but there are other ways.)

If you are struggling with a similar problem of getting comma-separated, tab-delimited, or spreadsheet data imported, perhaps this will help. The basic problem is that Bookends is set up for explicitly labeled, return-separated records - totally un-spreadsheet-like.

If you've been maintaining a huge collection in LibraryThing, and would you like to import it into Bookends, here's how. Note however that this is a pain and only migrates basic data - depending on your number of entries, it may not be worth your time.

1. Export from LibraryThing

From the <a href="http://www.librarything.com/extras.php">extras page</a>, select

<a href="http://www.librarything.com/export.php">Export all records as text (CSV format)</a>, and download the file to your desktop.

2. Prep data.

LibraryThing uses comma-delimited fields *with quotation marks* as a way of containing values with carriage returns carriage returns - and the Bookends Filter Designer and Source parser can't really handle that. Better to convert into tab-delimited. I used Excel for this purpose.

The only problem is that Excel can't really handle hard carriage returns in the middle of quotation marks either - so you should open your CSV file in a text editor beforehand and deleted any strat carriage returns (leaving each entry on one line). I used Smultron for this, but any text editor that supports no line wrapping will work.

Nest, I opened Excel, then used Excel to open the CSV - just double-clicking doesn't work, as it will skip the importer, which we need. During import, choosing Delimited, Commas, Quotes, done.

In Excel, I removed the column headings, added a new column on the left hand side, inserted the values 'REF 1' and 'REF 2', at the top then dragged down to mark autonumber all of the entries. Now the data is ready for Bookends.

Excel can save the file as CSV or Tab Delimited, however it will automatically add commas around everything, which Bookends can't handle. Instead, I drag-selected the squares of my grid, and cut and pasted into TextEdit - which gave me tab-delimited data with no commas. (However, if there were carriage returns in any of the fields this would cause problems, hence need to clean first.) I saved the file with as LTimport.txt

Then, I saved the file, and created a new filter in Bookends:

GENERAL
References start with: REF
[+] New references when consecutive tags are identical (I don't know why, and I've gone crazy trying to separate entries with different tags etc. etc., but any time this is off I start getting really weird errors.
Fiends end with a Return followed by: any valid tag
[+] Parse Authors/Editors: Surname First, Smart
Authors/Editors separated by: and
Keywords separated by: ,

SOURCE
Source 1 tag: REF
to tab Ignore (this is the REF tag)
to tab Ignore (this is the duplicate Authors column)
to tab Authors
to tab Title
to tab Date
to tab Call Num
to tab ISSN/ISBN
to tab Ignore (this is the nonfunctional Subjects column)
to tab Publisher
to tab Notes
to tab Keywords
to CR Ignore (this is the add date)

Save the filter, then run it on the file LTimport.txt.

The end result was readable, if not perfect, import of about 500 entries. A bit too much general info is overloaded into the Publisher field, but Author, Title, Date are all great, and that is what matters. Good luck, and here's hoping there is a much easier way.
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Post by Jon »

Hi, great work.



Another possible solution is to create a fake Bookends tab-delimited file. It has some header stuff, and all fields must be accounted for, but it would probably work, too. The arrangement of a Bookends tab-delimited field are mentioned in the User Guide. Bookends uses a backslash (\) in place of returns in these, BTW.

Jon
Sonny Software
jeremydouglass
Posts: 33
Joined: Wed Feb 15, 2006 10:06 am

Importing LibraryThing into Bookends built-in import

Post by jeremydouglass »

If you don't want to write your own import filter, this second method, as suggested by Jon, works just as well: massage the data into Bookends Tab-Delimited form, then import it using the built in filter. Here are instructions for LibraryThing - they may serve as an example for anyone trying to import a similar spreadsheet.

The LibraryThing comma delimited columns are:

AUTHOR F-L, AUTHOR L-F, TITLE, DATA, LC CALL NUMBER, ISBN, SUBJECTS, PUBLICATION, COMMENTS, TAGS, ENTRY DATE

The Bookends tab-delimited import format is:

ID, Type, Authors, Title, Ser Editors, Journal, Vol (Issue), Pages, Date, Publisher, Address, URL, Short Title, User 1, Edition, Abstract, Keywords, Notes, Translator, User4, Call Num, ISSN/ISBN, Language, User8, Trans Auth, Trans Title, Orig Pub, User12, # of Vols, User14, User16

So, to do this:

1. Export LibraryThing to CSV (as detailed in my comment above)
2. In a text editor, find any carriage returns other than those indicating a new entry, and delete or replace them with \
3. Import into Excel (as above)
4. Add a first column 'ID' autonumbered (rather than REF, as above)
5. Add a second column 'Type' filled with 'Book'
6. Reorganize the remaining columns into this order:

AUTHOR L-F, TITLE, (blank1), (blank2), (blank3), (blank4), DATA, PUBLICATIONS, (blank5), (blank6), (blank7), (blank8), (blank9), (blank10), TAGS, COMMENTS, (blank11), (blank12), LC CALL NUMBER, ISBN

7. Drag-select and copy the cells (skipping the header row) into a text file (as above), save
8. Import using "Bookends Tab-Delimited File"

Both imports produced identical results in my case.
Post Reply