Importing LibraryThing into Bookends
Posted: Wed Feb 15, 2006 10:20 am
For anyone interested, I've written an import filter for the purpose of taking LibraryThing.com data and importing it into Bookends. (This example uses Excel, but there are other ways.)
If you are struggling with a similar problem of getting comma-separated, tab-delimited, or spreadsheet data imported, perhaps this will help. The basic problem is that Bookends is set up for explicitly labeled, return-separated records - totally un-spreadsheet-like.
If you've been maintaining a huge collection in LibraryThing, and would you like to import it into Bookends, here's how. Note however that this is a pain and only migrates basic data - depending on your number of entries, it may not be worth your time.
1. Export from LibraryThing
From the <a href="http://www.librarything.com/extras.php">extras page</a>, select
<a href="http://www.librarything.com/export.php">Export all records as text (CSV format)</a>, and download the file to your desktop.
2. Prep data.
LibraryThing uses comma-delimited fields *with quotation marks* as a way of containing values with carriage returns carriage returns - and the Bookends Filter Designer and Source parser can't really handle that. Better to convert into tab-delimited. I used Excel for this purpose.
The only problem is that Excel can't really handle hard carriage returns in the middle of quotation marks either - so you should open your CSV file in a text editor beforehand and deleted any strat carriage returns (leaving each entry on one line). I used Smultron for this, but any text editor that supports no line wrapping will work.
Nest, I opened Excel, then used Excel to open the CSV - just double-clicking doesn't work, as it will skip the importer, which we need. During import, choosing Delimited, Commas, Quotes, done.
In Excel, I removed the column headings, added a new column on the left hand side, inserted the values 'REF 1' and 'REF 2', at the top then dragged down to mark autonumber all of the entries. Now the data is ready for Bookends.
Excel can save the file as CSV or Tab Delimited, however it will automatically add commas around everything, which Bookends can't handle. Instead, I drag-selected the squares of my grid, and cut and pasted into TextEdit - which gave me tab-delimited data with no commas. (However, if there were carriage returns in any of the fields this would cause problems, hence need to clean first.) I saved the file with as LTimport.txt
Then, I saved the file, and created a new filter in Bookends:
GENERAL
References start with: REF
[+] New references when consecutive tags are identical (I don't know why, and I've gone crazy trying to separate entries with different tags etc. etc., but any time this is off I start getting really weird errors.
Fiends end with a Return followed by: any valid tag
[+] Parse Authors/Editors: Surname First, Smart
Authors/Editors separated by: and
Keywords separated by: ,
SOURCE
Source 1 tag: REF
to tab Ignore (this is the REF tag)
to tab Ignore (this is the duplicate Authors column)
to tab Authors
to tab Title
to tab Date
to tab Call Num
to tab ISSN/ISBN
to tab Ignore (this is the nonfunctional Subjects column)
to tab Publisher
to tab Notes
to tab Keywords
to CR Ignore (this is the add date)
Save the filter, then run it on the file LTimport.txt.
The end result was readable, if not perfect, import of about 500 entries. A bit too much general info is overloaded into the Publisher field, but Author, Title, Date are all great, and that is what matters. Good luck, and here's hoping there is a much easier way.
If you are struggling with a similar problem of getting comma-separated, tab-delimited, or spreadsheet data imported, perhaps this will help. The basic problem is that Bookends is set up for explicitly labeled, return-separated records - totally un-spreadsheet-like.
If you've been maintaining a huge collection in LibraryThing, and would you like to import it into Bookends, here's how. Note however that this is a pain and only migrates basic data - depending on your number of entries, it may not be worth your time.
1. Export from LibraryThing
From the <a href="http://www.librarything.com/extras.php">extras page</a>, select
<a href="http://www.librarything.com/export.php">Export all records as text (CSV format)</a>, and download the file to your desktop.
2. Prep data.
LibraryThing uses comma-delimited fields *with quotation marks* as a way of containing values with carriage returns carriage returns - and the Bookends Filter Designer and Source parser can't really handle that. Better to convert into tab-delimited. I used Excel for this purpose.
The only problem is that Excel can't really handle hard carriage returns in the middle of quotation marks either - so you should open your CSV file in a text editor beforehand and deleted any strat carriage returns (leaving each entry on one line). I used Smultron for this, but any text editor that supports no line wrapping will work.
Nest, I opened Excel, then used Excel to open the CSV - just double-clicking doesn't work, as it will skip the importer, which we need. During import, choosing Delimited, Commas, Quotes, done.
In Excel, I removed the column headings, added a new column on the left hand side, inserted the values 'REF 1' and 'REF 2', at the top then dragged down to mark autonumber all of the entries. Now the data is ready for Bookends.
Excel can save the file as CSV or Tab Delimited, however it will automatically add commas around everything, which Bookends can't handle. Instead, I drag-selected the squares of my grid, and cut and pasted into TextEdit - which gave me tab-delimited data with no commas. (However, if there were carriage returns in any of the fields this would cause problems, hence need to clean first.) I saved the file with as LTimport.txt
Then, I saved the file, and created a new filter in Bookends:
GENERAL
References start with: REF
[+] New references when consecutive tags are identical (I don't know why, and I've gone crazy trying to separate entries with different tags etc. etc., but any time this is off I start getting really weird errors.
Fiends end with a Return followed by: any valid tag
[+] Parse Authors/Editors: Surname First, Smart
Authors/Editors separated by: and
Keywords separated by: ,
SOURCE
Source 1 tag: REF
to tab Ignore (this is the REF tag)
to tab Ignore (this is the duplicate Authors column)
to tab Authors
to tab Title
to tab Date
to tab Call Num
to tab ISSN/ISBN
to tab Ignore (this is the nonfunctional Subjects column)
to tab Publisher
to tab Notes
to tab Keywords
to CR Ignore (this is the add date)
Save the filter, then run it on the file LTimport.txt.
The end result was readable, if not perfect, import of about 500 entries. A bit too much general info is overloaded into the Publisher field, but Author, Title, Date are all great, and that is what matters. Good luck, and here's hoping there is a much easier way.