Endnote 8 XML to Bookends -- problems with filter

[pablitosan]

Hi - I'm converting from Endnote 8 and having some problems with the import process. Exporting in XML generally works better, but there seem to be some errors in mapping Endnote fields.

In particular, with book chapters, the title of the edited book gets put into the Bookends Serial Title field. I'm also concerned about having to re-link the thousands of PDFs I linked to references in Endnote, since the EN links tend to get mapped into the Bookends URL field, instead of to the Bookends native attachment system. Most of the rest seems OK after an hour's checkout, but since I'm about to import 12,000 references I'm naturally nervous about how much cleanup work there may be.

Since we can't edit the XML filter (the manual says), is there some way we can find out how the Endnote fields are mapped to Bookends fields in the XML filter? How about ways to automate relinking the PDFs, images, etc.?

[Jon]

In particular, with book chapters, the title of the edited book gets put into the Bookends Serial Title field.

I'm not sure what you mean. Are you dealing with a book chapter or an edited book? I exported a book chapter from EN 8 and imported to Bookends, and the book title field (in EN8) was imported into the book title field in Bookends.

I'm also concerned about having to re-link the thousands of PDFs I linked to references in Endnote, since the EN links tend to get mapped into the Bookends URL field, instead of to the Bookends native attachment system.

Bookends will import text in the EN8 Image field into Keywords and add the bullet to make it an attachment. It won't for the URL field, of course, because that may or may not be an attachment for EN8.

There is a Global Change -> Move option in Bookends, which will let you post-process your data. Then you'd have to do a find/replace for the URL prefix and replace with a •

Finally, for 12,000 refs, please make sure you increase database cache (Preferences). I'd go to 15 MB.

Jon
Sonny Software

[pablitosan]

Thanks Jon. I'm dealing with chapters from edited books. I've now tried this import process several times, using both EN7 XML and EN8 XML export, and every time the book title goes into the Bookends serial title field.

To be entirely clear: in EN8 this reference type is called a "book section," and the fields you are offered are, in order, author [of the chapter], year, title, editor, title [of the edited book], city, publisher, etc.

Bookends imports this type as a "book chapter" and (for me) places the book title in the Serial Title field. I was able to use Bookends' "move field" command to put them (mostly) in the right place, though.

[Jon]

Hi pablitosan,

Why don't you send me *one* reference exported from EN8 as XML and I'll take a look at it.

support@sonnysoftware.com

Jon
Sonny Software

[odysseus]

Hi - I'm converting from Endnote 8 and having some problems with the import process. Exporting in XML generally works better, but there seem to be some errors in mapping Endnote fields.

I'm seeing problems, too -- with EndNote 7. For example, the Call Number field isn't being properly mapped in Bookends. This mapping can't be adjusted, can it?

My quick testing gives me the impression that EndNote (Refer) export is generally more accurate than xml -- and it has the additional advantage that one can edit the Bookends filter -- but the advantage of xml is that the record numbers are preserved.

Is there anything that I'm missing? How can one improve on the xml import?

[Jon]

Is there anything that I'm missing? How can one improve on the xml import?

Hi odysseus,

By contacting us and explaining what the problem is. We've tweaked the import filters several times since 8 was released to deal with cases we didn't anticipate...

Jon
Sonny Software

[odysseus]

By contacting us and explaining what the problem is. We've tweaked the import filters several times since 8 was released to deal with cases we didn't anticipate...

Which is the recommended conversion method? EndNote/Refer via xml or text?

[Jon]

If you don't care about text styles (e.g. italic) or record numbers being carried over from EN, I'd use Refer. If either of those two matter to you, then use XML. Let me clarify my last post -- we have tweaked the XML import feature several times to accomodate situations we didn't anticipate. If you found another, let us know directly, with examples, and we'll see what can be done...

Jon
Sonny Software

[Henry]

I have an endnote 6 library of almost 4000 references; about 500 have attached pdf's. Based on an earlier suggestion by Jon, I've obtained a trial version of endnote 9, converted my library, and exported it as XML, so as to maintain the links to the attached pdf files. That process has worked quite well, thankfully.

What isn't working as well as I'd like: the mapping of endnote fields to bookends fields. A couple of endnote fields don't map at all as far as I can tell (the generic reference type fields "number of volumes" and "database provider"); whereas some other endnote fields are concatenated into single bookends fields (eg, EN "edition" and "type of work" both go to BE "User2"). It appears that 16 EN fields end up in the BE "notes" field; fortunately, the data from each EN field is delimited by a carriage return in the BE notes field.

One workaround would be to preprocess the EN library to move data from certain fields into other fields. Unfortunately, that will still leave me with several "orphan" fields. I have also considered adding a prefix to some of the EN fields which go into BE "notes" so that the later can later be parsed. But EN as far as I can tell does not provide a way for me to add a prefix only to fields which contain data (I have not found a way to search for either no content in a field or "any" content).

Here is a link to a pdf with a table that shows the existing mapping and the problems I'm experiencing: pdf table

The first column is the number of records in my library which have data in that field (obtained by exporting the endnote records in endnote export format, and doing searches on the field identifiers in a text editor).

Is there any way to modify the xml import mapping, for example to use some of the unused BE fields for the custom EN fields? Any other suggestions are also welcome!

[Jon]

Hi,

You can modify mapping for import filters, but not for XML export/import. If you don't have styled text, in that case, the EndNote (Refer) filter may be what you want (if you want to preserve styles, it won't be).

If you can, please send me an EN 9 XML export file with *one* reference that contains some examples of problematic data and I'll see if there are any improvements I can make (not all fields will map cleanly, and in some cases you're going to get two EN fields -> one Bookends field, but perhaps the import could be improved).

Jon
Sonny Software

[Henry]

I would gladly use EN's Endnote Export style, which exports all fields (the refer style appears to export only some of the fields), except I haven't figured out a way to maintain the links to my attached pdf's except with the xml import and export.

Here is one suggestion: could you map EN custom fields 1, 2 and 3, to BE fields User14, User15, and User16? These fields are rarely used for something specific, except for User14 which becomes Volume in a small number of reference types.

Henry

[Jon]

I would gladly use EN's Endnote Export style, which exports all fields (the refer style appears to export only some of the fields), except I haven't figured out a way to maintain the links to my attached pdf's except with the xml import and export.

Do you use keywords? If no, you could map the image field to Bookends keywords, then do a Global Change -> Change Field and put • *before* each keyword field that is not empty. I think that would work (unless EN outputs the entire pathname).

Here is one suggestion: could you map EN custom fields 1, 2 and 3, to BE fields User14, User15, and User16? These fields are rarely used for something specific, except for User14 which becomes Volume in a small number of reference types.

That might be possible. Does anyone else reading this have an objection?

Jon
Sonny Software

[Jon]

BTW, here is a workaround for importing images via EndNote Export. Do this with a fresh, empty Bookends database:

1. Use the EndNote Export style to ouput refs from EN.

2. Tweak the Bookends EndNote (Refer) import filter to import the keywords (%K) to User15 (for example) and the Image field into Keywords.

3. Import.

4. Find -> references where Keywords <> empty (select Keywords as the field to search in, put an asterisk in the first textbox)

4. Global Change -> Change Field -> put • before Keywords for all references in the hits list.

5. Global Change -> Move: move User15 to Keywords, deleting User15 when done.

You might try this with a small selection of references first, to make sure it all works (it should, I think).

Jon
Sonny Software

[thecritic]

Finally, for 12,000 refs, please make sure you increase database cache (Preferences). I'd go to 15 MB.

Would it be possible to have Bookends adjust the cache size automatically based on the number of references, instead of having the user adjust it?

[Jon]

No. The cache is set on application launch, not when a database is opened.

Jon
Sonny Software