Bookends for BibLaTeX — Insufficient number of UserX fields

[macula]

I am finally making a permanent (non-reversible) transition from Mellel to XeLaTeX and BibLaTeX. Please note that the latter is different from, and far superior to, the customary BibTeX system, especially for those working in the humanities.

BibLaTeX draws its immense power partly from its very sophisticated algorithms, and party from its very numerous set of fields. Each item type includes only a subset of these fields, of course. However, the large field set—much smaller than the number of available placeholders in the Bookends form views—makes the process of assigning BibLaTeX fields to bibliographic types in the Bookends "Preferences > Refs" pane particularly complicated and error prone, especially when one has to build on legacy material, as I do.

So, Jon, would you consider adding at least a dozen more UserX fields in the database (and, accordingly, the Bookends form views)? Thanks.

[Jon]

That's not something to be done lightly. Changing the number of fields would touch almost every function in Bookends (import, formatting, scanning, global changes, etc.). I'd consider it, but I'd probably implement it only if enough people found it useful (this is the first request I can recall).

Jon
Sonny Software

[macula]

Perfectly understandable. If I may daydream for a moment, it would be nice to have an arbitrary number of UserX fields, which could be added by the user in the manner of new rows in a spreadsheet. I have no idea how amenable your database design is to such an extension, but it would give a tremendous competitive advantage to the core technology of Bookends.

Speaking of competitive advantages, no bibliographic manager at the moment is truly adept at handling BibLaTeX databases. [UPDATE: Actually, two managers claim to be BibLaTeX-aware, but one is Windows-only, the other a half-baked open-source app.]This is largely due to the dizzying array of fields that the package makes provision for. Considering that BibLaTeX provides the only flawless implementation of the Chicago Style in LaTeX—and therefore is fast becoming indispensable for humanities scholars—the first manager to provide BibLaTeX support will be instantly appealing to a whole community of frustrated users. Internet forums are rife with such complaints.

If you decide to add at least a dozen more UserX fields, I would be happy to create and share a Bookends preferences file to provide BibLaTeX support out of the box.

[macula]

To digress and brainstorm on a new data model for Bookends—one that would render the application limitlessly expandable and adaptable:

1. Dispense with the current mapping between form fields (per reference type) and data field via the preferences pane. It is a bag of hurt. Things would be much easier if there were a 1-to-1 relation between form fields and data fields.

2. To this end, implement a global "new reference" form, which contains the entire set of fields that a style makes provision for. This global set of fields would be a property of the style, not an "application preference" imposed on all styles. The user would arbitrary add/remove fields on this form (e.g. using large "plus"/"minus" icons), and assign a unique global name to each field.

3. For every reference type in the style, allow the user to toggle which of these global fields are "active" (visible) and which are "inactive" (invisible).

Of course, this is all theoretical. It's easy to suggest radical changes when you don't have to get your hands dirty with maintenance, legacy code, and legacy databases…

[Nhaps]

I second the creation of new fields, any further streamlining between BE and Latex Biblatex. This could lead to a number of new Latex customers flocking to BE.

[macula]

I had forgotten about this thread but Nhaps's bump spurred me on to check out the status quo. It seems that BibLaTeX support is still uncommon among bibliographic managers, and probably incomplete when implemented.

I just had the opportunity to play around with JabRef. The database model is, in my opinion, exceptionally flexible, transparent, extensible, and lightning fast. Previews of my pretty huge BibLaTeX database, which bog Bookends down, run like a breeze on JabRef.

Case in point: to import my database (minus attachments) into JabRef, I simply opened my Bookends-exported .bib file. All reference types and fields were correctly identified and populated, including, notably, a couple of esoteric BibLaTeX fields that did not appear in the original JabRef templates. (In these cases, all I had to do is add these fields labels to the reference templates, whereby they were instantly populated.)

The comparison is somewhat unfair to Bookends, of course, which does not have the luxury of a hard-wired bibliography style (BibTeX/BibLaTeX), and therefore needs to add some layers of abstraction between bibliographic style and database contents.

If there is a weakness in the otherwise venerable and indomitable Bookends, it is that reference types and field labels are a global property of the application, whereas they should be a property of each individual bibliographic style. This is, I believe, a conceptual/architectural flaw, not merely a technical one. (Indeed, different bibliographic styles define their item types and fields differently.)

A BibTeX/BibLaTeX formatter hard-coded into Bookends would also help, because currently the formatter relies on a bunch of conditionals, which are terribly slow on Bookends.

That said, the overall user experience under JabRef is abysmal. I could not cope with it for longer than a few minutes.

[Jon]

…because currently the formatter relies on a bunch of conditionals, which are terribly slow on Bookends.

If some new code holds up through beta testing, I think you'll you'll find exporting BibTeX-formatted references is much faster in Bookends 12.2.

Jon
Sonny Software

[macula]

Thanks as ever, Jon. Looking forward to the next release.

[Nhaps]

Macula, thanks for your experience with JabRef and important observations about fields that need to be associates with references in BE. I am still struggling with a workflow for Latex from BE. Migration to BibDesk from BE was not good, and there is a learning curve as well. I have a powerful computer so lagging is not a problem for BE operations. To sum, we need better tools for LaTeX interchange in BE. Jon, how can I private message Macula?

[macula]

Thanks are due not to me but to Jon for developing the best product in the business and for the stellar customer support. I don't think that PM-ing is possible on this forum. But would it be necessary anyway? Unless there is a compeling reason to the contrary (e.g. private data), why not keep the discussion public?

[Nhaps]

Not a request to make the discussion private, but to ask further questions relating to LaTeX that are outside the purview of BE.

[macula]

If some new code holds up through beta testing, I think you'll you'll find exporting BibTeX-formatted references is much faster in Bookends 12.2.

Jon

Jon, I haven't yet run a benchmark on my approx. 1800 item database but will do so asap.

Meanwhile, could you clarify what exactly has been optimized in BE 12.2? Is it conditionals in format defintions that now run faster? Unfortunately, on my system, my custom BibLaTeX style is still laggy.

PS: Thanks for a fantastic release, the PDF annotation management is a marvel (now if I could have these notestreams imported or indexed by DevonThink

[Jon]

It's technical. But the way that Bookends handles large amounts of text is faster. That means when outputting hundreds of references, it doesn't slow down as more and more are processed. BibTeX is still slower than most normal formats, but it's a lot faster than before. The more references you output the more obvious this is.

Jon
Sonny Software