Page 1 of 1

Bookends and workflow

Posted: Wed Oct 14, 2015 5:46 pm
by ZC1ifi
I did not want to hijack the earlier Workflow question, so here goes. I have been working on a research project for the past months and obviously Bookends is central to that. In addition I use a very good document indexer that lets me perform all sorts of searches on my pdf collection, including proximity searches and shows me the results in situ. I use Word to do the writing and a PDF viewer with built-in note and highlight facility, and a browser for web searches and consulting databases like JSTOR, EBSCO and the like.

Jon has been working to integrate some extra functionality into Bookends, but wouldn't it be wonderful if Bookends could grow and become a fully-fledged content management system? A serious one that works, not a toy like DevonThink (which I just tried again and was severely disappointed by), or some of the others that also grew out of reference managers like Papers and Mendeley.

Such a system would:
1. keep track of all references
2. will index all documents (related to a project) and allow useful searching of those documents (proximity, boolean, metadata) with instant view of results
3. lets users set up projects that can incorporate any data you can throw at it (text, sound, video, image) from wherever (mainly web, but also the PC, NAS) and keeps track of it and allows you to annotate and organise it dynamically
4 lets you write up the results of the research with an export possibility to Word and such (rtf?)

All of this can and is done now by means of different apps which means that keeping it all together is a major undertaking. I prefer to concentrate on the research and not to have to worry about where I have put the various titbits of information and such a cms would do that for me.

Is there any interest in this? Is Jon interested in something like this? Perhaps as an add-on package?

Re: Bookends and workflow

Posted: Wed Oct 14, 2015 6:21 pm
by terrydev
I'm not sure you've explored DevonThink to the fullest if you think it's a toy. What you're describing it pretty much what DTPO does.

Re: Bookends and workflow

Posted: Wed Oct 14, 2015 11:38 pm
by Cassady
Must agree with the above on DTPO. Easily does that and more. Replication, Duplication, Annotation, Indexing, Tagging, Wiki-linking, Smart searching with AI - pretty much all one would need for information management.

That being said, I clearly remember the first time I imported a sample folder into it - was *severely* underwhelmed. Never touched it again for 6 months. Couldn't understand what all the fuss was about. But gradually, I started coming across more and more articles about it, and after forcing myself to use it again - and spending some time with the tutorials and forums, the proverbial penny dropped. The issue wasn't with DTPO, but with my understanding of what it could do, in relation to my data. Once I stopped treating it as a Finder replacement, everything changed.

BE is a fine citation manager. And with the features that Jon regularly adds, it is increasingly becoming a very useful information manager as well. But personally, I'm too far into the DTPO sphere to easily change. Plenty of hours has been spent setting everything up that side, that would not see me easily switch. That's not to suggest that someone who is starting out couldn't simply use BE for everything, and use it really effectively - but if you are someone that is looking at specific tools for specific elements of your workflow (i.e. citation manager AND an information manager), then DTPO and BE are fantastic.

One last thing though: BEoT... Incredibly useful to have your entire library potentially on your iDevice, with great functionanilty included to boot. DTPO's iOS client, DTTG is, quite frankly, nowhere in comparison. They're working on a major update - but it's been very long in the offing, with little clarity on when it might release. If you are a heavy iOS user, BE becomes significantly more compelling.

To answer OP's question - this is not something I would be interested in. But no doubt many others would. I just think that the suggestion, great as it is, would see BE have to transform into something that would end up trying to do too much, in too many different areas. I'd rather have a programme focused on core aspects, and do those very well - than try and span the entire sphere, from management to citations... That's always been my biggest complaint with some of the Reference Management competitors. They try and do too much, and don't necessarily end up doing any of it, very well. Personally, that just ends up justifying my preference in having specialist tools handle specialist components of my workflow. In the same manner that I would never want DTPO to try and manage citations, or Scrivener doing the same, I would be leery of BE trying to expand to do too much... But that's just me.

Re: Bookends and workflow

Posted: Thu Oct 15, 2015 4:26 pm
by ZC1ifi
I guess I asked for it and now I am sorry I used the term "toy". Just take it as it was intended, an expression of disappointment, rather than (too much of) a derogatory term. I was hoping to start a discussion on what I thought may be the future for Bookends, not one on the merits of DevonThink. But I guess you cannot do the one without the other. I'll be the first to admit that I haven't researched DT enough even if it is the second time in a few years that I have tried it. So by all means correct me if I am wrong but as I tried to explain in the first message my research revolves around one thing only, everything else is extra: books, articles, and (on and off-line) databases. That means that at the heart of a content management system is the reference manager to which the textual (in the broadest sense of the word) material is linked. To the best of my knowledge DevonThink, unlike Papers and Mendeley, is not based on a reference manager, nor does it let you browse scientific/academic databases the way Bookends and the other reference managers do. That in itself rules it out for serious academic research (it may well be great for non-academic types of research that does not depend on references so much, but that is not what I am talking about; I am also quite happy to accept that it may be great if you combine it with a good reference manager like Bookends, but again that is not what I am on about). The second thing that is essential to my type of research at least is good searches, and to be useful (to me) that means two things above all: (1) a wide variety of search options, especially proximity searches. DT seems to be able to support this in a limited sort of way, but what is the point of knowing that the two terms you searched for appear in a 400 page book without being told WHERE they appear? (2) it should show the relevant hits in situ (unlike Spotlight, DT does this at least, but there are some serious problems with the type of documents DT supports (or rather doesn't: it cannot handle Word 97 files for example).

In short a contents management system that would suit my needs has a first class reference manager like Bookends (I cannot think of any others in the same class and I have tried many, sometimes for years: EndNote, Papers, Sente) and second, a solid, fast and comprehensive indexing engine that can handle proximity searches, supports lots of different document formats, metadata (preferably also metadata of image and sound), and is capable of showing hits in situ. After I switched to OSX a few years ago I have found both (after having used the first-rate dtSearch for years on Windows I was enormously relieved to eventually find a similar desktop search engine for OSX in the shape of CTM Foxtrot), but what is lacking is integration. To this I would love to have some of the capabilities that DT offers: project management that allows one to gather threads and notes from all over.

So DT and any of the other content management systems I have looked at does not really offer what I am looking for, hence my original suggestion.

Re: Bookends and workflow

Posted: Thu Oct 15, 2015 6:22 pm
by terrydev
Briefly, DevonThink has exactly the search you require.

Re: Bookends and workflow

Posted: Fri Oct 16, 2015 5:43 pm
by ZC1ifi
So what did I do wrong when I entered search term1 NEAR/5 term2 and was presented with documents that did not show the hits of both terms but of the single terms? So when I searched for Shakespeare NEAR Catholicism I was presented with page after page in which either of these words occurred.

Re: Bookends and workflow

Posted: Sat Oct 17, 2015 3:38 am
by terrydev
One for the DT forum I would have thought. And if you do go that route you might explain is some detail what you entered where when searching.

Re: Bookends and workflow

Posted: Sat Oct 17, 2015 4:57 am
by ZC1ifi
It is a pity that my points are not addressed, merely negated. I was hoping for a constructive discussion on how Bookends could be the centre of a contents management system, not have the whole thing end up in a plaidoyer for a particular product. Anyone else have thoughts on this matter?

Re: Bookends and workflow

Posted: Sat Oct 17, 2015 6:16 am
by NilsMS
ZC1ifi wrote:It is a pity that my points are not addressed, merely negated. I was hoping for a constructive discussion on how Bookends could be the centre of a contents management system, not have the whole thing end up in a plaidoyer for a particular product. Anyone else have thoughts on this matter?
I disagree with you, your question was answered. Generally, I am afraid that your asking for a "swiss army knife"-app is not going to work. For one, and being an academic myself, a content management system, as brilliant as it might be, is simply not enough for serious work. You also need to have the CMS (and the reference manager) tied closely with what you write, and here at the latest you need at least another app. In my opinion, your best bet is to combine those apps which are the best tools in their area. When you search the forums here or at the Scrivener-/Devonthink-sites you will find plenty of suggestions for such workflows. I am quite satisfied with cross-linking my main apps (BE, DTPO, Scrivener) by way of Applescripts, but YMMV.



Re: Bookends and workflow

Posted: Sat Oct 17, 2015 8:46 am
by Jon
Let me pop in here. Bookends is first and foremost a reference manager, and the features we maintain and add are aimed at supporting this. Believe me, that's a full-time job. There there are related activities, such as content-management, "data mining", word processing, etc. These are all major categories in themselves. I've added some features to support these (e.g. live search, direct access to SQL for complicated searches (I wonder if "near" is supported?), the tag cloud, etc.), but these fall short of what would be required in a good dedicated app. Instead of trying to be everything to all people, we've teamed up with developers of some of the best apps in some of these categories. For content management, we've worked with DevonTHINK and Eastgate Software to directly exchange data between Bookends and DTP and Tinderbox. For document management, we have worked with RedleX and Nisus to allow direct interactions between Mellel, Nisus Writer Pro, and Bookends. We've also worked with Literature & Latte to ensure that Bookends works well with Scrivener. Trying to cram all of these activities into one application would be bound to disappoint.

If you (or anyone) has specific ideas of how to improve the integration between Bookends and other apps, please make suggestions (if it's with a developer we don't already have a relationship with it's worth writing to them first to see if they want to pursue it). If it's a good idea, makes sense to implement, and the other developer is on board we'd certainly look into it.

Finally, although I don't use it regularly, I've only heard good things about DTP from our users. They have an active forum and very responsive tech support -- questions about how to use DTP for complex content management problems are likely to get fruitful replies.

Sonny Software

Re: Bookends and workflow

Posted: Sun Oct 18, 2015 10:17 am
by terrydev
My aim was not to negate your question but simply to point out that you have already an application that does what you want. I don't believe this is the place sort out DTP issues, however, and that's why I encourage you to bring it to the DevonThink forum.

As far as your core issue goes, no I don't agree, as it happens. Bookends is a very good reference manager. DTPO is an equally good CMS. No disrespect to Jon, but excellence in making a Reference Manager does not necessarily guarantee that this will translate to a CMS (and vice versa). Plus, a little application diversity is a good thing, imho. My core work is spread across DTPO, Bookends, Foxtrot Pro, Ulysses and Word. To lose any one of those apps would be a pain in the extreme. But if they are part of some all-in-one app that went down for some reason, it would be a disaster.

Re: Bookends and workflow

Posted: Mon Oct 19, 2015 4:20 am
by ZC1ifi
I don't know how to react. You refuse to address the issue I raise, yet you deny that they are issues and refer me instead to the DT forum. Of course I went there when I first noticed my problems and what do I find there when I search for for proximity:

"Cluttered Proximity Search Results ... The results returned using the "NEAR" search operator highlights all the search terms rather than just the search terms within the proximity of the query. Going through the Forum archives I found that this was a... " (top of the search list); "NEAR (proximity) search rendered useless?! Any suggestions ... When one performs a proximity searching using "NEAR", or "NEAR/10", command + G (which should find the next instance) does nothing. In order to find the next instance of any one of the te... " (second position).

The "best" answer I found was no answer at all but a workaround that is not a workaround: "This topic has been discussed many times previously in several threads. If you have set one or more proximity operators in a query, the results returned by the search will be documents that indeed meet the criteria. There are, however, non-trivial logical issues with highlighting only those occurrences of the query terms used in the proximity criterion that meet the criterion. Users have noted that a couple of Windows search apps claim to be able to do that. But using real world documents in my databases, I found that a number of them were improperly or confusingly highlighted in the Windows searches, so that their approach was untrustworthy. A possible workaround when n is small (4 or less) is to open a document that meets the proximity criterion under Preview, and search for one of the proximity criterion terms. Then inspect the text strings in the search list. If the desired result can be defined as an exact string, use that string enclosed in quotation marks, instead of a proximity search for terms contained within the string."

So referring me to the DevonThink Forum for an answer implies that an answer can be found there and that is indeed what it suggests to all readers of this thread. This is somewhat disingenuous as you can see since there is no answer.

Re: Bookends and workflow

Posted: Tue Oct 20, 2015 3:30 pm
by terrydev
My point was that perhaps you might ask the question there.