Automated PubMed search not working

[casey]

My automated PubMed searches do not seem to be executed anymore. The automation worked briefly, but now I no longer get any search results even after weeks of not running Bookends. (The search date is updated in the search list, however.) I have the frequency set to 1 day, and I made sure all of the searches I want automated are checked in the search list. If I run the searches manually (either through Bookends or MyPubMed), I can find new references since my last search date. Is this a bug, or am I missing something?

Also, as time goes by with this problem I worry that I will have a backlog of references to add to my db's. Strangely, if I manually execute a search, I find references that pre-date the latest reference that I have in my database, often by a year or more. For example, I created a database with all the PubMed references found by the search "Insect Viruses"[MeSH]. Several weeks after creating this database, I run the same search again, and now I find references from 2004 that are not in the database. Is it possible that PubMed can take this long to enter references into its database? (i.e., can references this old be "new" to PubMed, or "new" to that MeSH subset? If so, is there any way to search for references by PubMed entry date, rather than publication date? Thanks.

[Jon]

Hi casey,

First, check the PubMed import filter and make sure that you have the automated search option checked ON. If you do, send me the filter and I'll look at it.

Bookends uses the web services interface for PubMed. It's identical to the one used in a browser. I think you can search by entry data -- check out the help for advanced PubMed searches link (it's at the bottom of the advanced search drawer).

Jon
Sonny Software

[casey]

Hi casey,

First, check the PubMed import filter and make sure that you have the automated search option checked ON. If you do, send me the filter and I'll look at it.

Bookends uses the web services interface for PubMed. It's identical to the one used in a browser. I think you can search by entry data -- check out the help for advanced PubMed searches link (it's at the bottom of the advanced search drawer).

Jon
Sonny Software

I don't see a checkbox for automated search when I open the PubMed import filter for editing. Do you mean the search list? The searches in question are definitely checked there.

In any case, this problem may be moot. I was able to answer my previous questions ("Is it possible that PubMed can take this long to enter references into its database? (i.e., can references this old be "new" to PubMed, or "new" to that MeSH subset? If so, is there any way to search for references by PubMed entry date, rather than publication date?"), and the answers are both "Yes". First, I discovered that you can search PubMed by entry date (using the [edat] qualifier) rather than publication date. Several test searches convinced me that this is the qualifier used for the "Entered within last..." pulldown menu in the advanced internet search window. I then discovered that you can search by the date an entry was indexed with MeSH headings (use [mhda] qualifier), and several searches demonstrated that there can be a 6 week to 6 month lag between entry date and indexing date. This explains the "old" (by publication date) references showing up in recent searches.

Finally, I searched in bookends and at NCBI using either my MeSH search terms or free text entries combined with indexing date or entry date, and it appears that nothing new is showing up in the database of indexed references (Medline) for the last couple months that matches my MeSH search terms. This is odd, as the free text search showed that papers in this field are regularly being added to PubMed during this time period, but it explains my original issues.

I have tried an automated search guaranteed to generate new refs daily (HIV), and found that search to be working fine. I'll be more confident once I finally receive hits from my other automated searches, but for the moment I think everything is ok; Medline just hasn't been updated with the refs in my field.

Casey

P.S. Any idea when v8.0.5 will be posted? I would really like to have that "manual deletion of automated search hits" feature you mentioned earlier.

[Jon]

I don't see a checkbox for automated search when I open the PubMed import filter for editing. Do you mean the search list? The searches in question are definitely checked there.

I mean the list window that appears when you use the Saved Searches popup menu at the bottom left of the filter window.

I have tried an automated search guaranteed to generate new refs daily (HIV), and found that search to be working fine. I'll be more confident once I finally receive hits from my other automated searches, but for the moment I think everything is ok; Medline just hasn't been updated with the refs in my field.

Glad to hear it's working for you as it should (Bookends, not PubMed).

P.S. Any idea when v8.0.5 will be posted? I would really like to have that "manual deletion of automated search hits" feature you mentioned earlier.

I don't have a firm date. Bookends 8.0.5 is done. But the database engine is being updated (bug fixes and speed improvements, which are very nice) and along with this has come problems with large databases (e.g. >= 10,000 references). We're working on these, and making headway, but aren't quite there yet. We're eager to get 8.0.5 out, too...

Jon
Sonny Software