Google scholar searches not working

A place for users to ask each other questions, make suggestions, and discuss Bookends.
jameslcooke
Posts: 3
Joined: Sat Mar 14, 2009 11:09 pm

Google scholar searches not working

Post by jameslcooke »

Searches with Google Scholar via Bookends worked fine until about noon today. After then searches with Google Scholar return no matches for any searches initiated either via the Online Search window or the Autocomplete Paper window. Searches with other search engines work fine. The same Google Scholar searches initiated via a normal web browser work fine.
melghowell
Posts: 3
Joined: Mon Feb 13, 2012 9:57 pm

Re: Google scholar searches not working

Post by melghowell »

I swear I have this problem every time I am under a deadline and really need to use this feature. It's almost enough to send me back to EndNote.
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Google scholar searches not working

Post by Jon »

Why in the world do you think this is a problem with Bookends? If any internet operation "suddenly stops working", it's obviously because something changed with the Internet source. FWIW, Google Scholar searches are working for me (Bookends 113.7) at the moment. Your problem is that either that you're using an old version of Bookends, that there is an intermittent problems with the GS servers, or it's a local issue on your machine.

Jon
Sonny Software
melghowell
Posts: 3
Joined: Mon Feb 13, 2012 9:57 pm

Re: Google scholar searches not working

Post by melghowell »

Upon discovering that the searches stopped working, I immediately verified that I am on the current version.

Can you enlighten us as to what's the optimal combination of things we should have running or installed to ensure that we don't keep running into this problem?

I am quite put off by your generally defensive tone when it comes to this issue and issues around hanging scans in complex MS Word documents (another problem that I encountered that cost me far too much time to manually fix). I am sure that Bookends is working just fine, but I am not so sure that it's actually well designed to do the job it claims to do if these are regular problems for which there is no offer of a solution that amounts to more than "they suck, and it's not our problem."

I am terribly disappointed that this is the third time I've had my work halted abruptly in the past year for this issue. There's nothing wrong with Google Scholar if I go through a browser. What you're saying is that your product simply doesn't work and, frankly, at $99 that is simply unacceptable. Early on, I praised your product to any academic Mac user who would listen. At this point, though, I'm regretting the purchase.
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Google scholar searches not working

Post by Jon »

The only optimal configuration is to have the latest update. As for Internet searches, they are entirely dependent on the online source and the user's local configuration. When a source like Google Scholar modifies their output it can remove the cues that Bookends needs to parse the response, for example. It's not a matter of "they suck", it's a matter of "they changed some behavior Bookends relied on and now Bookends needs to be updated". Dealing with online sites is a moving target. When that happens we usually get a fix out within a day. If you look at the update history, you'll see that almost every release contains notes about at least one update to deal with changes made by an online service provider.

http://www.sonnysoftware.com/new/new.html

But that's not the case here. I just tried Google Scholar again and it is still working for me. And I've had no other reports (to tech support) about a problem. And as I said, if it was working for you before and suddenly stopped, it's not internal to Bookends (which hasn't changed). But something did. Sounds local, as I said. Could be an issue with a proxy server, firewall, or something else.

Jon
Sonny Software
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Google scholar searches not working

Post by Jon »

The original poster contacted me off forum and mentioned that the problem occurred after executing 55 Autocomplete Paper transactions, and it stopped working at 50. This suggests the possible problem in his case.

Google Scholar has a defense mechanism against robotic automated retrieval engines, and will halt searches if you download too many references over a short period of time. It's not searching that triggers this, it's importing reference metadata to your library. They don't publish what the limits are, but it happens if you import a lot in a short period. In my past experience it had to be hundreds at once, but since it's not documented Google can change the limit at any time.

When this happens, you can to go Google Scholar using a browser from the same ip address and see if there is a link there that will confirm you are a real person. If there is you can click it and then search/download using Bookends again. If not, wait a day and try again. As I said, this probably explains the problem experienced by the first poster (we'll see), but not the second.

Jon
Sonny Software
melghowell
Posts: 3
Joined: Mon Feb 13, 2012 9:57 pm

Re: Google scholar searches not working

Post by melghowell »

Thank you for giving a more detailed explanation of how this works and what might actually be causing the problem. I also acknowledge that updates are generally pretty responsive, though you're SOL when it comes to waiting for the App Store to update.

The first user and I are in a similar situation, as it turns out:

I imported data from 113 search results when it stopped yielding any new results. I was doing this one record at a time while reviewing search results. I quit Bookends and my browser and rebooted my machine. Upon reboot, I launched Chrome and was able to search Google Scholar. I launched Bookends and tried the same search there, yielding no results.

I just went through this procedure again and am at a loss. Nothing at all changed with my system, so I can only assume that Google Scholar stopped liking my copy of Bookends for some reason.

Any other suggestions?
jameslcooke
Posts: 3
Joined: Sat Mar 14, 2009 11:09 pm

Re: Google scholar searches not working

Post by jameslcooke »

My Google Scholar searching is working again.

The diagnosis: I was cut off by Google!

Google cut me off because they thought I was a robot search. While I only entered about 50 manual searches via Bookends, that was enough to convince Google that I was bad news for them. So Google shut off my access. It turns out that other Google Scholar users have this problem with straight browser searches or with other applications like Bookends. I did an intensive search around the Google help and user group sites and while there are a number of entries on this problem, Google provides absolutely no details about how this cut-off occurs. If your behavior is construed by them to be in violation of their vague conditions of use policy, then they will cut you off. Sometime they might offer a captcha page to ascertain if there is a human and not a robot doing the search both and sometimes not.

The remedy: wait.

Although Google will not say how long you have to wait to have your access restored, mine was restored in 24 hours. Others have experience a 1-2 day wait time. There is a place (https://support.google.com/websearch/contact/ban) where you can lodge a complaint but there is no guarantee that they will respond or even fix the problem.

Google giveth and Google taketh away.

I appreciate what Google has offered us with Google Scholar. I don't appreciate that Google will cut us off without warning.
lbowman
Posts: 5
Joined: Sat Dec 19, 2009 6:25 pm

Re: Google scholar searches not working

Post by lbowman »

The weird thing is that if I use the "bookends browser" window, or just go straight to safari, Google Scholar continues to work.
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Google scholar searches not working

Post by Jon »

Google doesn't publish any information about this, so one just has to guess. Bookends Browser and the direct search (which uses a TCP socket) have different "user agent" signatures. It's possible that Google is tying the "web bot" behavior to the user agent, not the ip address. BTW, my recent tests suggest that importing > 20 refs at a time (not finding >20 refs, importing them) can trigger the ban. In Bookends 12.2 that 20 ref import limit will be enforced.

In any case, your access should be restored after a wait of an indefinite period.

Jon
Sonny Software
LeanneW
Posts: 2
Joined: Thu Jun 05, 2014 8:26 am

Re: Google scholar searches not working

Post by LeanneW »

I've just purchased Bookends, and what a godsend it has proven to be. :D

The reason I have posted in this particular thread is because the online search function (via Google Scholar) does not produce any matches for any article I search for. However, when I use GS via my browser I am asked by GS to type a word to prove I am not a robot. Am I in Google Scholar Jail? I really shouldn't be because I wouldn't say I have performed that many searches in a short period of time. I have only downloaded a maximum of five articles today.

Anyone who has had the same experience - when will GS allow me to search again? And is this robot block causing the 'no results' when I search via Bookends?

Thanks

LeanneW
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Google scholar searches not working

Post by Jon »

Hi, it's the webbot block (the fact that Google asked you to verify your identity tells me that). The good news is that the block will be lifted (usually in 24 hr or so). In the meantime, you can use Bookends Browser (instead of direct search) to search GS, too, and any reference with a doi should be recognized as such and be importable.

Jon
Sonny Software
LeanneW
Posts: 2
Joined: Thu Jun 05, 2014 8:26 am

Re: Google scholar searches not working

Post by LeanneW »

Thanks for your quick reply. Bit annoying on google's part, but hey ho.
KenJoh
Posts: 1
Joined: Tue Aug 05, 2014 10:32 am

Re: Google scholar searches not working

Post by KenJoh »

I am experiencing the same issue even when I change to JSTOR or even the Library of Congress. I have to resort to the browser mode, which works, but then is there a way to import citations into my library?
Jon
Site Admin
Posts: 10048
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Google scholar searches not working

Post by Jon »

Then this is a completely different issue from the Google-enforced search limit. Please double check your System Preference network settings (e.g. firewall, proxy server, etc.) Any followup should be directly with tech support.

Jon
Sonny Software
Post Reply