Wrong number of results in Google Scrape with Python

Question

I was trying to learn web scraping and I am facing a freaky issue... My task is to search Google for news on a topic in a certain date range and count the number of results.

Answer 1

To add to Vikas' answer, Google will also fail to use 'custom date range' for some user-agents. That is, for certain user-agents, Google will simply search for 'recent' results instead of your specified date range.

I haven't detected a clear pattern in which user-agents will break the custom date range. It seems that including a language is a factor.

Here are some examples of user-agents that break cdr:

Mozilla/5.0 (Windows; U; Windows NT 6.1; fr-FR) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27

Mozilla/4.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/5.0)