Google Trends error: Sorry, our systems are a little stressed out right now and need to take a deep breath

问题

I have been working with Google Trends recently using Python's pytrends. And I have been experiencing the following error during random requests:

Response did not parse. See server response for details.
Sorry, our systems are a little stressed out right now and need to take a deep breath. Please try again in a few moments.

Upon searching I discovered that this was related to the Google Trends Quota limits. For example, this discusses about the mentioned issue.

But in my case, I have been experiencing this even during my first request after a long period of time and I continue to get it several times until I finally get a proper response. A successful response occurs once in around 5 requests, whereas the rest are erroneous.

Note: it has to be noted that I came across this issue only today although I never came across this when I used Google Trends during the last two days.

The associated code fragments are as follows:

    pytrends = get_pytrends()
    payload = {'q': 'chelsea', 'date' : 'now 12-H'}
    print(json.dumps(pytrends.trend(payload, return_type='json'), indent=4))

get_pytrends()

def get_pytrends():

    try:
        google_username = os.environ['GOOGLE_USERNAME']
        google_password = os.environ['GOOGLE_SECRET']
    except KeyError:
        sys.stderr.write("GOOGLE_* environment variables not set\n")
        sys.exit(1)

    pytrends = TrendReq(google_username, google_password, custom_useragent=None)

    return pytrends

What causes this issue and how should I handle it?

回答1:

The term is called load shedding: it means that google servers are smart enough to recognize when they get a load they can't handle (too many requests) so they start dropping some of them "on the floor" (meaning, respond with the error you're seeing, instead of processing it properly).

The way to deal with it from the client-side (your code) is to implement a retry mechanism with an exponential backoff sleep in between retries.

You can see more about their SLA in their docs.

来源：https://stackoverflow.com/questions/41991213/google-trends-error-sorry-our-systems-are-a-little-stressed-out-right-now-and

标签

python

google-trends