I am trying to scrape Google results when I search \"What is 2+2\", but the following code is returning \'NoneType\' object has no attribute \'text\'
.
A big gotcha when parsing websites is that the source code can look very different in your browser when compared to what requests
sees. The difference is javascript, which can hugely modify the DOM in a javascript capable browser.
I'd suggest 3 options:
requests
to get the page, and then examine it closely - does that tag exist when the page is retrieved by a non-js enabled agent?