web-crawler

问题 I am trying to scrape a website, I have got this error: AttributeError: 'NoneType' object has no attribute 'text' at ---> 12 for x in soup.select("div.site-content")] The code used is: rq = req.get("https://stopcensura.net/category/cronaca") soup = BeautifulSoup(rq.content, 'html.parser') scrape_info = [(x.h3.a.text, x.time.text) for x in soup.select("div.site-content")] I would like to get infnormation on title ( entry-title ), date ( class="date" ), the author ( <div class="by-author vcard

问题 Basically I'm trying to run some code (Python 2.7) if the content on a website changes, otherwise wait for a bit and check it later. I'm thinking of comparing hashes , the problem with this is that if the page has changed a single byte or character, the hash would be different. So for example if the page display the current date on the page, every single time the hash would be different and tell me that the content has been updated. So... How would you do this? Would you look at the Kb size

来源： https://stackoverflow.com/questions/50446485/how-can-i-scrape-through-the-microsoft-cve-webpage-that-assigns-its-content-dyna

来源： https://stackoverflow.com/questions/50446485/how-can-i-scrape-through-the-microsoft-cve-webpage-that-assigns-its-content-dyna

来源： https://stackoverflow.com/questions/51110539/scrapy-cant-find-form-on-page

来源： https://stackoverflow.com/questions/62480695/python-scrapping-of-a-site-that-contains-powerbi-graphs

来源： https://stackoverflow.com/questions/62480695/python-scrapping-of-a-site-that-contains-powerbi-graphs

来源： https://stackoverflow.com/questions/37623279/how-to-download-youtube-videos-at-1080p-resolution

来源： https://stackoverflow.com/questions/62648045/is-there-a-way-to-use-a-proxy-in-puppeteer-for-firefox

来源： https://stackoverflow.com/questions/57649734/web-scraping-video-information-from-youtube-using-python