Issue scraping with Beautiful Soup

后端 未结 2 527
心在旅途
心在旅途 2021-02-10 00:48

I\'ve been scraping websites before using this same technique. But with this website it seems to not work.

import urllib2
from BeautifulSoup import BeautifulSoup         


        
2条回答
  •  长发绾君心
    2021-02-10 01:37

    It means that URL you are accessing is a GIF image, not a web page. In fact, I ran the script and saved "page" to a file, and you get a 1x1 pixel white (or possibly transparent) GIF.

    The reason you don't get that with an actual web browser may in fact be because they don't want you to scrape it.

    From their terms of use: "You may not copy, reproduce, republish, download, post, broadcast, transmit or otherwise use the Site's content in any way except for your own personal, non-commercial use. "

    You could maybe fake a web-browser with some work, but I'd still recommend you to talk to WeatherOnline instead. They want you to pay for their data, but if you do so, you will surely get a nice API you can use instead of screen scraping.

提交回复
热议问题