Scraper in Python gives “Access Denied”

前端 未结 3 1259
青春惊慌失措
青春惊慌失措 2020-12-19 12:35

I\'m trying to code a scraper in Python to get some info from a page. Like the title of the offers that appear on this page:
https://www.justdial.com/Panipat/Saree-Retai

3条回答
  •  有刺的猬
    2020-12-19 13:08

    Try this:

    import bs4
    import requests
    
    def extract_source(url):
         agent = {"User-Agent":"Mozilla/5.0"}
         source=requests.get(url, headers=agent).text
         return source
    
    def extract_data(source):
         soup=bs4.BeautifulSoup(source, 'lxml')
         names=soup.findAll('title')
         for i in names:
         print i
    
    extract_data(extract_source('https://www.justdial.com/Panipat/Saree-Retailers/nct-10420585'))
    

    I added 'lxml' to potentially avoid parse error.

提交回复
热议问题