Not able to scrap the images from Flipkart.com website the src attribute is coming emtpy

好久不见. 提交于 2021-02-08 11:18:21

问题


I am able to scrap all the data from flipkart website except the images using the code below:

jobs = soup.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})

for job in jobs:
    product_name = job.find('a',{'class':'_2mylT6'})
    product_name = product_name.text if product_name else "N/A"

    product_offer_price = job.find('div',{'class':'_1vC4OE'})
    product_offer_price = product_offer_price.text if product_offer_price else "N/A"

    product_mrp = job.find('div',{'class':'_3auQ3N'})
    product_mrp = product_mrp.text if product_mrp else "N/A"

    product_link = job.find('a',{'class':'_3dqZjq'})
    product_link = product_link.get('href') if product_link else "N/A"
    product_link = url+ product_link

    product_img = job.find('div',{'class':'_3ZJShS _31bMyl'})


    print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
      format(product_name,product_offer_price,product_mrp,product_link,product_img))
    print('\n')

Results for eg.:

product name UV Protection Wayfarer Sunglasses (54)
product offer price ₹8,000
product mrp ₹8,890
product link https://www.flipkart.com/search?q=rayban/ray-ban-wayfarer- 
product image <img alt="" class="_3togXc" src=""/>

When I am manually inspecting the page the src is there but when scraping it is coming empty as above


回答1:


As I mentioned on comment use selenium.

from selenium import webdriver
from bs4 import BeautifulSoup

driver=webdriver.Chrome()
driver.get('https://www.flipkart.com/search?q=rayban/ray-ban-wayfarer')
time.sleep(3)
soup=BeautifulSoup(driver.page_source,'html.parser')
url='"https://www.flipkart.com'
jobs = soup.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})

for job in jobs:
    product_name = job.find('a',{'class':'_2mylT6'})
    product_name = product_name.text if product_name else "N/A"

    product_offer_price = job.find('div',{'class':'_1vC4OE'})
    product_offer_price = product_offer_price.text if product_offer_price else "N/A"

    product_mrp = job.find('div',{'class':'_3auQ3N'})
    product_mrp = product_mrp.text if product_mrp else "N/A"

    product_link = job.find('a',{'class':'_3dqZjq'})
    product_link = product_link.get('href') if product_link else "N/A"
    product_link = url+ product_link

    product_img =job.find('div',{'class':'_3ZJShS _31bMyl'}).find('img')['src']

    print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
      format(product_name,product_offer_price,product_mrp,product_link,product_img))
    print('\n')



回答2:


The images sources are dynamically added by the javascript part.

You should use selenium to get the page source.

Check that code:

from bs4 import BeautifulSoup as soup
from selenium import webdriver

url = 'https://www.flipkart.com/search?q=rayban'

driver = webdriver.Firefox()
driver.get(url)

html = driver.page_source
page = soup(html)

jobs = page.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})

for job in jobs:
    product_name = job.find('a',{'class':'_2mylT6'})
    product_name = product_name.text if product_name else "N/A"

    product_offer_price = job.find('div',{'class':'_1vC4OE'})
    product_offer_price = product_offer_price.text if product_offer_price else "N/A"

    product_mrp = job.find('div',{'class':'_3auQ3N'})
    product_mrp = product_mrp.text if product_mrp else "N/A"

    product_link = job.find('a',{'class':'_3dqZjq'})
    product_link = product_link.get('href') if product_link else "N/A"
    product_link = url+ product_link

    product_img = job.find('div',{'class':'_3ZJShS _31bMyl'})


    print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
      format(product_name,product_offer_price,product_mrp,product_link,product_img))
    print('\n')

OUTPUT:

product name Aviator Sunglasses (58)
product offer price ₹4,760
product mrp ₹5,290
product link https://www.flipkart.com/search?q=rayban/ray-ban-aviator-sunglasses/p/itmf3yh25kzjsapz?pid=SGLDPXDY4CMRNEY9&lid=LSTSGLDPXDY4CMRNEY9TNASSX&marketplace=FLIPKART&srno=s_1_1&otracker=search&fm=organic&iid=5ed8f60d-e315-4440-bfdc-76049d80e5da.SGLDPXDY4CMRNEY9.SEARCH&qH=95c0daefc80c4a70
product image <div class="_3ZJShS _31bMyl" style="padding-top:120.00%"><img alt="" class="_3togXc" src="https://rukminim1.flixcart.com/image/454/545/sunglass/e/y/9/0rb3129iw0228-rayban-58-original-imadqb2nzmwzfup6.jpeg?q=50"/></div>
.
.
.


来源:https://stackoverflow.com/questions/57287726/not-able-to-scrap-the-images-from-flipkart-com-website-the-src-attribute-is-comi

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!