问题
I am able to scrap all the data from flipkart website except the images using the code below:
jobs = soup.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})
for job in jobs:
product_name = job.find('a',{'class':'_2mylT6'})
product_name = product_name.text if product_name else "N/A"
product_offer_price = job.find('div',{'class':'_1vC4OE'})
product_offer_price = product_offer_price.text if product_offer_price else "N/A"
product_mrp = job.find('div',{'class':'_3auQ3N'})
product_mrp = product_mrp.text if product_mrp else "N/A"
product_link = job.find('a',{'class':'_3dqZjq'})
product_link = product_link.get('href') if product_link else "N/A"
product_link = url+ product_link
product_img = job.find('div',{'class':'_3ZJShS _31bMyl'})
print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
format(product_name,product_offer_price,product_mrp,product_link,product_img))
print('\n')
Results for eg.:
product name UV Protection Wayfarer Sunglasses (54)
product offer price ₹8,000
product mrp ₹8,890
product link https://www.flipkart.com/search?q=rayban/ray-ban-wayfarer-
product image <img alt="" class="_3togXc" src=""/>
When I am manually inspecting the page the src is there but when scraping it is coming empty as above
回答1:
As I mentioned on comment use selenium.
from selenium import webdriver
from bs4 import BeautifulSoup
driver=webdriver.Chrome()
driver.get('https://www.flipkart.com/search?q=rayban/ray-ban-wayfarer')
time.sleep(3)
soup=BeautifulSoup(driver.page_source,'html.parser')
url='"https://www.flipkart.com'
jobs = soup.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})
for job in jobs:
product_name = job.find('a',{'class':'_2mylT6'})
product_name = product_name.text if product_name else "N/A"
product_offer_price = job.find('div',{'class':'_1vC4OE'})
product_offer_price = product_offer_price.text if product_offer_price else "N/A"
product_mrp = job.find('div',{'class':'_3auQ3N'})
product_mrp = product_mrp.text if product_mrp else "N/A"
product_link = job.find('a',{'class':'_3dqZjq'})
product_link = product_link.get('href') if product_link else "N/A"
product_link = url+ product_link
product_img =job.find('div',{'class':'_3ZJShS _31bMyl'}).find('img')['src']
print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
format(product_name,product_offer_price,product_mrp,product_link,product_img))
print('\n')
回答2:
The images sources are dynamically added by the javascript part.
You should use selenium to get the page source.
Check that code:
from bs4 import BeautifulSoup as soup
from selenium import webdriver
url = 'https://www.flipkart.com/search?q=rayban'
driver = webdriver.Firefox()
driver.get(url)
html = driver.page_source
page = soup(html)
jobs = page.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})
for job in jobs:
product_name = job.find('a',{'class':'_2mylT6'})
product_name = product_name.text if product_name else "N/A"
product_offer_price = job.find('div',{'class':'_1vC4OE'})
product_offer_price = product_offer_price.text if product_offer_price else "N/A"
product_mrp = job.find('div',{'class':'_3auQ3N'})
product_mrp = product_mrp.text if product_mrp else "N/A"
product_link = job.find('a',{'class':'_3dqZjq'})
product_link = product_link.get('href') if product_link else "N/A"
product_link = url+ product_link
product_img = job.find('div',{'class':'_3ZJShS _31bMyl'})
print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
format(product_name,product_offer_price,product_mrp,product_link,product_img))
print('\n')
OUTPUT:
product name Aviator Sunglasses (58)
product offer price ₹4,760
product mrp ₹5,290
product link https://www.flipkart.com/search?q=rayban/ray-ban-aviator-sunglasses/p/itmf3yh25kzjsapz?pid=SGLDPXDY4CMRNEY9&lid=LSTSGLDPXDY4CMRNEY9TNASSX&marketplace=FLIPKART&srno=s_1_1&otracker=search&fm=organic&iid=5ed8f60d-e315-4440-bfdc-76049d80e5da.SGLDPXDY4CMRNEY9.SEARCH&qH=95c0daefc80c4a70
product image <div class="_3ZJShS _31bMyl" style="padding-top:120.00%"><img alt="" class="_3togXc" src="https://rukminim1.flixcart.com/image/454/545/sunglass/e/y/9/0rb3129iw0228-rayban-58-original-imadqb2nzmwzfup6.jpeg?q=50"/></div>
.
.
.
来源:https://stackoverflow.com/questions/57287726/not-able-to-scrap-the-images-from-flipkart-com-website-the-src-attribute-is-comi