问题
I am trying to scrape project URLs from Indiegogo, but I had no success after hours. I can not scrape them either using XPath or Beautifulsoup. The output of the following code does not contain the information I want:
soup.find_all("div")
Also, Beutifulsoup did not work:
import requests
from bs4 import BeautifulSoup
url = 'https://www.indiegogo.com/explore/all?project_type=campaign&project_timing=ending_soon&sort=trending'
page = requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
project_name_list = soup.find(class_='exploreDetail-campaigns row')
project_name_list_items = project_name_list.find_all('a')
print(project_name_list_items)
for project_name in project_name_list_items:
links = project_name.get('href')
print(links)
Actually, the following command returns nothing.
soup.find(class_='exploreDetail-campaigns row')
How should I know if a website is in javascript format or another format?
来源:https://stackoverflow.com/questions/61536072/how-to-scrape-project-urls-from-indiegogo-using-beautifulsoup