Web scraping with beautifulsoup not finding anything

问题

I'm trying to scrape coinmarketcap.com just to get an update of a certain currency price, also just to learn how to web scrape. I'm still a beginner and can't figure out where I'm going wrong, because whenever I try to run it, it just tells me there are none. Although I know that line does exist. Any help is appreciated!

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
price = soup.find('data-currency-price data-usd=')
print (price)

回答1:

You can use the class attribute to get the value.

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
price = soup.find('span' ,attrs={"class" : "h2 text-semi-bold details-panel-item--price__value"})
print (price.text)

Output :

0.006778

回答2:

If you are going to be doing alot of this consider doing a single call using the official API and get all the prices. Then extract what you want. The following is from the site with an amendment by me to show the desired value for electroneum. The API guidance shows how to retrieve one at a time as well, though that requires a higher plan than the basic.

from requests import Request, Session
from requests.exceptions import ConnectionError, Timeout, TooManyRedirects
import json

url = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest'
parameters = {
      'start': '1',
      'limit': '5000',
      'convert': 'USD',
  }
headers = {
      'Accepts': 'application/json',
      'X-CMC_PRO_API_KEY': 'yourKey',
  }

session = Session()
session.headers.update(headers)

try:
    response = session.get(url, params=parameters)
    # print(response.text)
    data = json.loads(response.text)
    print(data['data'][64]['quote']['USD']['price'])
except (ConnectionError, Timeout, TooManyRedirects) as e:
    print(e)

You can always deploy a loop and check against a desired list e.g.

interested = ['Electroneum','Ethereum']
for item in data['data']:
    if item['name'] in interested:
        print(item)

For your current example:

You can use an attribute selector for data-currency-value

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
soup.select_one('[data-currency-value]').text

回答3:

You can get the value like this:

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
price = soup.find("span", id="quote_price").get('data-usd')
print (price)

回答4:

You should try to be more specific in how you want to FIND the item.

you currently are using soup.find('') I am not sure what you have put inside this as you wrote data-currency-price data-usd= Is that an ID a class name?

why not try finding the item using an ID.

soup.find(id="link3")

or find by tag

soup.find("relevant tag name like div or a")

or something like this

find_this = soup.find("a", id="ID HERE")

回答5:

import requests
from bs4 import BeautifulSoup

url = 'https://coinmarketcap.com/currencies/electroneum/'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, 'html.parser')
x=soup(id="quote_price").text
print (x)

Look for ID better,or search through soup.find_all(text="data-currency-price data-usd")[1].text

来源：https://stackoverflow.com/questions/54931173/web-scraping-with-beautifulsoup-not-finding-anything

标签

python

web-scraping

beautifulsoup

cryptocurrency