Webscraping with urllib

问题

I am looking to get some information off the CME website Namely I want to get the Futures Yield and the Futures DV01 for the 10y Treasury Note Future. Found this little snippet on an old thread:

import urllib.request
class AppURLopener(urllib.request.FancyURLopener):
    version = "Mozilla/5.0"
opener = AppURLopener()
fh = opener.open('http://www.cmegroup.com/tools-information/quikstrike/treasury-analytics.html')

It throws a deprecation warning and I am not quite sure how I get the info from the website. Can someone enlighten me please what the new syntax should be and and how to get the info. Thanks

回答1:

Run the script when you are done installing selenium.

from selenium import webdriver ; from bs4 import BeautifulSoup

driver = webdriver.Chrome()
driver.get("http://www.cmegroup.com/tools-information/quikstrike/treasury-analytics.html")

driver.switch_to_frame(driver.find_element_by_tag_name("iframe"))
soup = BeautifulSoup(driver.page_source, 'html.parser')
driver.quit()

table = soup.select('table.grid')[0]
list_of_rows = [[t_data.text for t_data in item.select('th,td')]
                for item in table.select('tr')]

for data in list_of_rows:
    print(data)

I think, this is the table [partial picture] you are after:

来源：https://stackoverflow.com/questions/45936918/webscraping-with-urllib