问题
since a week ago i have been trying to scrape a table from this site https://www.bi.go.id/id/moneter/informasi-kurs/transaksi-bi/Default.aspx but i dont have an idea what to write,i am very confused. iam trying to scrape table of kurs transaction from 2015-2020(20 nov 2015-20 nov 2020, but the problem is the link between the default date and the date that I chose is still the same.please help me in any way,Thank you before !
import requests
from bs4 import BeautifulSoup
import pandas as pd
headers={
"User-Agent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.101 Safari/537.36",
"X-Requested-With":"XMLHttpRequest"
}
url = "https://www.bi.go.id/id/moneter/informasi-kurs/transaksi-bi/Default.aspx"
import requests
from lxml import html
response = requests.get(url)
content= response.content
print(content)
回答1:
You need to use Selenium. You can install Selenium and then you can install a driver. I use chrome and then once you install it make a note of that path and set your DRIVER_PATH to the location
In the code below what i do is basically request the link you posted and then I enter the dates which you can change. Finally i click on the submit button. That generates the table within the date range. Now you can write follow up code to scrape the information from the table.
Code
import requests
from selenium import webdriver
DRIVER_PATH = 'Yourpath/chromedriver'
driver = webdriver.Chrome(executable_path=DRIVER_PATH)
driver.get('https://www.bi.go.id/id/moneter/informasi-kurs/transaksi-bi/Default.aspx')
start_date = driver.find_element_by_id("ctl00_PlaceHolderMain_biWebKursTransaksiBI_txtFrom")
start_date.send_keys("15-Nov-20")
end_date = driver.find_element_by_id("ctl00_PlaceHolderMain_biWebKursTransaksiBI_txtTo")
end_date.send_keys("20-Nov-20")
submit_button = driver.find_element_by_id("ctl00_PlaceHolderMain_biWebKursTransaksiBI_btnSearch1").click()
来源：https://stackoverflow.com/questions/65056697/scraping-table-with-python-based-on-dates