R rvest retrieve empty table

问题

I'm trying two strategies to get data from a web table:

library(tidyverse)  
library(rvest)                        

webpage <- read_html('https://markets.cboe.com/us/equities/market_statistics/book/')
data <- html_table(webpage, fill=TRUE)
data[[2]]

''

library("httr")
library("XML")

URL <- 'https://markets.cboe.com/us/equities/market_statistics/book/'
temp <- tempfile(fileext = ".html")
GET(url = URL, user_agent("Mozilla/5.0"), write_disk(temp))

df <- readHTMLTable(temp)
df <- df[[2]]

Both of them are returning an empty table.

回答1:

Values are retrieved dynamically from another endpoint you can find in the network tab when refreshing your url. You need to add a referer header for the server to return json containing the table data.

library(httr)

headers = c('Referer'='https://markets.cboe.com/us/equities/market_statistics/book/')
d <- content(httr::GET('https://markets.cboe.com/json/bzx/book/FIT', httr::add_headers(.headers=headers)))
print(d$data)

来源：https://stackoverflow.com/questions/58684947/r-rvest-retrieve-empty-table