Scrape website with R by navigating doPostBack

后端 未结 1 1349
梦毁少年i
梦毁少年i 2021-01-13 10:35

I want to extract a table periodicaly from below site.

price list changes when clicked building block names(BLOK 16 A, BLOK 16 B, BLOK 16 C, ...) . URL doesn\'t cha

相关标签:
1条回答
  • 2021-01-13 11:14
    library(rvest)    
    url<-"http://www.kentkonut.com.tr/tr/modul/projeler/daire_fiyatlari.aspx?id=44"
        pgsession<-html_session(url) 
        t<-html_table(html_nodes(read_html(pgsession), css = "#ctl00_ContentPlaceHolder1_DataList1"), fill= TRUE)[[1]]
        even_indices<-seq(2,length(t$X1),2)
        t<-t[even_indices,]
        t<-t[2:(length(t$X1)),]
    

    EDITED CODE:

    library(rvest)    
    url<-"http://www.kentkonut.com.tr/tr/modul/projeler/daire_fiyatlari.aspx?id=44"
    pgsession<-html_session(url)
    pgform<-html_form(pgsession)[[1]]
    page<-rvest:::request_POST(pgsession,"http://www.kentkonut.com.tr/tr/modul/projeler/daire_fiyatlari.aspx?id=44",
                               body=list(
                                 `__VIEWSTATE`=pgform$fields$`__VIEWSTATE`$value,
                                 `__EVENTTARGET`="ctl00$ContentPlaceHolder1$DataList2$ctl01$lnk_blok",
                                 `__EVENTARGUMENT`="",
                                 `__VIEWSTATEGENERATOR`=pgform$fields$`__VIEWSTATEGENERATOR`$value,
                                 `__VIEWSTATEENCRYPTED`=pgform$fields$`__VIEWSTATEENCRYPTED`$value,
                                 `__EVENTVALIDATION`=pgform$fields$`__EVENTVALIDATION`$value
                               ),
                               encode="form"
                               )
    # in the above example change eventtarget as "ctl00$ContentPlaceHolder1$DataList2$ctl02$lnk_blok" to get different table
    
    t<-html_table(html_nodes(read_html(page), css = "#ctl00_ContentPlaceHolder1_DataList1"), fill= TRUE)[[1]]
    even_indices<-seq(2,length(t$X1),2)
    t<-t[even_indices,]
    t<-t[2:(length(t$X1)),]
    
    0 讨论(0)
提交回复
热议问题