我正在尝试抓取数据,但是在抓取数据时遇到了麻烦。我可以使用RSelenium浏览网站。您可以在下面找到我的代码。我想从每个下拉列表中抓取名称,以便将其存储在对象中并运行循环。
library(RSelenium)
library(rvest)
library(XML)
library(RCurl)
rd<-rsDriver()
remDr<-rd[["client"]]
url<-"https://kvk.icar.gov.in/facilities_list.aspx"
jsScript <- "var element = arguments[0]; return element.outerHTML;"
webpage<-read_html(url)
remDr$navigate("https://kvk.icar.gov.in/facilities_list.aspx")
remDr$refresh()
#First drop down
stateEle<-remDr$findElement("id", "ContentPlaceHolder1_ddlState")
#webElem <- remDr$findElement("id", "ContentPlaceHolder1_ddlDistrict")
stateHTML <- remDr$executeScript(jsScript, list(stateEle))[[1]]
statedoc <- htmlParse(appHTML)
states<-doc["//option", fun = function(x) xmlGetAttr(x, "name")]
stateEle$clickElement()
stateEle$sendKeysToElement(states[[30]])
stateEle$clickElement()
#Second drop down
distEle<-remDr$findElement("id", "ContentPlaceHolder1_ddlDistrict")
distHTML <- remDr$executeScript(jsScript, list(distEle))[[1]]
distdoc <- htmlParse(appHTML)
districts<-doc["//option", fun = function(x) xmlGetAttr(x, "value")]
distEle$clickElement()
distEle$sendKeysToElement(list(distdoc[[2]]))
distEle$clickElement()
#Third drop down
kvkEle<-remDr$findElement("id", "ContentPlaceHolder1_ddlKvk")
appHTML <- remDr$executeScript(jsScript, list(kvkEle))[[1]]
kvkdoc <- htmlParse(appHTML)
kvk<-doc["//option", fun = function(x) xmlGetAttr(x, "value")]
kvkEle$clickElement()
kvkEle$sendKeysToElement(list(kvk[[2]]))
kvkEle$clickElement()
#submitting the values
submitEle<-remDr$findElement("id", "ContentPlaceHolder1_btnSubmit")
submitEle$clickElement()
我也想将结果刮入数据框。
答案 0 :(得分:0)
使用您的代码
stateEle<-remDr$findElement("id", "ContentPlaceHolder1_ddlState")
从这里开始,如果要获取所有值进行循环,请使用:
library(magrittr)
stateEle$getElementText()[[1]] %>% strsplit(., '\\n')
这将提供文本元素列表,您可以在其中进一步删除"--Select--"
选项:
stateEle$getElementText()[[1]] %>% strsplit(., '\\n') %>% unlist %>% setdiff(., '--Select--')
对所有其他选择列表重复此操作。