Question

我对R来说相对较新（和R一起打屁股的新品牌），如果我忽视了一些明显的东西，请提前道歉！

按照本教程，我一直在努力学习如何使用RSelenium：https://rawgit.com/petrkeil/Blog/master/2017_08_15_Web_scraping/web_scraping.html#advanced-scraping-with-rselenium

在终端中运行以下命令后（docker运行-d -p 4445：4444 selenium / standalone-firefox），我尝试运行下面的R代码，仅从上面的超链接教程中稍作修改：

get.tree <- function(genus, species) 
{
  # navigate to the page
  browser <- remoteDriver(port=4445L)
  browser$open(silent = T)

  browser$navigate("http://www.bgci.org/global_tree_search.php?sec=globaltreesearch")
  browser$refresh()

  # create r objects from the web search input and button elements

  genusElem <- browser$findElement(using = 'id', value = "genus-field")
  specElem <- browser$findElement(using = 'id', value = "species-field")
  buttonElem <- browser$fiendElement(using = 'class', value = "btn_ohoDO")

  # tell R to fill in the fields

  genusElem$sendKeysToElement(list(genus))
  specElem$sendKeysToElement(list(species))

  # tell R to click the search button

  buttonElem$clickElement()

  # get output

  out <- browser$findElement(using = "css", value = "td.cell_1O3UaG:nth-child(4)") # the country origin
  out <- out$getElementText()[[1]] # extract actual text string
  out <- strsplit(out, split = "; ")[[1]] # turns into character vector

  # close browser

  browser$close()

    return(out)
}

# Now let's try it:

get.tree("Abies", "alba")

但在完成所有这些后，我收到以下错误：

Selenium消息：无法解码来自牵线木偶的响应构建信息：   版本：＆＃39; 3.6.0＆＃39;，修订版：＆＃39; 6fbf3ec767＆＃39;，时间：   ＆＃39; 2017-09-27T16：15：40.131Z＆＃39;系统信息：主持人：＆＃39; d260fa60d69b＆＃39;，ip：   ＆＃39; 172.17.0.2＆＃39;，os.name：＆＃39; Linux＆＃39;，os.arch：＆＃39; amd64＆＃39;，os.version：   ＆＃39; 4.9.49-moby＆＃39;，java.version：＆＃39; 1.8.0_131＆＃39;驱动程序信息：driver.version：   未知

错误：摘要：UnknownError详细信息：未知的服务器端错误   处理命令时发生。类：   org.openqa.selenium.WebDriverException进一步的细节：运行   errorDetails方法

任何人都知道这意味着什么以及我哪里出错了？

非常感谢你的帮助！

Answer 1

利用它所做的XHR请求来检索内联结果并抛出RSelenium：

library(httr)
library(tidyverse)

get_tree <-  function(genus, species) {

  GET(
    url = sprintf("https://data.bgci.org/treesearch/genus/%s/species/%s", genus, species), 
    add_headers(
      Origin = "http://www.bgci.org", 
      Referer = "http://www.bgci.org/global_tree_search.php?sec=globaltreesearch"
    )
  ) -> res

  stop_for_status(res)

  matches <- content(res, flatten=TRUE)$results[[1]]

  flatten_df(matches[c("id", "taxon", "family", "author", "source", "problems", "distributionDone", "note", "wcsp")]) %>% 
    mutate(geo = list(map_chr(matches$TSGeolinks, "country"))) %>% 
    mutate(taxas = list(map_chr(matches$TSTaxas, "checkTaxon")))

}

xdf <- get_tree("Abies", "alba")

xdf
## # A tibble: 1 x 8
##      id      taxon   family author     source distributionDone        geo      taxas
##   <int>      <chr>    <chr>  <chr>      <chr>            <chr>     <list>     <list>
## 1 58373 Abies alba Pinaceae  Mill. WCSP Phans              yes <chr [21]> <chr [45]>

glimpse(xdf)
## Observations: 1
## Variables: 8
## $ id               <int> 58373
## $ taxon            <chr> "Abies alba"
## $ family           <chr> "Pinaceae"
## $ author           <chr> "Mill."
## $ source           <chr> "WCSP Phans"
## $ distributionDone <chr> "yes"
## $ geo              <list> [<"Albania", "Andorra", "Austria", "Bulgaria", "Croatia", "Czech Republic", "Fr...
## $ taxas            <list> [<"Abies abies", "Abies alba f. columnaris", "Abies alba f. compacta", "Abies a...

您很可能需要在某些时候修改get_tree()，但这比将Selenium或Splash或者phantomjs或Headless Chrome作为依赖项更好。

得到RSelenium错误：＆＃34;无法解读来自牵线木偶的响应＆＃34;

1 个答案: