Question

我想在下面的页面中提取表格

https://www.mcxindia.com/market-data/spot-market-price

我尝试了rvest和RCurl，但在这两种情况下，下载的页面与我在浏览器中看到的页面不同。我假设有某种形式的重定向，我无法检测或遵循

任何帮助将不胜感激

PS：对phantomjs不感兴趣

这是我迄今为止所尝试过的：

1。 HTTR

base_url <- "https://www.mcxindia.com/market-data/spot-market-price"
ua       <- "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
library(httr)
library(XML)
doc <- POST(base_url,user_agent(ua),set_cookies(`_ga` = "GA1.2.543290785.1505100652",`_gid`="GA1.2.1409943545.1505881384",`_gat`="1"))
doc <- htmlParse(doc)
poptable<-readHTMLTable(doc,which=7)

结果：找不到数据!!!!

2。 RCurl

library(RCurl)
curl <- getCurlHandle()
curlSetOpt(curl = curl,
           ssl.verifypeer = FALSE,
           useragent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
           timeout = 60,
           followlocation = TRUE,
           cookiejar = "./cookies",
           cookiefile = "./cookies")
newDoc = getURL("https://www.mcxindia.com/market-data/spot-market-price", curl=curl)
newDoc <- htmlParse(newDoc)
poptable<-readHTMLTable(newDoc,which=7)

结果：找不到数据!!!!

另外，我有兴趣知道如何获取excel文件（请参阅小excel图标）

Answer 1

以下是答案

library(rvest)
library(stringi)
library(V8)

  ctx <- v8()
  pg <- read_html("https://www.mcxindia.com/market-data/spot-market-price")
  html_nodes(pg, xpath=".//script[contains(., 'Data')]")[[1]] %>% 
    html_text() %>% stri_unescape_unicode() %>% stri_replace_all_fixed('\\\\', '')%>% 
    ctx$eval() -> ignore_the_blank_return_value
  data <- ctx$get("vSMP")$Data[,c("Symbol","TodaysSpotPrice","Unit")]

享受!!!

使用rvest或RCurl或httr在R中抓一页

1 个答案: