在foreach()内部,CURL句柄变为陈旧

时间:2015-04-29 14:56:01

标签: r

好吧,所以我最近发现我可以在登录屏幕后面查询网站以查找CSV报告。然后我想,同时这样做会不会更好?毕竟有些报告要比其他报告花费更长的时间,如果我一次查询10个不同的报告会更有效率。所以我现在两次在头脑中玩弄HTTPS协议,现在又进行并行处理。我认为我的frankencode几乎就在那里,但它给了我一个

"Error in ( : task 1 failed - "Stale CURL handle being passed to libcurl"

请注意" curl"是非常流行的" html"变量确实登录成功。在它的并行块中发生了一些事情,使它变得陈旧。

library(RCurl)
library(doParallel)
registerDoParallel(cores=4)

agent="Firefox/23.0" 

options(RCurlOptions = list(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl")))
curl = getCurlHandle()
curlSetOpt(
  cookiejar = 'cookies.txt' ,
  useragent = agent,
  followlocation = TRUE ,
  autoreferer = TRUE ,
  curl = curl
)
un="username@domain.com"
pw="password"

html = postForm(paste("https://login.salesforce.com/?un=", un, "&pw=", pw, sep=""), curl=curl)


urls = c("https://xyz123.salesforce.com/00O400000046ayd?export=1&enc=UTF-8&xf=csv", 
         "https://xyz123.salesforce.com/00O400000045sWu?export=1&enc=UTF-8&xf=csv", 
         "https://xyz123.salesforce.com/00O400000045z3Q?export=1&enc=UTF-8&xf=csv")

x <- foreach(i=1:4, .combine=rbind, .packages=c("RCurl")) %dopar% {
  xxx <- getURL(urls[i], curl=curl)
}

0 个答案:

没有答案