Question

我正试图从http://stats.nba.com/team/#!/1610612742/中删除名册数据。到目前为止，我已经尝试过RCurl和XML包，我试过的代码如下：

library(RCurl)
library(XML)
webpage <- getURL("http://stats.nba.com/team/#!/1610612742/")
webpage <- readLines(tc <- textConnection(webpage));
pagetree <- htmlTreeParse(webpage, useInternalNodes = TRUE)
x <- unlist(xpathApply(pagetree,"//*nba-stat-table_overflow/player",xmlValue))
Content <- gsub(pattern = "([\t\n])",
            replacement = " ", x = x, ignore.case = TRUE)

我认为我的xpathApply函数格式错误。我应该用什么路径来到名册表？

Answer 1

你可以这样做：

require(rvest)
require(httr)
require(purrr)


ses <- html_session("http://stats.nba.com/team/#!/1610612742/",
                    user_agent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"))
doc <- ses %>% jump_to("http://stats.nba.com/stats/commonteamroster?LeagueID=00&Season=2016-17&TeamID=1610612742")
res <- content(doc$response, "parsed")

res$resultSets[[1]]$rowSet %>% 
  map_df(~as.data.frame(t(.)))

#           V1   V2 V3                  V4 V5  V6   V7  V8           V9 V10 V11               V12     V13
#1  1610612742 2016 00     Justin Anderson  1 G-F  6-6 228 NOV 19, 1993  23   1          Virginia 1626147
#2  1610612742 2016 00          J.J. Barea  5   G  6-0 185 JUN 26, 1984  32  10      Northeastern  200826
#3  1610612742 2016 00        Andrew Bogut  6   C  7-0 260 NOV 28, 1984  32  11              Utah  101106

res$resultSets[[2]]$rowSet %>% 
  map_df(~as.data.frame(t(.)))

#          V1   V2        V3      V4        V5                V6                V7 V8                                     V9
#1 1610612742 2016 CAR107961    Rick  Carlisle     Rick Carlisle     rick_carlisle  1                             Head Coach
#2 1610612742 2016 HUN524472  Melvin      Hunt       Melvin Hunt       melvin_hunt  2                        Assistant Coach
#3 1610612742 2016 CAN081621   Kaleb   Canales     Kaleb Canales     kaleb_canales  2                        Assistant Coach

我是如何找到这个的：

我检查了网站发出的所有XHR电话并发现它需要一个会话（这就是为什么我使用html_session创建一个）并设置一个用户代理（不确定这是真的需要......）没有UA我的请求被卡住了> 30秒..

刮掉NBA.com的数据

1 个答案: