Question

我创建了一个页面抓取功能，可以抓取一些数据。我希望能够创建URL列表，以便可以在函数调用中传递多个参数，以构建不同的URL。有没有办法使用httr::modify_url来做到这一点？

我创建一个URL的代码如下：

library(tidyverse)
#> Registered S3 methods overwritten by 'ggplot2':
#>   method         from 
#>   [.quosures     rlang
#>   c.quosures     rlang
#>   print.quosures rlang
library(httr)

# Arguments for Function
hand = NULL
prp = "P"
month = NULL
year = 2019
pitch_type = "FA"
report_type = "pfx"
lim = 0

url <- httr::modify_url("https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php",
                        query = list(
                          hand = hand,
                          reportType = report_type,
                          prp = prp,
                          month = month,
                          year = year,
                          pitch = pitch_type,
                          ds = "velo",
                          lim = lim
                        ))

# Single Query Result
url
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=FA&ds=velo&lim=0"

我想知道我是否可以使用上面的httr::modify_url查询和purrr::reduce(paste0)的某种组合来为其他参数创建URL：

# Requested Query
pitch_type = c("FA", "SI")
report_type = c("pfx", "outcome")

# URL Generating Function for User inputs
generate_urls <- function(hand = NULL, report_type = c("pfx", "outcome"), prp = "P", month = NULL, year = NULL, pitch_type = c("FA", "SI"), lim = 0) {
# Not sure of what to put in function for modify_url call
}





# Result 
"https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=FA&ds=velo&lim=0"
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=FA&ds=velo&lim=0"
"https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=SI&ds=velo&lim=0"
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=SI&ds=velo&lim=0"
"https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=outcome&prp=P&year=2019&pitch=FA&ds=velo&lim=0"
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=outcome&prp=P&year=2019&pitch=FA&ds=velo&lim=0"
"https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=outcome&prp=P&year=2019&pitch=SI&ds=velo&lim=0"
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=outcome&prp=P&year=2019&pitch=SI&ds=velo&lim=0"

Answer 1

这是使用dydyverse函数的选项。首先，我们可以定义要遍历的参数空间

params <- list(
  hand = NULL,
  prp = "P",
  year = 2019,
  month = NULL,
  pitch_type = c("FA", "SI"),
  report_type = c("pfx", "outcome"),
  lim = 0
)

然后我们可以使用

获取所有URL

library(tidyverse) # tidyr for crossing(); purrr for pmap(), map_chr()
library(httr)  
baseurl <- "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php"
crossing(!!!params) %>% 
  pmap(list) %>% 
  map_chr( ~modify_url(baseurl, query=.x) )

crossing()负责获取所有可能的参数组合。然后pmap(list)将小标题的每一行都转换成自己的列表（这是我们需要传递给query=的{{1}}参数的条件。最后，我们调用url生成函数每组参数并返回一个字符串。

使用modify_url构建多个URL

1 个答案: