使用SelectorGadget的网页抓取页面

Question

我在这里的网页具有简单元素。我只想通过Web抓取排在后面的数据的下部（站信息和探测索引）。但是，我的SelectorGadget无法正确处理所需的数据，它一直显示数据的上部。该如何改善？

install.packages("rvest")
library(rvest)
sounding <- html("http://weather.uwyo.edu/cgi-bin/sounding?region=seasia&TYPE=TEXT%3ALIST&YEAR=2019&MONTH=01&FROM=2012&TO=2012&STNM=48615")
sounding %>%html_node("h3,pre") %>%html_text()

Answer 1

原始答案

您想要一个不同的选择器。确保消除了您不想抓取的所有其他“黄色”选择：

library(rvest)
#> Loading required package: xml2
sounding <- read_html("http://weather.uwyo.edu/cgi-bin/sounding?region=seasia&TYPE=TEXT%3ALIST&YEAR=2019&MONTH=01&FROM=2012&TO=2012&STNM=48615")

sounding %>%
  html_nodes("h3+ pre") %>% 
  html_text()
#> [1] "\n                         Station identifier: WMKC\n                             Station number: 48615\n                           Observation time: 190120/1200\n                           Station latitude: 6.16\n                          Station longitude: 102.28\n                          Station elevation: 5.0\n                            Showalter index: 1.26\n                               Lifted index: -2.86\n    LIFT computed using virtual temperature: -3.38\n                                SWEAT index: 187.99\n                                    K index: 14.40\n                         Cross totals index: 19.00\n                      Vertical totals index: 23.90\n                        Totals totals index: 42.90\n      Convective Available Potential Energy: 409.13\n             CAPE using virtual temperature: 595.76\n                      Convective Inhibition: -26.90\n             CINS using virtual temperature: -8.60\n                           Equilibrum Level: 228.72\n Equilibrum Level using virtual temperature: 226.79\n                   Level of Free Convection: 819.49\n             LFCT using virtual temperature: 871.25\n                     Bulk Richardson Number: 240.00\n          Bulk Richardson Number using CAPV: 349.48\n  Temp [K] of the Lifted Condensation Level: 294.55\nPres [hPa] of the Lifted Condensation Level: 938.33\n     Mean mixed layer potential temperature: 299.97\n              Mean mixed layer mixing ratio: 17.45\n              1000 hPa to 500 hPa thickness: 5782.00\nPrecipitable water [mm] for entire sounding: 46.56\n"

^{由reprex package（v0.2.1）于2019-01-20创建}

更新的答案

返回并清理成类似于网站上的表的表。我敢肯定它可以做得更优雅，但这足以使它成为表格格式。

library(rvest)
#> Loading required package: xml2
library(tidyverse)
sounding <- read_html("http://weather.uwyo.edu/cgi-bin/sounding?region=seasia&TYPE=TEXT%3ALIST&YEAR=2019&MONTH=01&FROM=2012&TO=2012&STNM=48615")

raw_dat <- sounding %>%
  html_nodes("h3+ pre") %>% 
  html_text()

raw_dat %>% 
  str_split(pattern = "\n", simplify = T) %>% 
  map_chr(str_squish) %>% 
  tibble(x = .) %>% 
  separate(x, into = c("Station", "Value"), sep = ": ") %>% 
  filter(!is.na(Value))
#> Warning: Expected 2 pieces. Missing pieces filled with `NA` in 2 rows [1,
#> 32].
#> # A tibble: 30 x 2
#>    Station                                 Value      
#>    <chr>                                   <chr>      
#>  1 Station identifier                      WMKC       
#>  2 Station number                          48615      
#>  3 Observation time                        190120/1200
#>  4 Station latitude                        6.16       
#>  5 Station longitude                       102.28     
#>  6 Station elevation                       5.0        
#>  7 Showalter index                         1.26       
#>  8 Lifted index                            -2.86      
#>  9 LIFT computed using virtual temperature -3.38      
#> 10 SWEAT index                             187.99     
#> # … with 20 more rows

^{由reprex package（v0.2.1）于2019-01-20创建}