如何使用gocolly编写此等效代码

时间:2019-06-21 13:30:15

标签: go web web-scraping

我尝试使用https://github.com/gocolly/colly.,我尝试访问URL并将完整响应保存在本地光盘上,例如访问google.com,并保存URL google.html的完整响应正文。

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="bn"><head><meta content="text/html; charset=UTF-8" http-;google.pmc=JSON.parse(pmc);})();</script>        </body></html>

func main() {
    response, err := http.Get("https://www.google.com")
    if err != nil {
        log.Fatal(err)
    }
    defer response.Body.Close()

    outFile, err := os.Create("google.html")
    if err != nil {
        log.Fatal(err)
    }
    defer outFile.Close()

    // Copy data from HTTP response to file
    _, err = io.Copy(outFile, response.Body)
    if err != nil {
        log.Fatal(err)
    }
}

我的预期输出:

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="bn"><head><meta content="text/html; charset=UTF-8" http-;google.pmc=JSON.parse(pmc);})();</script>        </body></html>

0 个答案:

没有答案