我正在用Go编写一个爬虫,该爬虫对Trendmicro网站进行爬网以获取安全报告(以pdf形式),然后当它在pdf上被点击时使用wget下载pdf。之后,我使用从github(https://github.com/sajari/docconv)中找到的库将pdf转换为文本,然后将该文本打印到终端。出于某种原因,该代码对于前两个pdf来说运行良好,但是一旦到达第三个pdf,则表示未找到pdf,即使它下载的pdf相同,并且位于同一目录中。程序跟踪如下:
HTTP request sent, awaiting response... 200 OK
Length: 2268867 (2.2M) [application/pdf]
Saving to: ‘rpt-2016-annual-security-roundup-a-record-year-for-enterprise-threats.pdf’
rpt-2016-annual-sec 100%[===================>] 2.16M 1.02MB/s in 2.1s
2019-02-20 15:25:07 (1.02 MB/s) - ‘rpt-2016-annual-security-roundup-a-record-year-for-enterprise-threats.pdf’ saved [2268867/2268867]
2019/02/20 15:25:07 open rpt-2016-annual-security-roundup-a-record-year-for-enterprise-threats.pdf : no such file or directory
exit status 1
我比较了两个字符串,看它们是否不同,但没有不同。再说一次,前两个下载就很好。这是代码的受影响部分:
pdfName := strings.Replace(link, "https://documents.trendmicro.com/assets/rpt/", "", -1)
fmt.Printf("PDF name: %s\n", pdfName)
existingPdf := 0;
if len(pdfs) > 0 {
for i := 0; i < len(pdfs); i++ {
if pdfs[i] == pdfName {
existingPdf = 1;
}
}
}
if existingPdf == 0 {
pdfs = append(pdfs, pdfName)
command := "wget " + link
cmd := exec.Command("/bin/bash", "-c", command, ">", "/dev/null", "2>&1")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Run()
res, err := docconv.ConvertPath(pdfName)
if err != nil {
log.Fatal(err)
}
fmt.Println(res)
}
编辑:如果您想要完整的代码和/或想要运行它,我已将其放入pastebin(https://pastebin.com/5JkJDamV)