使用JavaScript Cheerio进行网页抓取图像

时间:2020-04-03 11:36:22

标签: javascript node.js web-scraping cheerio

当尝试从亚马逊网络抓取图像时收到错误。目的是下载书籍封面。

(node:26904) UnhandledPromiseRejectionWarning: Error: undefined is not a valid uri or options object.
    at request (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\request\index.js:44:11)
    at PromisseHandle._RejectOrResolve (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\node-image-downloader\src\image-downloader.js:86:5)
    at new Promise (<anonymous>)
    at ImageDownloader (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\node-image-downloader\src\image-downloader.js:98:26)
    at Request._callback (C:\Users\Isaac\Desktop\webscrape_with_cheerio\index.js:22:13)
    at Request.self.callback (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\request\request.js:185:22)
    at Request.emit (events.js:210:5)
    at Request.<anonymous> (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\request\request.js:1154:10)
    at Request.emit (events.js:210:5)
    at IncomingMessage.<anonymous> (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\request\request.js:1076:12)
(node:26904) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:26904) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with 
a non-zero exit code.

到目前为止,这是我的代码。我相信我的错误源于这一行;

var imagesrc = $(“。a行中心对齐点击式img”)。attr('src')

我可能需要通过书籍网址访问Class,尽管我可能错了。

const express = require('express');
const cheerio = require('cheerio');
const download = require('node-image-downloader');
const request = require('request');

const app = express();

app.get('/', (req, res) => {
    var url = "https://www.amazon.com/Black-Swan-Improbable-Robustness-Fragility/dp/081297381X"

    // makeing a request
    request(url,(error, response,html) => {
        if (!error){
            //console.log(html);
            var $ = cheerio.load(html)

            var imagesrc = $(".a-row center-align litb-on-click img").attr('src')

            //download the image

            download({
                imgs: [
                    {
                     uri:imagesrc                       
                    }
                ],
                dest:'./downloads'
            })
                .then((info) => {
                    console.log("Download Complete")
                    process.exit(1)
            })
        }
    })
})
app.listen(5000)

1 个答案:

答案 0 :(得分:0)

这里行点击中心对齐的litb是类

要选择一个类,我们需要使用class-selector。 (点)

请尝试如下更新您的imagesrc变量

var imagesrc = $(".a-row.center-align.litb-on-click img").attr('src');

空格不是强制性的。