刮擦并将其作为json在快递中返回

时间:2016-09-15 18:38:07

标签: javascript json cheerio

我正在从网站上抓取数据然后我想在浏览器中将其显示为json,但是即使我console.log()配方数组它显示数据,但它不会向浏览器发送任何内容怎么没有在浏览器中显示json数组?

router.get('/scrape', function(req, res, next) {

  res.json(scrapeUrl("http://url"));

})

function scrapeUrl(url) {
  request(url, function(error, response, html){

    // First we'll check to make sure no errors occurred when making the request
    if(!error){

      var $ = cheerio.load(html);
      var recipes = [];



      $('div.article-block a.picture').each(function(i, elem) {

        var deepUrl = $(this).attr('href');

        if(!$(this).attr('href').indexOf("tema") > -1) {
          request(deepUrl, function(error, response, html){

            // First we'll check to make sure no errors occurred when making the request
            if(!error){


              var $ = cheerio.load(html);

              var image = $('div.article div.article-main-pic img').attr('src');
              var title = $('div.recipe h2.fn').text();

              var object = {url: deepUrl, title : title, image : image};

              recipes.push(object);

            }

          });

        }

      });
      return recipes;
    }
  });
}

2 个答案:

答案 0 :(得分:0)

你ScrapeURl是异步功能。

所以要修复它就像这样

router.get('/scrape', function(req, res, next) { scrapeUrl("http://url", function(err, resp){ res.json(resp); }); });

在scrapeUrl中添加回调

scrapeUrl (url, cb) {
  request(url, function(error, response, html){
    //rest of your code
    //Make sure callback are called after last request return results.
    cb(err, resp)
  });
}

希望这有帮助。

答案 1 :(得分:0)

您的函数只返回数组,而不是将该数组放在DOM中。

如果您只想在浏览器中显示数据数组,可以创建一个容器,将内容放入:

<div id="content"></div>

然后用以下代码替换你的返回值:

$('#content').text(JSON.stringify(recipes));