Question

I'm using nightmare.js to scrape webpage content.

After authenticating the nightmare instance, I loop through multiple pages (that require a login) and then call the run method that executes all the page loads.

From each page I want to grab specific content so I call the evaluate function which ensures we're executing inside the browser scope. Whatever is returned from the evaluate function becomes the argument in the run method

But I'm trying to run the evaluate function multiple times (once for each page). But the evaluate function can only return output to the run method once. I tried pushing content to a global variable (in the main scope) but can't access it from inside the browser scope.

Can anyone advise on how to have multiple evaluate methods run on one nightmare instance and extract information from each?

var Nightmare = require("nightmare");

//Creates the authenticated nightmare instance

var scraper = new Nightmare()
  .goto('https://www.example.com/signin')
  .type('#login', 'username')
  .type('#password', 'password')
  .click('#btn')
  .run(function(err, nightmare) {
    if (err) {
      console.log(err);
    }
    console.log('Done.');
  });

for (var i = 0; i < 4; i++) {
  scraper
    .goto('https://www.example.com/page'+i)
    .wait(1000)
    .evaluate(function(){
      return $('#result > h3').text()
    })
}

scraper.run(function(err, result) {
  console.log(result)
  if (err) {
    console.log(err);
  }
}); )

Answer 1

我对噩梦并不是特别了解，但看起来你可能只是出现异步问题，这在js中是一个难题。

好消息是，您可以重新构建代码并依赖生成器使其工作相当简单。

要实现的关键是，当您在函数名称之前使用*时，该函数将成为生成器函数，允许您使用yield关键字。只要你yield该代码行在转到下一行代码之前等待返回的promise完成，它就会返回那个产生的值。您可以使用vo库将生成器函数转换为回调函数，该函数返回所有生成结果的数组。

var Nightmare = require('../nightmare')
var vo = require('vo')

vo(run)(function(err, result) {
  if (err) throw err
  console.log('#result > h3: ', result)
})

function *run() {
  var nightmare = Nightmare();
  yield nightmare
    .goto('https://www.example.com/signin')
    .type('#login', 'username')
    .type('#password', 'password')
    .click('#btn')

  for (var i = 0; i < 4; i++) {
    yield nightmare
      .goto('https://www.example.com/page'+i)
      .wait(1000)
      .evaluate(function(){
        return $('#result > h3').text()
      })
  }

  yield nightmare.end()
}

How to have multiple evaluate methods run in nightmare.js?

1 个答案: