Question

我正在使用cheerio从确实（至少50个职位）中抓取职位信息。

我已经使用cheerio获得了抓取的数据。但是我不知道如何使用async / await方法将这些数据存储在对象中

工具：node.js，cheerio，请求。

const cheerio = require('cheerio');
const request = require('request');

const fetchIndeedData = async () => {
    let start = 0;
    let title = [];
    let company = [];
    let location = [];
    let summary = [];

    for (let i = 0; i < 5; i++) {
        let url = `https://ca.indeed.com/jobs?q=full+stack+developer&l=Toronto,+ON&start=${start}`;
        await request(url, (err, response, body) => {
            const $ = cheerio.load(body);

            $('#resultsCol .jobsearch-SerpJobCard .title a').each((i, item) => {
                title.push(item.attribs.title);
            });

            $('#resultsCol .jobsearch-SerpJobCard .company a').each((i, item) => {
                company.push($(item).text().trim());
            });

            $('#resultsCol .jobsearch-SerpJobCard .location').each((i, item) => {
                location.push($(item).text());
            });

            $('#resultsCol .jobsearch-SerpJobCard .summary ').each((i, item) => {
                summary.push($(item).text());
            });

        });
        start += 10;
    }

    const jobPostings = {
        title,
        location,
        company,
        summary
    };

    return jobPostings;
};

const getData = fetchIndeedData().then(data => console.log(data));

调用getData函数时无法获取任何数据。当我在返回之前运行console.log（jobPostings）时。我仍然无法获取数据。有人知道吗？

Answer 1

这就是我认为使用异步/等待（you will need request-promise or similar）时的样子：

const fetchIndeedData = async () => {
  let urls = [0,1,2,3,4].map(start => `https://ca.indeed.com/jobs?q=full+stack+developer&l=Toronto,+ON&start=${start}`)
  let responses = await Promise.all(urls.map(url => request.get(url)))
  let $$ = responses.map(response => cheerio.load(response))
  return $$.map($ => {
    return {
      title: $('title').text(),
      // more cheerio code here
    }
  })
}

;(async function(){
  const data = await fetchIndeedData()
  console.log(data)
})()

如何使用async / await方法获取函数返回

1 个答案: