Question

如何将page.evaluate的数据推送到一个csv文件中以便我进行分析？

谢谢。

我正在做一些网络爬虫，循环运行页面数并提取评论选项描述。

但是，尽管我能够console.log结果，但是我似乎无法将它们结合在一起或以csv文件格式获得它们。有人可以帮忙吗？

感谢刚刚开始学习编码，还有很多东西要学习.....

puppeteer = require('puppeteer');
const CREDS = require('./creds');


(async function main() {
    try{
        const URL = 'https://www.qoo10.sg/gmkt.inc/Login/Login.aspx'; 

        const browser = await puppeteer.launch({ headless: false });
        const page = await browser.newPage();


        await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36')

        await page.goto(URL ,{waitUntil:'networkidle0'});


        const USERNAME_SELECTOR = '#login_id';
        const PASSWORD_SELECTOR = '#passwd';
        const BUTTON_SELECTOR = '#dv_member_login > fieldset > a';

        await page.click(USERNAME_SELECTOR);
        await page.keyboard.type(CREDS.username);

        await page.click(PASSWORD_SELECTOR);
        await page.keyboard.type(CREDS.password);

        await page.click(BUTTON_SELECTOR);

        await page.waitForNavigation({waitUntil:'networkidle0'});

        const itemUrl = 'https://www.qoo10.sg/item/FANCO-FANCO-CEILING-FAN-FFM2000-FFM3000-FFM4000-FFM6000-LIGHT-KITS/496619203?banner_no=1305330';

        await page.goto(itemUrl, {waitUntil:'networkidle0'});

        const photoreviewsnumber = await page.evaluate(() => document.querySelector('#divPremiumList > div.rv_tit.photo > h2 > span').innerText);
        const photoreviewspage = Math.ceil(photoreviewsnumber/5);

            for (let i = 0; i < photoreviewspage; i++) {
            const button = await page.$('#photo_review_paging > a.next');
            button.click();



            const options = await page.evaluate(() => 
                Array.from(document.querySelectorAll('p.dtl')).map(partner => partner.innerText)
            );

            await page.waitFor(500);

            console.log(result);
            console.log(i);
        }

        await browser.close();

    } catch(e) {
        console.log('our error', e);
    }
})();

Answer 1

我不确定您遇到什么问题，但是也许这些更改会有所帮助？

    const allOptions = [];
    for (let i = 0; i < photoreviewspage; i++) {
      const button = await page.$('#photo_review_paging > a.next');
      await button.click();
      await page.waitFor(500);

      const options = await page.evaluate(() =>
        Array.from(document.querySelectorAll('p.dtl')).map(partner => partner.innerText)
      );

      console.log(options);
      allOptions.push(options);
      console.log(i);
    }
    console.log(allOptions.join('\n'));

否则，请添加更多详细信息，您可以获得什么数据与应该获得什么数据。

如何将结果从Puppeteer page.evaluate推送到csv文件？

1 个答案: