Question

我开始用 puppeteer 学习 javascript 中的网页抓取。我发现了一个我喜欢的视频，它展示了木偶操作者，我正在尝试抓取与视频 (link) 相同的信息。该页面与视频略有不同，因此我使用了我认为正确的标签。

当我尝试查找 "h3" 标签时出现问题。该标签存在于 DOM 中，但我的代码拒绝承认它的存在，但在查找 "h2" 标签时工作“正常”。

我想知道的是为什么我的代码没有检索到它。

网页：https://marketingplatform.google.com/about/partners/find-a-partner?utm_source=marketingplatform.google.com&utm_medium=et&utm_campaign=marketingplatform.google.com%2Fabout%2Fpartners%2F

// normal things to launch it
const puppeteer = require("puppeteer");

(async() => {
   const browser = await puppeteer.launch();
   const page = await browser.newPage();
   const url = "https://marketingplatform.google.com/about/partners/find-a-partner?utm_source=marketingplatform.google.com&utm_medium=et&utm_campaign=marketingplatform.google.com%2Fabout%2Fpartners%2F";
   
   await page.goto(url);

   // here comes the problem

   //                                                  this doesn't work    v
   const h3 = await page.evaluate(() => document.querySelector("h3").textContent);
   console.log(h3); //the error is because it tries to get the text content of null meaning it didn't found "h3"

   //                                                  this DOES work      v
   const h2 = await page.evaluate(() => document.querySelector("h2").textContent);
   console.log(h2);
   

   //await browser.close();
})();

我知道 "h3" 存在。如果你能解释一下发生的事情，我将不胜感激，这样我就可以了解更多谢谢。

Answer 1

页面上不存在h3标头，我们需要在waitForSelector之前等待：

// normal things to launch it
const puppeteer = require("puppeteer");

(async() => {
   const browser = await puppeteer.launch();
   const page = await browser.newPage();
   const url = "https://marketingplatform.google.com/about/partners/find-a-partner?utm_source=marketingplatform.google.com&utm_medium=et&utm_campaign=marketingplatform.google.com%2Fabout%2Fpartners%2F";
   
   await page.goto(url);

   await page.waitForSelector('h3')
   const h3 = await page.evaluate(() => document.querySelector("h3").textContent);
   console.log(h3);
   const h2 = await page.evaluate(() => document.querySelector("h2").textContent);
   console.log(h2);
   
   await browser.close(); // don't forget close it.
})();

输出为：

Viden
 Find your perfect match.

使用 puppeteer 进行网页抓取找不到 CSS 标签

1 个答案: