使用Puppeteer在<h2>标签之间刮擦<p>标签

Question

我是木偶游戏的新手，正在学习抓取网页。该网页的结构如下：

我想做的是刮掉<p>和<h2> Status </h2>之间的所有<h2>Naam</h2>标签。使用当前代码，我可以在此页面上抓取所有<p>标签。直到现在，我才尝试抓取<p>之后到<h2> Status </h2>之后的所有<h2>Naam</h2>标签。

我当前的代码：

const puppeteer = require('puppeteer');

const plaatsengids = async (place) => {
    //Creates a Headless Browser Instance in the Background
    const browser = await puppeteer.launch();

    //Creates a Page Instance, similar to creating a new Tab
    const page = await browser.newPage();

    //Navigate the page to url
    await page.goto('https://plaatsengids.nl/'+place);

  /*  page.waitForSelector('.title').then(async function(){
        const title = await page.$eval('.title', element => element.innerHTML);
    })*/

    //Finds the first element with the id 'hplogo' and returns the source attribute of that element
    const Title = await page.$eval('.title', element => element.innerHTML);
    const description = await page.$eval('.body p', element => element.innerHTML);

let content = await page.evaluate(() => {
    
    let divs = [...document.querySelectorAll('.body p')];
    return divs.map((div) => div.textContent.replace("- ",""));
  });



    //Closes the Browser Instance
    await browser.close();
    return content;
};




module.exports = plaatsengids;

相关网页为： https://www.plaatsengids.nl/Stein

Answer 1

您可以使用Node.compareDocumentPosition()：

const puppeteer = require('puppeteer');

(async function main() {
  try {
    const browser = await puppeteer.launch();
    const [page] = await browser.pages();

    await page.goto('https://www.plaatsengids.nl/Stein');

    const paragraphs = await page.evaluate(() => {
      const status = document.querySelector('h2[name="status"]');
      const naam = document.querySelector('h2[name="naam"]');

      return [...document.querySelectorAll('p')]
        .filter(p => p.compareDocumentPosition(status) & Node.DOCUMENT_POSITION_PRECEDING &&
                     p.compareDocumentPosition(naam) & Node.DOCUMENT_POSITION_FOLLOWING)
        .map(p => p.innerText);
    });

    console.log(paragraphs);

    await browser.close();
  } catch (err) {
    console.error(err);
  }
})();