Question

我想在我的nodejs应用程序中刮一个HTML页面并形成一个head标签列表。例如：

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
    <link rel="stylesheet" href="style.css">
    <link rel="shortcut icon" href="favicon.ico" type="image/x-icon">
    <script src="script.src"></script>
</head>
<body>
    ...
</body>
</html>

所需的输出：

['<meta charset="UTF-8">','<meta name="viewport" content="width=device-width, initial-scale=1.0">','<title>Document</title>', ...etc]

但是我有点困惑，因为meta标签不会“关闭”，所以它需要的不仅仅是简单的正则表达式和拆分。我想使用DOMParser，但是我在节点环境中。我尝试使用xmldom npm软件包，但它只返回了换行符（\r\n）的列表。

Answer 1

使用request npm请求页面，然后在获得响应后，使用cheerio npm进行解析并从原始数据中获取所需内容。

注意：cheerio具有类似于jQuery的语法

 var request = require('request');
 var cheerio = require('cheerio')

app.get('/scrap',(req,res)=>{

request('---your website url to scrap here ---', function (error, response, body) {    
       var $ = cheerio.load(body.toString())
       let headContents=$('head').children().toString();
       console.log('headContents',headContents)
  });

});

Answer 2

一种选择是使用Cheerio解析HTML并从每个元素中提取所需的信息：

const cheerio = require('cheerio');
const htmlStr = `<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
    <link rel="stylesheet" href="style.css">
    <link rel="shortcut icon" href="favicon.ico" type="image/x-icon">
    <script src="script.src"></script>
</head>
<body>
    ...
</body>
</html>`;
const $ = cheerio.load(htmlStr);
const headTags = [];
$('head > *').each((_, elm) => {
  headTags.push({ name: elm.name, attribs: elm.attribs, text: $(elm).text() });
});
console.log(headTags);

输出：

[ { name: 'meta', attribs: { charset: 'UTF-8' }, text: '' },
  { name: 'meta',
    attribs:
     { name: 'viewport',
       content: 'width=device-width, initial-scale=1.0' },
    text: '' },
  { name: 'title', attribs: {}, text: 'Document' },
  { name: 'link',
    attribs: { rel: 'stylesheet', href: 'style.css' },
    text: '' },
  { name: 'link',
    attribs:
     { rel: 'shortcut icon',
       href: 'favicon.ico',
       type: 'image/x-icon' },
    text: '' },
  { name: 'script', attribs: { src: 'script.src' }, text: '' } ]

NodeJS-阅读HTML Head标记

2 个答案: