使用Beautifulsoup Python提取没有HTML标签的文本

时间:2016-08-09 19:49:05

标签: python web-scraping beautifulsoup

我尝试提取这部分文字,但我不知道该怎么做,我在本地使用了几个html文件。



<HTML><HEAD><STYLE>SOME STYLE CODE</STYLE></HEAD><META http-equiv=Content-Type content="text/html; charset=utf-8">
<BODY>
<H1>SOME TEXT I DONT WANT</H1>
THIS TEXT IS WHICH I WANT
<H1>ANOTHER TEXT I DONT WANT</H1>
ANOTHER TEXT THAT I WANT
[.. Continues ..]
</BODY></HTML>
&#13;
&#13;
&#13;

感谢您的帮助

编辑:我尝试过这段代码,但有时打印h1标签

import glob
from bs4 import BeautifulSoup
for file in glob.glob('Logs/Key*.html'):
    with open(file) as f:
        htmlfile = f.read()
        soup = BeautifulSoup(htmlfile, 'html.parser')
        c = soup.find('body').findAll()
        for i in c:
            print i.nextSibling
编辑2:其实问题是html文件只有一行,所以当我尝试用这个html运行该代码时,还会打印h1标签:

from bs4 import BeautifulSoup
htmlfile = '<HTML><HEAD><STYLE>SOME STYLE CODE</STYLE></HEAD><META http-equiv=Content-Type content="text/html; charset=utf-8"><BODY><H1>SHIT</H1>WANTED<H1>SHIT</H1><H1>SHIT</H1>WANTED<H1>SHIT</H1>WANTED<H1>SHIT</H1>WANTED</BODY><HTML>'
soup = BeautifulSoup(htmlfile, 'html.parser')
c = soup.find('body').findAll()
for i in c:
    print i.nextSibling

打印:

WANTED
<h1>SHIT</h1>
WANTED
WANTED
WANTED

1 个答案:

答案 0 :(得分:2)

现在你可以将HTML_TEXT作为你从废弃网址获得的html。

var page = require('webpage').create();
page.onResourceRequested = function(request) {
  console.log('Request ' + JSON.stringify(request, undefined, 4));
};
page.onResourceReceived = function(response) {
  console.log('Receive ' + JSON.stringify(response, undefined, 4));
};