将HTML解析为纯文本

时间:2012-11-15 01:37:24

标签: python

我正在尝试使用我在多个帖子中推荐的MLStripper类来从电子邮件中删除html以获取纯文本。由于“@”符号,strip_tags函数在尝试解析时遇到问题。我想这个类不够强大,只能解析有效的html标签,如何修复下面的任何建议来处理“@”或其他库从文本中删除html?我还需要删除像&同样。

的Python:

from HTMLParser import HTMLParser

class MLStripper(HTMLParser):
            def __init__(self):
                 self.reset()
                 self.fed = []
            def handle_data(self, d):
                 self.fed.append(d)
            def get_data(self):
                 return ''.join(self.fed)

            def strip_tags(self, html):
                 s = MLStripper()
                 s.feed(html)
                 return s.get_data()

 ML = MLStripper()
 test = ML.strip_tags("<div><br>On Sep 27, 2012, at 4:11 PM, Mark Smith <marksmith@gmail.com> wrote</br></div>")
 print test

错误:

Traceback (most recent call last):
  File "IMAPReader.py", line 49, in <module>
    strippedText = ML.strip_tags("<marksmith@gmail.com>")
  File "IMAPReader.py", line 22, in strip_tags
    s.feed(html)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/HTMLParser.py", line 108, in feed
    self.goahead(0)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/HTMLParser.py", line 148, in goahead
    k = self.parse_starttag(i)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/HTMLParser.py", line 229, in parse_starttag
    endpos = self.check_for_whole_start_tag(i)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/HTMLParser.py", line 304, in check_for_whole_start_tag
    self.error("malformed start tag")
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/HTMLParser.py", line 115, in error
    raise HTMLParseError(message, self.getpos())
HTMLParser.HTMLParseError: malformed start tag, at line 1, column 9

2 个答案:

答案 0 :(得分:3)

如果您希望获得无效标记,则不需要HTML解析器。看看BeautifulSoup:

http://www.crummy.com/software/BeautifulSoup/

他们有一个很好的例子,可以完全按照自己的意愿行事:

http://www.crummy.com/software/BeautifulSoup/bs4/doc/

from bs4 import BeautifulSoup

html_doc = """
<html><head><title>The Dormouse's story</title></head>

<p class="title"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>

<p class="story">...</p>
"""

soup = BeautifulSoup(html_doc)

print(soup.get_text())

...返回

# The Dormouse's story
#
# The Dormouse's story
#
# Once upon a time there were three little sisters; and their names were
# Elsie,
# Lacie and
# Tillie;
# and they lived at the bottom of a well.
#
# ...

答案 1 :(得分:0)

您使用的是哪个版本的Python?我使用Python 2.7.2运行你的代码,我得到了同样的错误。然后我在一台带有 Python 2.7.3 的计算机上运行它,它运行得很好。这很奇怪所以我查了一下,我发现一些文档说Python中包含的HTML解析器在后来的版本中变得更加宽松。尝试升级到2.7.3,它应该可以工作。