Question

我正在尝试使用NLTK库训练数据。我按照一步一步的过程。我做了第一步，但在做第二步时我收到了以下错误：

TypeError: a bytes-like object is required, not 'list'

我尽力纠正它，但我再次遇到同样的错误。

这是我的代码：

from bs4 import BeautifulSoup
import urllib.request 
response = urllib.request.urlopen('http://php.net/') 
html = response.read()
soup = BeautifulSoup(html,"html5lib")
text = soup.get_text(strip=True)
print (text)

这是我的错误

C:\python\lib\site-packages\bs4\__init__.py:181: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html5lib"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 8 of the file E:/secure secure/chatbot-master/nltk.py. To get rid of this warning, change code that looks like this:

 BeautifulSoup(YOUR_MARKUP})

to this:

 BeautifulSoup(YOUR_MARKUP, "html5lib")

  markup_type=markup_type))
Traceback (most recent call last):
  File "E:/secure secure/chatbot-master/nltk.py", line 8, in <module>
    soup = BeautifulSoup(html)
  File "C:\python\lib\site-packages\bs4\__init__.py", line 228, in __init__
    self._feed()
  File "C:\python\lib\site-packages\bs4\__init__.py", line 289, in _feed
    self.builder.feed(self.markup)
  File "C:\python\lib\site-packages\bs4\builder\_html5lib.py", line 72, in feed
    doc = parser.parse(markup, **extra_kwargs)
  File "C:\python\lib\site-packages\html5lib\html5parser.py", line 236, in parse
    parseMeta=parseMeta, useChardet=useChardet)
  File "C:\python\lib\site-packages\html5lib\html5parser.py", line 89, in _parse
    parser=self, **kwargs)
  File "C:\python\lib\site-packages\html5lib\tokenizer.py", line 40, in __init__
    self.stream = HTMLInputStream(stream, encoding, parseMeta, useChardet)
  File "C:\python\lib\site-packages\html5lib\inputstream.py", line 148, in HTMLInputStream
    return HTMLBinaryInputStream(source, encoding, parseMeta, chardet)
  File "C:\python\lib\site-packages\html5lib\inputstream.py", line 416, in __init__
    self.rawStream = self.openStream(source)
  File "C:\python\lib\site-packages\html5lib\inputstream.py", line 453, in openStream
    stream = BytesIO(source)
TypeError: a bytes-like object is required, not 'list'

Answer 1

您可以通过实施简单的标签剥离器来实现它。

def strip_tags(html, invalid_tags):
    soup = BeautifulSoup(html)
    for tag in soup.findAll(True):
        if tag.name in invalid_tags:
            s = ""
            for c in tag.contents:
                if not isinstance(c, NavigableString):
                    c = strip_tags(unicode(c), invalid_tags)
                s += unicode(c)
            tag.replaceWith(s)
    return soup

html = "<p>Love, <b>Hate</b>, and <i>Hap<b>piness</b><u>y</u></i></p>"
invalid_tags = ['b', 'i', 'u']
print strip_tags(html, invalid_tags)

结果是：

<p>Love, Hate, and Happiness</p>

Answer 2

您的代码按原样运行。

UserWarning: No parser was explicitly specified是您的陈述是soup = BeautifulSoup(html)。

TypeError: a bytes-like object is required, not 'list'错误可能是由于依赖项问题造成的。

bs4 documentation表示如果您没有指定解析器，例如BeautifulSoup(markup)，它会使用您系统上安装的最佳HTML解析器：

如果您没有指定任何内容，您将获得已安装的最佳HTML解析器。 Beautiful Soup将lxml的解析器列为最佳，然后是html5lib，然后是Python的内置解析器。

在我的系统上，使用BeautifulSoup(html, "html.parser")工作得很好，速度不错，没有任何警告。 html.parser附带Python的标准库。

文档还summarizes每个解析器库的优点和缺点：

试试BeautifulSoup(html, "html.parser")。它应该工作。

如果您想要速度，可以试试BeautifulSoup(html, "lxml")。如果您没有lxml的HTML解析器，则在Windows上可能需要使用pip install lxml进行安装。

Answer 3

对于正在寻找适用于python 3的答案的任何人

invalidTags = ['br','b','font']
def stripTags(html, invalid_tags):
    soup = BeautifulSoup(html, "lxml")

    for tag in soup.findAll(True):
        if tag.name in invalid_tags:
            s = "::"
            for c in tag.contents:
                if not isinstance(c, NavigableString):
                    c = stripTags(str(c), invalid_tags)
                s += str(c)
            tag.replaceWith(s)
    return soup

如何使用BeautifulSoup清除html中的标签

3 个答案: