无法解析注释中的某些文本

时间:2018-02-15 08:13:25

标签: python python-3.x web-scraping beautifulsoup

我尝试从下面的代码段中解析评论中的内容,但它似乎根本不起作用。我怎样才能使它工作?我的目的是将文本放在p标记内,输出应为:

Hi there!!
Hi again!!

脚本我已经尝试过:

from bs4 import BeautifulSoup, Comment

content="""
<!-- comment --><a href="https://extratorrent.ag/"><p>Hi there!!</p></a>
<!-- comment1 --><a href="https://thepiratebay.se/"><p>Hi again!!</p></a>
"""
soup = BeautifulSoup(content, 'lxml')
for comment in soup.find_all(string=lambda text:isinstance(text,Comment)):
    data = BeautifulSoup(comment.next_element,"lxml")
    for item in data.select("p"):
        print(item.text)

我遇到的错误:

Traceback (most recent call last):
  File "C:\AppData\Local\Programs\Python\Python35-32\Social.py", line 9, in <module>
    data = BeautifulSoup(comment.next_element,"lxml")
  File "C:\AppData\Local\Programs\Python\Python35-32\lib\site-packages\bs4\__init__.py", line 191, in __init__
    markup = markup.read()
TypeError: 'NoneType' object is not callable

1 个答案:

答案 0 :(得分:2)

切换到html.parser,然后只需访问里面的p标记。

html.parser的优势在于它不会在汤数据周围添加额外的<html><body>...</body></html>标记。然后,您只需使用p访问comment.next_element.p.text代码的内容。

soup = BeautifulSoup(content, 'html.parser')
for comment in soup.find_all(string=lambda text: isinstance(text, Comment)):
    print(comment.next_element.p.text)

Hi there!!
Hi again!!