我尝试使用elt.itertext()
(v3.5.0b1)迭代子树的文本内容,如下所示:
import lxml.html.soupparser as soupparser
import requests
doc = requests.get("http://f10.5post.com/forums/showthread.php?t=1142017").content
tree = soupparser.fromstring(doc)
nodes = tree.getchildren()
for elt in nodes:
for t in elt.itertext():
print t
但是我一直在说错误
File "src/lxml/iterparse.pxi", line 248, in lxml.etree.iterwalk.__init__ (src/lxml/lxml.etree.c:134032)
File "src/lxml/apihelpers.pxi", line 67, in lxml.etree._rootNodeOrRaise (src/lxml/lxml.etree.c:15220)
ValueError: Input object has no element: HtmlComment
有没有办法跳过所有HTML评论?此外,这个错误究竟意味着什么?
由于
答案 0 :(得分:0)
这很正常。
>>> from lxml import etree
>>> doc = '''
... <html><!-- PAGENAV POPUP -->
... <div class="vbmenu_popup" id="pagenav_menu" style="display:none">
... <table cellpadding="4" cellspacing="1" border="0">
... <tr>
... <td class="thead" nowrap="nowrap">Go to Page...</td>
... </tr>
... <tr>
... <td class="vbmenu_option" title="nohilite">
... <form action="index.php" method="get" onsubmit="return this.gotopage()" id="pagenav_form">
... <input type="text" class="bginput" id="pagenav_itxt" style="font-size:11px" size="4" />
... <input type="button" class="button" id="pagenav_ibtn" value="Go" />
... </form>
... </td>
... </tr>
... </table>
... </div>
... <!-- / PAGENAV POPUP -->
... </html>'''
>>> root = etree.fromstring(doc)
>>> nodes = root.getchildren()
>>> nodes
[<!-- PAGENAV POPUP -->, <Element div at 0x10367f290>, <!-- / PAGENAV POPUP -->]
>>> for elt in nodes:
... for t in elt.itertext():
... print t
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "lxml.etree.pyx", line 1406, in lxml.etree._Element.itertext (src/lxml/lxml.etree.c:48845)
File "lxml.etree.pyx", line 2763, in lxml.etree.ElementTextIterator.__cinit__ (src/lxml/lxml.etree.c:64747)
File "iterparse.pxi", line 219, in lxml.etree.iterwalk.__init__ (src/lxml/lxml.etree.c:125303)
File "apihelpers.pxi", line 72, in lxml.etree._rootNodeOrRaise (src/lxml/lxml.etree.c:13689)
ValueError: Input object has no element: lxml.etree._Comment
如上所示
>>> nodes
[<!-- PAGENAV POPUP -->, <Element div at 0x10367f290>, <!-- / PAGENAV POPUP -->]
注意:不推荐使用getchildren。您可以使用列表。
>>> list(root)
[<!-- PAGENAV POPUP -->, <Element div at 0x10367f290>, <!-- / PAGENAV POPUP -->]
节点是Element 和注释的列表。如果您查看itertext()的工作方式:
创建文本迭代器。迭代器按文档顺序遍历此元素和所有子元素,并返回所有内部文本。
另一方面,如果不是在列表上进行迭代,而是直接在根元素上迭代:
>>> for t in root.itertext():
... print t
...
我得到了所有文字和很多空格。 :)
如果您仍想迭代节点列表。你可以用
来推断自然>>> [item.tag for item in nodes]
[<built-in function Comment>, 'div', <built-in function Comment>]
您也可以
>>> [item.__class__ for item in nodes]
[<type 'lxml.etree._Comment'>, <type 'lxml.etree._Element'>, <type 'lxml.etree._Comment'>]