Question

我想迭代一些html并将数据存储到字典中。每次迭代都以：

开头

<h1 class="docDisplay" id="docTitle">

我有以下代码：

html = '<html><body><h1 class="docDisplay" id="docTitle">Data1</h1><p>other data<\p><h1 class="docDisplay" id="docTitle">Data2</h1><p>other data2<\p></html>'

soup=BeautifulSoup(html)
newdoc = soup.find('h1', id="docTitle")
title = newdoc.findNext(text=True)
data = title.findAllNext('p',text=True)
data_dict = {}
data_dict[title] = {'data': data}
print data_dict

现在，输出是

{u'Data1': {'data': [u'other data<\\p>', u'Data2', u'other data2<\\p>']}}

我希望输出为：

{u'Data1': {'data': [u'other data<\\p>']}, u'Data2': {'data': [u'other data2<\\p>']}}

一旦我到达新的h1标签，我无法弄清楚如何重新开始。有什么想法吗？

Answer 1

为了匹配每个标题下的段落文本，我会尝试这样的事情（你可能需要根据你想要的确切输出格式来修改它）：

    from BeautifulSoup import BeautifulSoup

    html = """ 
    <html>
    <head>
    </head>

    <body>
      <h1 class="docDisplay" id="docTitle">Data1</h1>
      <p>other data</p>
      <p>Another paragraph under the first heading.</p>
      <h1 class="docDisplay" id="docTitle">Data2</h1>
      <p>other data2</p>
      <div><p>This paragraph is NOT a sibling of the header</p></div>
    </body>
    </html>
"""

soup = BeautifulSoup(html)

data_dict = {}
stuff_under_current_heading = []

firstHeader = soup.find('h1', id="docTitle")
for tag in [firstHeader] + firstHeader.findNextSiblings():
    if tag.name == 'h1':
        stuff_under_current_heading = []
        # I chose to strip excess whitespace from the header name:
        data_dict[tag.string.strip()] = {'data': stuff_under_current_heading}
        # Modifying the list modifies the value in the dictionary.
    # Take every <p> tag encountered between here and the next heading
    # and associate it with the most recently-seen <h1> tag.
    elif tag.name == 'p':
        stuff_under_current_heading.append(tag.string)
    # Include <p> tags that are not siblings of the <h1> tag but
    # are still part of the content under the header.
    else:
        stuff_under_current_heading.extend(tag.findAll('p', text=True))

print data_dict

此输出

{u'Data1': {'data': [u'other data', u'Another paragraph under the first heading.']},   
 u'Data2': {'data': [u'other data2', u'This paragraph is NOT a sibling of the header']}}

Answer 2

@samplebias：@Lynch是对的。如果OP没有正确关闭他/她的标签，他们就不能指望解析器能够读懂他们的想法。

尝试修复HTML，它可能会起作用。 =）

如何使用Python迭代标记？

2 个答案: