将HTML(无序)列表转换为嵌套的Python字典

时间:2019-06-03 19:27:28

标签: python web-scraping beautifulsoup html-parsing

如果我有一个嵌套的HTML(无序)列表,看起来像这样:

<<ul style="">
  <li class="jstree-last jstree-open" id="wfo-7000000004">
    <ins class="jstree-icon"> </ins>
    <a class="" href="taxon/wfo-7000000004">
      <ins class="jstree-icon"> </ins>
      Acoraceae
    </a>
    <ul style="">
      <li class="jstree-last jstree-open" id="wfo-4000000350">
        <ins class="jstree-icon"> </ins>
        <a class="" href="taxon/wfo-4000000350">
          <ins class="jstree-icon"> </ins>
          Acorus
        </a>
        <ul style="">
          <li class="jstree-open" id="wfo-0000350733">
            <ins class="jstree-icon"> </ins>
            <a class="" href="taxon/wfo-0000350733">
              <ins class="jstree-icon"> </ins>
              Acorus calamus
            </a>
            <ul style="">
              <li class="jstree-leaf" id="wfo-0000350841">
                <ins class="jstree-icon"> </ins>
                <a class="" href="taxon/wfo-0000350841">
                  <ins class="jstree-icon"> </ins>
                  Acorus calamus var. americanus
                </a>
              </li>
              <li class="jstree-last jstree-leaf" id="wfo-0000350949">
                <ins class="jstree-icon"> </ins>
                <a class="" href="taxon/wfo-0000350949">
                  <ins class="jstree-icon"> </ins>
                  Acorus calamus var. angustatus
                </a>
              </li>
            </ul>
          </li>
          <li class="jstree-last jstree-leaf" id="wfo-0000352676">
            <ins class="jstree-icon"> </ins>
            <a class="" href="taxon/wfo-0000352676">
              <ins class="jstree-icon"> </ins>
              Acorus gramineus
            </a>
          </li>
        </ul>
      </li>
    </ul>
  </li>
</ul>

如何在Python中用它形成嵌套字典?例如:

{
  Acorales: {
    Acoraceae: {
      Acorus: {
        Acoruscalamus: [
          Acoruscalamusvar.americanus,
          Acoruscalamusvar.angustatus
        ],
        Acorusgramineus
      }
    }
  }
}

我认为 Beautiful Soup HTML Parser 之类的库都具有执行此操作的功能(在python中使用for循环),但是我无法弄清楚。感谢您的帮助!

我尝试过这种方式:

def create_dic(soup):
    return {li.a.get_text().replace("\xa0", ""): create_dic(li)
            for ul in soup('ul', recursive=False)
            for li in ul('li', recursive=False)}

但是,输出是这样的(列表中应该包含Acorus calamus var。americanus和Acorus calamus var。angustatus,而Acorus gramineus不是字典):

{'Acorales': {'Acoraceae': {'Acorus': {'Acorus calamus': {'Acorus calamus var. americanus': {},
                                                          'Acorus calamus var. angustatus': {}},
                                       'Acorus gramineus': {}}}}}

1 个答案:

答案 0 :(得分:0)

我会回答这个问题,因为要使 Parsing nested HTML list with BeautifulSoup 的答案起作用,您必须调用 beautifulsoup 来解析您的 html uls。我还将问题标记为重复,因此如果重复则关闭/删除。

from bs4 import BeautifulSoup

htmlbody = '''
<<ul style="">
  <li class="jstree-last jstree-open" id="wfo-7000000004">
    <ins class="jstree-icon"> </ins>
    <a class="" href="taxon/wfo-7000000004">
      <ins class="jstree-icon"> </ins>
      Acoraceae
    </a>
    <ul style="">
      <li class="jstree-last jstree-open" id="wfo-4000000350">
        <ins class="jstree-icon"> </ins>
        <a class="" href="taxon/wfo-4000000350">
          <ins class="jstree-icon"> </ins>
          Acorus
        </a>
        <ul style="">
          <li class="jstree-open" id="wfo-0000350733">
            <ins class="jstree-icon"> </ins>
            <a class="" href="taxon/wfo-0000350733">
              <ins class="jstree-icon"> </ins>
              Acorus calamus
            </a>
            <ul style="">
              <li class="jstree-leaf" id="wfo-0000350841">
                <ins class="jstree-icon"> </ins>
                <a class="" href="taxon/wfo-0000350841">
                  <ins class="jstree-icon"> </ins>
                  Acorus calamus var. americanus
                </a>
              </li>
              <li class="jstree-last jstree-leaf" id="wfo-0000350949">
                <ins class="jstree-icon"> </ins>
                <a class="" href="taxon/wfo-0000350949">
                  <ins class="jstree-icon"> </ins>
                  Acorus calamus var. angustatus
                </a>
              </li>
            </ul>
          </li>
          <li class="jstree-last jstree-leaf" id="wfo-0000352676">
            <ins class="jstree-icon"> </ins>
            <a class="" href="taxon/wfo-0000352676">
              <ins class="jstree-icon"> </ins>
              Acorus gramineus
            </a>
          </li>
        </ul>
      </li>
    </ul>
  </li>
</ul>

'''

def ul_to_dict(ul):
    result = {}
    for li in ul.find_all("li", recursive=False):
        key = next(li.stripped_strings)
        ul = li.find("ul")
        if ul:
            result[key] = ul_to_dict(ul)
        else:
            result[key] = None
    return result

# Let BeautifulSoup do it's magic and parse ul from the HTML.
htmlbody = BeautifulSoup(htmlbody).ul
# run our function
ul_to_dict(htmlbody)