Question

我正在尝试抓取HTML页面

 productsoup = BeautifulSoup(productdriver.page_source,"lxml");

此python脚本提供了包含以下元素ID部分的html

<div style="padding-top: 10px;" id="government_funding">
    <h2>Sampling of Recent Funding Actions/Set Asides</h2>
    <p style="font-style: italic; font-size: .8em;">In order by amount of set aside monies.</p>
    <ul>
        <li><span style="color: green;">$14,450</span> - Thursday the 17th of August 2017<br><span style="font-weight: bold; font-size: 1.2em;">National Institutes Of Health</span> <br> NATIONAL INSTITUTES OF HEALTH NICHD<br>AVANTI POLAR LIPIDS:1109394 [17-010744]
            <hr>
        </li>
        <li><span style="color: green;">$5,455</span> - Thursday the 31st of August 2017<br><span style="font-weight: bold; font-size: 1.2em;">National Institutes Of Health</span> <br> NATIONAL INSTITUTES OF HEALTH NICHD<br>AVANTI POLAR LIPIDS:1109394 [17-004567]
            <hr>
        </li>
        <li><span style="color: green;">$5,005</span> - Tuesday the 8th of August 2017<br><span style="font-weight: bold; font-size: 1.2em;">National Institutes Of Health</span> <br> NATIONAL INSTITUTES OF HEALTH NIAID<br>CUSTOM LIPID SYNTHESIS (24:0-10:0 PE) 100 MG PACKAGED IN 10-10MG VIALS POWDER PER QUOTE #DQ-000665
            <hr>
        </li>
        <li><span style="color: green;">$5,005</span> - Thursday the 17th of August 2017<br><span style="font-weight: bold; font-size: 1.2em;">National Institutes Of Health</span> <br> NATIONAL INSTITUTES OF HEALTH NIAID<br>CUSTOM LIPID SYNTHESIS (24:0-10:0 PE) 100 MG PACKAGED IN 10-10MG VIALS POWDER PER QUOTE #DQ-000665
            <hr>
        </li>
    </ul>
</div>

这只是html的一部分，此部分由id =“ government_funding”标识。为id =“ goverment_funding”中的每个li打印价格，日期，代理商。所以li的输出为

价格= $ 14,450

日期= 2017年8月17日

机构=国立卫生研究院

SubAgency =美国国立卫生研究院NICHD

如何编码上面的输出？

到数据源的链接是这个 https://www.collierreporting.com/company/avanti-polar-lipids-inc-alabaster-al

Answer 1

您可以遍历li标签和随后的span值，并使用re.findall访问数据：

import re
def all_data(d):
  a, b = [i.text for i in d.find_all('span')]
  return [a, *re.findall('\w+\sthe\s\w+\sof\s\w+\s\d+', d.text), b]

results = [all_data(b) for b in productsoup.find('div', {'id':'government_funding'}).find_all('li')]

输出：

[['$14,450', 'Thursday the 17th of August 2017', 'National Institutes Of Health'], ['$5,455', 'Thursday the 31st of August 2017', 'National Institutes Of Health'], ['$5,005', 'Tuesday the 8th of August 2017', 'National Institutes Of Health'], ['$5,005', 'Thursday the 17th of August 2017', 'National Institutes Of Health']]

Python Beautiful汤检索内部的HTML内容和元素ID

1 个答案: