Question

我有一个脚本，可以从页面中提取数据。我可以抓取大部分内容，但是在“ hr”标记之后会出现一些文本；我不确定该如何抓取。 HTML代码如下：

<div itemprop="articleBody" class="article-body">
            <p itemprop="immediateRelease" class="immediateRelease">IMMEDIATE RELEASE</p>
            <h1 itemprop="headline">HEADLINE</h1>
            <div class="hidden-lg meta">
                <ul>
                    <li><time pubdate="" datetime="Jan. 23, 2019">Jan. 23, 2019</time></li>
                    <li>News Release</li>

                    <li>Release No: NR-014-19</li>

                </ul>
            </div>

                <hr>

Text Text Text <br>
<br>
Text Text Text <br>
<br>
Text Text Text.<br>
<br>
Text Text Text  <a href="mailto: Text Text Text " class="ApplyClass"> Text Text Text </a>.<br>
<p>&nbsp;</p>
<p>E Text Text Text </p>

            </div>

如何提取hr标记后直到div标记结束的文本？对于其他元素，我使用了类似的内容：

    for meta in soup.find_all('div',class_='hidden-lg meta'):
    data = meta.text.splitlines()

    d['date'] = data[2]
    d['type'] = data[3]
    d['release'] = data[4]

Answer 1

这有点棘手，似乎是一种解决方法，但是您可以使用bs4元素的next_sibling属性并测试type。但这有效：

from urllib.request import urlopen
import bs4
import requests
import json
from selenium import webdriver

html = """<div itemprop="articleBody" class="article-body">
            <p itemprop="immediateRelease" class="immediateRelease">IMMEDIATE RELEASE</p>
            <h1 itemprop="headline">HEADLINE</h1>
            <div class="hidden-lg meta">
                <ul>
                    <li><time pubdate="" datetime="Jan. 23, 2019">Jan. 23, 2019</time></li>
                    <li>News Release</li>

                    <li>Release No: NR-014-19</li>

                </ul>
            </div>

                <hr>

Text Text Text <br>
<br>
Text Text Text <br>
<br>
Text Text Text.<br>
<br>
Text Text Text  <a href="mailto: Text Text Text " class="ApplyClass"> Text Text Text </a>.<br>
<p>&nbsp;</p>
<p>E Text Text Text </p>

            </div>"""

soup = bs4.BeautifulSoup(html,'html.parser')
div = soup.find('div')
text =  ''
el = div.find('hr')
while(el):
    el = el.next_sibling
    if isinstance(el, bs4.element.Tag):
        text += el.get_text()
    elif isinstance(el, bs4.element.NavigableString):
        text += el

print(text)

输出：

Text Text Text 

Text Text Text 

Text Text Text.

Text Text Text   Text Text Text .
 
E Text Text Text

在BeautifulSoup中的<hr />标记后提取文本

1 个答案: