如何删除所有html标记并连接文本

时间:2019-02-07 19:47:51

标签: python-3.x web-scraping beautifulsoup

我正在使用BeautifulSoup。

review =page_soup.findAll("div",{"class":"content"})

下面是我的评论输出

 review    = [<div class="content">
    <div class="text show-more__control">AAAAA.</div>
    <div class="actions text-muted">
                        3 out of 8 found this helpful.
                            <span>
                                Was this review helpful? <a href="/registration/signin?ref_=urv"> Sign in</a> to vote.
                            </span>
    <br/>
    <a href="/review/rw1429145/?ref_=tt_urv">Permalink</a>
    </div>
    </div>, <div class="content">
    <div class="text show-more__control">BBBBB.</div>
    <div class="actions text-muted">
                        1 out of 2 found this helpful.
                            <span>
                                Was this review helpful? <a href="/registration/signin?ref_=urv"> Sign in</a> to vote.
                            </span>
    <br/>
    <a href="/review/rw2895175/?ref_=tt_urv">Permalink</a>
    </div>
    </div>]

我希望它变成这样的原始文本。

AAAAA.BBBBB.

1 个答案:

答案 0 :(得分:0)

您可以在textdiv上获得show-more__control

divs=page_soup.find_all('div',class_="show-more__control")
texts=[x.text for x in divs]
print(''.join(texts))

如果其他地方有show_more__control,则可以使用

contents=page_soup.find_all('div',class_="content")
texts=[x.find('div',class_='show-more__control').text for x in contents]
print(''.join(texts))

修改:修改后的答案以反映您的问题中的变化

原始问题的带有答案的html

html="""
<div class="content">
<div class="text show-more__control">AAAAA.<br/><br/>Ted's Evaluation -- 1 of 3: You can find something better to do with this part of your life.</div>
<div class="actions text-muted">
                    3 out of 8 found this helpful.
                        <span>
                            Was this review helpful? <a href="/registration/signin?ref_=urv"> Sign in</a> to vote.
                        </span>
<br/>
<a href="/review/rw1429145/?ref_=tt_urv">Permalink</a>
</div>
</div>, <div class="content">
<div class="text show-more__control">BBBBB.</div>
<div class="actions text-muted">
                    1 out of 2 found this helpful.
                        <span>
                            Was this review helpful? <a href="/registration/signin?ref_=urv"> Sign in</a> to vote.
                        </span>
<br/>
<a href="/review/rw2895175/?ref_=tt_urv">Permalink</a>
</div>
</div>
"""
from bs4 import BeautifulSoup
soup=BeautifulSoup(html,"html.parser")
divs=soup.find_all('div',class_="show-more__control")
texts=[x.contents[0] for x in divs]
print(''.join(texts))

输出:

AAAAA.BBBBB.

在这种情况下,仅使用text属性就可以给出输出

AAAAA.Ted's Evaluation -- 1 of 3: You can find something better to do with this part of your life.BBBBB.