bs4 python网络抓取

时间:2020-09-06 10:13:08

标签: python web-scraping beautifulsoup

我只想仅访问该特定div中的文本。 结构如下:

<div class="edgtf-pli-text"><h4 class="edgtf-pli-title entry-title" itemprop="name">
Crash Landing on You</h4></div>

,代码为:

import requests
from bs4 import BeautifulSoup
page = requests.get('https://kdramaclicks.com/kdrama/romantic-comedy/')
soup = BeautifulSoup(page.content,'html.parser')
names = soup.find_all('div',class_='edgtf-pli-text')
print(names)

我将如何模制代码,以便仅显示文本,即“崩溃着陆在您身上?”

我真的是新手,所以请帮我一点忙,如果有什么好用的api,也可以推荐wiki表

3 个答案:

答案 0 :(得分:1)

使用get_text()方法在标签内提取文本。

for name in names:
    print(name.get_text(strip=True))

Crash Landing on You
Meow, The Secret Boy
Seven First Kisses
What’s Wrong with Secretary Kim
Touch Your Heart
The Secret Life of My Secretary
Strong Girl Bong-soon
Suspicious Partner
Secret Garden
She Was Pretty
Shopping King Louis
Oh My Venus
My Love from the Star
My First First Love
Legend of the Blue Sea
The Big Hit
Her Private Life
Beating Again
Emergency Couple
Clean with Passion for Now
Be Melodramatic

答案 1 :(得分:1)

import requests
from bs4 import BeautifulSoup


def main(url):
    r = requests.get(url)
    soup = BeautifulSoup(r.content, 'html.parser')
    target = [item.get_text(strip=True) for item in soup.select(
        "h4.edgtf-pli-title.entry-title")]
    print(target)


main("https://kdramaclicks.com/kdrama/romantic-comedy/")

输出:

['Crash Landing on You', 'Meow, The Secret Boy', 'Seven First Kisses', 'What’sWrong with Secretary Kim', 'Touch Your Heart', 'The Secret Life of My Secretary', 'Strong Girl Bong-soon', 'Suspicious Partner', 'Secret Garden', 'She Was Pretty', 'Shopping King Louis', 'Oh My Venus', 'My Love from the Star', 'My FirstFirst Love', 'Legend of the Blue Sea', 'The Big Hit', 'Her Private Life', 'Beating Again', 'Emergency Couple', 'Clean with Passion for Now', 'Be Melodramatic']

答案 2 :(得分:0)

您可以使用BeautifulSoup标签的.text属性,然后使用.strip()(以删除每个韩语剧名中的前面的“ \ n”(换行符))。

import requests
from bs4 import BeautifulSoup


page = requests.get('https://kdramaclicks.com/kdrama/romantic-comedy/')
soup = BeautifulSoup(page.content,'html.parser')
names = soup.find_all('div',class_='edgtf-pli-text')
for name in names:
    print(name.text.strip())