如何在beautifulsoup的多个列表中采用特定元素?

时间:2019-02-16 16:50:32

标签: python-3.x web-scraping beautifulsoup html-parsing

我很难提取一些特定的标记(及其字符串内容)并将其存储到变量中(以便以后可以将这些变量保存到CSV文件中)。

from bs4 import BeautifulSoup
from requests_html import HTMLSession
session = HTMLSession()
r = session.get('https://www.khanacademy.org/profile/DFletcher1990/')
r.html.render(sleep=5)
soup=BeautifulSoup(r.html.html,'html.parser')

user_info_table=soup.find('table', class_='user-statistics-table')

for tr in user_info_table.find_all('tr'):
    tds=tr.find_all('td')
    print(tds)

我想收集:

  • "4 years ago"并将其存储到名为date的变量中,
  • "932,915"并将其存储到名为points的变量中,
  • "372"并将其存储到名为videos的变量中。

我不太了解bs4.element.ResultSet的行为...

1 个答案:

答案 0 :(得分:2)

您可以将其视为列表。

from bs4 import BeautifulSoup
from requests_html import HTMLSession
session = HTMLSession()
r = session.get('https://www.khanacademy.org/profile/DFletcher1990/')
r.html.render(sleep=10)
soup=BeautifulSoup(r.html.html,'html.parser')
user_info_table=soup.find('table', class_='user-statistics-table')
dates,points,videos=[tr.find_all('td')[1].text for tr in user_info_table.find_all('tr')]
print(dates,points,videos,sep="\n")

输出

4 years ago
932,915
372