我正在尝试将表行中的所有数据放入一个列表中,但是其中一些是表头,有些是表数据,不确定如何同时获取它们。
这是使用bs4的Python 3.7
import requests, bs4
url = 'https://www.basketball-reference.com/players/a/abrinal01.html'
res = requests.get(url)
res.raise_for_status()
soup = bs4.BeautifulSoup(res.text, 'html.parser')
elems = soup.select('#per_game')
table = soup.find("table", { "id" : "per_game" })
table_rows = table.find_all('tr')
for tr in table_rows:
td = tr.find_all('td')
row = [i.text for i in td]
print(row)
我可以将所有表数据放入一个列表中,因此第一列的所有内容都正确,但第一列本身的数据不正确。
答案 0 :(得分:1)
要在数据中包含标题,可以将find_all('td')
和find_all('th')
的输出合并。这就是你想要的吗?
import requests, bs4
url = 'https://www.basketball-reference.com/players/a/abrinal01.html'
res = requests.get(url)
res.raise_for_status()
soup = bs4.BeautifulSoup(res.text, 'html.parser')
elems = soup.select('#per_game')
table = soup.find("table", { "id" : "per_game" })
table_rows = table.find_all('tr')
for tr in table_rows:
td = tr.find_all('th') + tr.find_all('td')
row = [i.text for i in td]
print(row)
产生此输出:
['Season', 'Age', 'Tm', 'Lg', 'Pos', 'G', 'GS', 'MP', 'FG', 'FGA', 'FG%', '3P', '3PA', '3P%', '2P', '2PA', '2P%', 'eFG%', 'FT', 'FTA', 'FT%', 'ORB', 'DRB', 'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS']
['2016-17', '23', 'OKC', 'NBA', 'SG', '68', '6', '15.5', '2.0', '5.0', '.393', '1.4', '3.6', '.381', '0.6', '1.4', '.426', '.531', '0.6', '0.7', '.898', '0.3', '1.0', '1.3', '0.6', '0.5', '0.1', '0.5', '1.7', '6.0']
['2017-18', '24', 'OKC', 'NBA', 'SG', '75', '8', '15.1', '1.5', '3.9', '.395', '1.1', '2.9', '.380', '0.4', '0.9', '.443', '.540', '0.5', '0.6', '.848', '0.3', '1.2', '1.5', '0.4', '0.5', '0.1', '0.3', '1.7', '4.7']
['2018-19', '25', 'OKC', 'NBA', 'SG', '31', '2', '19.0', '1.8', '5.1', '.357', '1.3', '4.1', '.323', '0.5', '1.0', '.500', '.487', '0.4', '0.4', '.923', '0.2', '1.4', '1.5', '0.6', '0.5', '0.2', '0.5', '1.7', '5.3']
['Career', '', '', 'NBA', '', '174', '16', '16.0', '1.8', '4.5', '.387', '1.3', '3.4', '.368', '0.5', '1.1', '.443', '.525', '0.5', '0.6', '.880', '0.3', '1.1', '1.4', '0.5', '0.5', '0.1', '0.4', '1.7', '5.3']