Question

这是使用BeautifulSoup解析HTML的脚本的一部分。我正在尝试从页面中获取链接，这些链接稍后会被使用。一切似乎都工作正常，但是我只想获得其中的一些链接，而不是全部，我的意思是我只对放在页面第一个表中的链接感兴趣。我确实意识到我可以手动缩短列表，但这对我来说真的行不通。

这是页面的网址：https://www.spotrac.com/nba/atlanta-hawks/cap/

有什么办法可以做到吗？

from bs4 import BeautifulSoup
from urllib.request import Request, urlopen
import re

req = Request("https://www.spotrac.com/nba/atlanta-hawks/cap/")
html_page = urlopen(req)

soup = BeautifulSoup(html_page, features="html.parser")
links = []


for link in soup.find_all('a'):
    links.append(link.get('href'))
players=[]
i=0
while i<len(links):
    if "redirect/player" in links[i]:
        players.append(links[i])
    i+=1
print(players)

Answer 1

您可以在下面调整此代码。

import requests
from bs4 import BeautifulSoup

url = 'https://www.spotrac.com/nba/atlanta-hawks/cap/'

headers = {'Host': 'www.spotrac.com',
           'Referer': 'https://www.spotrac.com/nba/atlanta-hawks/cap/',
           'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'}

pageTree = requests.get(url, headers=headers)
soup = BeautifulSoup(pageTree.content, 'lxml')

table = soup.find('tbody')
links = table.find_all('a')

for item in links:
    print(str(item.text), str(item['href']))

结果：

Kent Bazemore https://www.spotrac.com/redirect/player/11079/
Miles Plumlee https://www.spotrac.com/redirect/player/10851/
Dewayne Dedmon https://www.spotrac.com/redirect/player/13536/
Trae Young https://www.spotrac.com/redirect/player/26971/
Alex Len https://www.spotrac.com/redirect/player/13318/
Taurean Prince https://www.spotrac.com/redirect/player/20217/
Justin Anderson https://www.spotrac.com/redirect/player/17849/
John Collins https://www.spotrac.com/redirect/player/23614/
Kevin Huerter https://www.spotrac.com/redirect/player/26985/
DeAndre' Bembry https://www.spotrac.com/redirect/player/20226/
Omari Spellman https://www.spotrac.com/redirect/player/26996/
Vince Carter https://www.spotrac.com/redirect/player/2590/
Tyler Dorsey https://www.spotrac.com/redirect/player/23642/
Jaylen Adams https://www.spotrac.com/redirect/player/27343/
Jordan Sibert https://www.spotrac.com/redirect/player/18240/

如果这是您需要的，请将此答案标记为已接受。

Answer 2

您可以先在包含第一组名称的表上使用find来缩小范围：

...
table = soup.find('tbody')

for link in table.find_all('a'):
...

通过查看源代码，您似乎想要的数据位于tbody中。

Answer 3

我会考虑使用first-child CSS选择器来限制到第一个表；您可以使用descendant combinator来选择表中的href属性（您的链接）。

import requests
from bs4 import BeautifulSoup as bs
url = 'https://www.spotrac.com/nba/atlanta-hawks/cap/'
res = requests.get(url)
soup = bs(res.content, 'lxml')
links = [item['href'] for item in soup.select('table:first-child [href]')]
print(links)

从特定表进行Python BeautifulSoup解析

3 个答案: