Question

我正在从nhl赔率/投注赔率信息中对我自己的知识进行一些数据分析。我可以提取一些信息，但不能提取整个数据集。我想将游戏列表及其关联列表拉入panda数据框，但是我已经能够围绕html标签执行正确的循环。我已经尝试过findAll选项和xpath路由。我也不成功。

from bs4 import BeautifulSoup
import requests

page_link = 'https://www.thespread.com/nhl-hockey-public-betting-chart'

page_response = requests.get(page_link, timeout=5)

# here, we fetch the content from the url, using the requests library
page_content = BeautifulSoup(page_response.content, "html.parser")


# Take out the <div> of name and get its value
name_box = page_content.find('div', attrs={'class': 'datarow'})
name = name_box.text.strip()

print (name)

Answer 1

这是我对您问题的解决方案。

from bs4 import BeautifulSoup
import requests

page_link = 'https://www.thespread.com/nhl-hockey-public-betting-chart'

page_response = requests.get(page_link, timeout=5)

# here, we fetch the content from the url, using the requests library
page_content = BeautifulSoup(page_response.content, "html.parser")


for cell in page_content.find_all('div', attrs={'class': 'datarow'}):
    name = cell.text.strip()
    print (name)

Answer 2

此脚本遍历每个数据行，分别提取每个项目，然后将它们附加到pandas DataFrame中。

from bs4 import BeautifulSoup
import requests
import pandas as pd

page_link = 'https://www.thespread.com/nhl-hockey-public-betting-chart'

page_response = requests.get(page_link, timeout=5)

# here, we fetch the content from the url, using the requests library
page_content = BeautifulSoup(page_response.content, "html.parser")


# Take out the <div> of name and get its value
tables = page_content.find_all('div', class_='datarow')

# Iterate through rows
rows = []

# Iterate through each datarow and pull out each home/away separately
for table in tables:
    # Get time and date
    time_and_date_tag = table.find_all('div', attrs={"class": "time"})[0].contents
    date = time_and_date_tag[1]
    time = time_and_date_tag[-1]
    # Get teams
    teams_tag = table.find_all('div', attrs={"class": "datacell teams"})[0].contents[-1].contents
    home_team = teams_tag[1].text
    away_team = teams_tag[-1].text
    # Get opening
    opening_tag = table.find_all('div', attrs={"class": "child-open"})[0].contents
    home_open_value = opening_tag[1]
    away_open_value = opening_tag[-1]
    # Get current
    current_tag = table.find_all('div', attrs={"class": "child-current"})[0].contents
    home_current_value = current_tag[1]
    away_current_value = current_tag[-1]
    # Create list
    rows.append([time, date, home_team, away_team,
                 home_open_value, away_open_value,
                 home_current_value, away_current_value])

columns = ['time', 'date', 'home_team', 'away_team',
           'home_open', 'away_open',
           'home_current', 'away_current']

print(pd.DataFrame(rows, columns=columns))

如何使用python从html页面提取多个值？

2 个答案: