Question

我正在尝试使用Python 3将本网站的图表从网页上抓取到.csv文件中：2016 NBA National TV Schedule

图表开始于：

Tuesday, October 25
8:00 PM Knicks/Cavaliers TNT
10:30 PM Spurs/Warriors TNT
Wednesday, October 26
8:00 PM Thunder/Sixers ESPN
10:30 PM Rockets/Lakers ESPN

我正在使用这些软件包：

from bs4 import BeautifulSoup
import requests
import pandas as pd
import numpy as np

我想要的.csv文件中的输出如下所示：

这是网站上图表到.csv文件中的前六行。注意如何多次使用多个日期。如何实施刮板以获得此输出？

Answer 1

import re
import requests
import pandas as pd
from bs4 import BeautifulSoup
from itertools import groupby

url = 'https://fansided.com/2016/08/11/nba-schedule-2016-national-tv-games/'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

days = 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday'
data = soup.select_one('.article-content p:has(br)').get_text(strip=True, separator='|').split('|')

dates, last = {}, ''
for v, g in groupby(data, lambda k: any(d in k for d in days)):
    if v:
        last = [*g][0]
        dates[last] = []
    else:
        dates[last].extend([re.findall(r'([\d:]+ [AP]M) (.*?)/(.*?) (.*)', d)[0] for d in g])

all_data = {'Date':[], 'Time': [], 'Team 1': [], 'Team 2': [], 'Network': []}
for k, v in dates.items():
    for time, team1, team2, network in v:
        all_data['Date'].append(k)
        all_data['Time'].append(time)
        all_data['Team 1'].append(team1)
        all_data['Team 2'].append(team2)
        all_data['Network'].append(network)

df = pd.DataFrame(all_data)
print(df)

df.to_csv('data.csv')

打印：

                      Date      Time    Team 1     Team 2 Network
0      Tuesday, October 25   8:00 PM    Knicks  Cavaliers     TNT
1      Tuesday, October 25  10:30 PM     Spurs   Warriors     TNT
2    Wednesday, October 26   8:00 PM   Thunder     Sixers    ESPN
3    Wednesday, October 26  10:30 PM   Rockets     Lakers    ESPN
4     Thursday, October 27   8:00 PM   Celtics      Bulls     TNT
..                     ...       ...       ...        ...     ...
159      Saturday, April 8   8:30 PM  Clippers      Spurs     ABC
160       Monday, April 10   8:00 PM   Wizards    Pistons     TNT
161       Monday, April 10  10:30 PM   Rockets   Clippers     TNT
162    Wednesday, April 12   8:00 PM     Hawks     Pacers    ESPN
163    Wednesday, April 12  10:30 PM  Pelicans    Blazers    ESPN

[164 rows x 5 columns]

并保存data.csv（来自Libre Office的屏幕截图）：

如何通过网络使用Python抓取图表？

1 个答案: