Question

我一直在研究一个项目，目的是从业余曲棍球站点抓取时间表并将其以可以上传到Sports Engine应用程序中的格式导出到csv。我已经设法以纯文本格式获取了我想要的数据，但是现在需要弄清楚如何进行转换，以便可以将其导出到csv。

以下是该脚本的示例输出，为简洁起见，将其缩短。

AL1602·11月6日·Atom A联赛·FVC 3航班·FINALMSA竞技场·阿伯茨福德，BCLANGLEY MHA ATOM A4 EAGLES2-6ABBOTSFORD ATOM A2 HAWKS AL1607·11月10日·Atom A League·FVC 3FINAL任务休闲中心·北·任务，冰时间冲突导致BCTime变化CSABBOTSFORD ATOM A2 HAWKS5-4MISSION MHA ATOM A2

这是脚本的示例输出，但仅使用print(tables)来显示格式，而不仅仅是打印出文本。

[<tr class="gamelist-row"><td class="game-details"><div class="game-meta text-muted">AL1602 · Nov 6<a class="text-muted" href="/leagues/786?scheduleId=1265&groupId=5" title="Atom A League · FVC Flight 3"> · Atom A League · FVC Flight 3</a></div><div class="game-time">FINAL</div><div class="game-arena">MSA Arena<span class="text-muted"> · Abbotsford, BC</span></div></td><td><div class="game-matchup"><a class="team-link" href="/teams/4688?scheduleId=1265&groupId=5"><div class="d-flex flex-row" style="min-width: 125px;"><div class="pr-2"><div alt="LANGLEY MHA ATOM A4 EAGLES" class="team-logo" style='background-image: url("https://s3-ca-central-1.amazonaws.com/hisports-logos/1537488764672.png");'></div></div><div class="d-flex flex-fill flex-column justify-content-center"><span class="team-name text-uppercase">LANGLEY MHA ATOM A4 EAGLES</span></div></div></a><div class="game-result score"><div class="result result-loss">2</div><span class="text-muted"> - </span><div class="result result-win">6</div></div><a class="team-link" href="/teams/4326?scheduleId=1265&groupId=5"><div class="d-flex flex-row flex-row-reverse" style="min-width: 125px;"><div class="pl-2"><div alt="ABBOTSFORD ATOM A2 HAWKS" class="team-logo" style='background-image: url("https://s3-ca-central-1.amazonaws.com/hisports-logos/1538567502609.jpg");'></div></div><div class="d-flex flex-fill flex-column justify-content-center"><span class="team-name text-uppercase text-right">ABBOTSFORD ATOM A2 HAWKS</span></div></div></a></div></td></tr>, <tr class="gamelist-row"><td class="game-details"><div class="game-meta text-muted">AL1607

下面是脚本。

from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup

#launch url
url = "https://games.pcaha.ca/teams/4326"

#create a new Firefox session
driver = webdriver.Firefox()
driver.implicitly_wait(30)
driver.get(url)

#After opening the url above, Selenium finds the table with the schedule
games = driver.find_elements_by_id("table-responsive")

#Selenium hands the page source to Beautiful Soup
soupsource=BeautifulSoup(driver.page_source, 'lxml')
soupsource.prettify()

#Beautiful Soup grabs the class gamelist-row
tables = soupsource.find_all("tr", class_="gamelist-row")

# prints out the text only
for x in tables:
    print(x.text)

Answer 1

import csv

with open('file.csv', mode='w') as csv_file:
fieldnames = ['header1', 'header2', 'header3']
     writer = csv.DictWriter(csv_file, fieldnames=fieldnames)

     writer.writeheader()
     writer.writerow({'field1': 'John Smith', 'field2': 'Accounting','field3': 'November'})

尝试使用此小片段来写入csv文件。修改它以满足您的需求！

使用Python和Selenium进行网络抓取

1 个答案: