Python美丽的汤Web抓取特定数字

时间:2015-06-12 19:37:29

标签: python html web-scraping beautifulsoup html-parsing

this page上,每个团队的最终得分(数字)具有相同的班级名称class="finalScore"

当我打电话给客队的最终得分(在顶部)时,代码会毫无问题地调用该号码。如果...... favLastGM ='A'

当我试图调用主队的最终得分时(在底部),代码给我一个错误。如果...... favLastGM ='H'

以下是我的代码:

import pickle
import math
import urllib2
from lxml import etree
from bs4 import BeautifulSoup
from urllib import urlopen

#Last Two Game info Home [H] or Away [A]
favLastGM = 'A' #Higher week number 2

#Game Info (Favorite) Last Game Played - CBS Sports (Change Every Week)
favPrevGMInfoUrl = 'http://www.cbssports.com/nfl/gametracker/boxscore/NFL_20140914_NE@MIN'
favPrevGMInfoHtml = urlopen(favPrevGMInfoUrl).read()
favPrevGMInfoSoup = BeautifulSoup(favPrevGMInfoHtml)
if favLastGM == 'A': #This Gives Final Score of Away Team - Away Score
    favScore = favPrevGMInfoSoup.find_all("td", { "class" : "finalScore" })
elif favLastGM == 'H':
    favScore = favPrevGMInfoSoup.find_all("td", { "class" : "finalScore" })[1]
else:
    print("***************************************************")
    print("NOT A VALID ENTRY - favLastGM  !")
    print("***************************************************")


print ("Enter: Total Points Allowed from Favored Team Defense for last game played: "),
print favScore[0].text

如果favLastGM ='H'

,这是我得到的错误
  

Traceback(最近一次调用最后一次):文件   “C:/Users/jcmcdonald/Desktop/FinalScoreTest.py”,第26行,in       print favScore [0] .text文件“C:\ Python27 \ lib \ site-packages \ bs4 \ element.py”,第905行,in   的的GetItem       return self.attrs [key] KeyError:0

3 个答案:

答案 0 :(得分:2)

class="finalScore"只有两个元素,第一个是主队得分,第二个是客队得分:

>>> from urllib import urlopen
>>> from bs4 import BeautifulSoup
>>> 
>>> favPrevGMInfoUrl = 'http://www.cbssports.com/nfl/gametracker/boxscore/NFL_20140914_NE@MIN'
>>> 
>>> favPrevGMInfoSoup = BeautifulSoup(urlopen(favPrevGMInfoUrl))
>>> score = [item.get_text() for item in favPrevGMInfoSoup.find_all("td", {"class": "finalScore"})]
>>> score
[u'30', u'7']

仅供参考,而不是.find_all("td", {"class": "finalScore"}),您可以使用CSS selector.select("td.finalScore")

答案 1 :(得分:0)

在您的代码中,您要为favScore分配不同类型的对象。所以在第一种情况下,你有:

if favLastGM == 'A': #This Gives Final Score of Away Team - Away Score
    favScore = favPrevGMInfoSoup.find_all("td", { "class" : "finalScore" })

你最终得到一个清单......

faveScore = [<td class="finalScore">30</td>, <td class="finalScore">7</td>]

而在第二种情况下,你有:

elif favLastGM == 'H':
    favScore = favPrevGMInfoSoup.find_all("td", { "class" : "finalScore" })[1]

你最终得到了一个BeautfulSoup元素......

favScore = <td class="finalScore">7</td>

您可以通过执行此操作来解决此问题(请注意[0]):

if favLastGM == 'A': #This Gives Final Score of Away Team - Away Score
    favScore = favPrevGMInfoSoup.find_all("td", { "class" : "finalScore" })[0]
elif favLastGM == 'H':
    favScore = favPrevGMInfoSoup.find_all("td", { "class" : "finalScore" })[1]

然后在最后做:

print favScore.text

答案 2 :(得分:0)

我轻微扩展到@ alecxe的答案,明确选择 home away 团队(而不是依赖于数组的隐式排序):

from urllib import urlopen
from bs4 import BeautifulSoup

favPrevGMInfoUrl = 'http://www.cbssports.com/nfl/gametracker/boxscore/NFL_20140914_NE@MIN'

favPrevGMInfoSoup = BeautifulSoup(urlopen(favPrevGMInfoUrl))

home_score = favPrevGMInfoSoup.find("tr", {"class": "teamInfo homeTeam"}).find("td", {"class": "finalScore"}).get_text()
away_score = favPrevGMInfoSoup.find("tr", {"class": "teamInfo awayTeam"}).find("td", {"class": "finalScore"}).get_text()

print home_score, away_score