使用BeautifulSoup解析Weather Underground时遇到问题

时间:2011-11-02 05:30:11

标签: python beautifulsoup

我正在尝试调整代码以从wunderground中提取信息。然而,我试图改编的剧本是在2008年编写的,地下天气的格式化也发生了变化。我在使用soup.body.nobr.b.string时遇到了麻烦。我想从给定的网站中提取每日的降水数据。 http://www.wunderground.com/history/airport/KBUF/2011/5/2/DailyHistory.html     import urllib2     来自BeautifulSoup导入BeautifulSoup

# Create/open a file called wunder.txt (which will be a comma-delimited file)
f = open('wunder-data.txt', 'w')

# Iterate through year, month, and day
for y in range(1980, 2007):
  for m in range(1, 13):
    for d in range(1, 32):

      # Check if leap year
      if y%400 == 0:
        leap = True
      elif y%100 == 0:
        leap = False
      elif y%4 == 0:
        leap = True
      else:
        leap = False

      # Check if already gone through month
      if (m == 2 and leap and d > 29):
        continue
      elif (m == 2 and d > 28):
        continue
      elif (m in [4, 6, 9, 10] and d > 30):
        continue

      # Open wunderground.com url
      url = "http://www.wunderground.com/history/airport/KBUF/"+str(y)+ "/" + str(m) + "/" + str(d) + "/DailyHistory.html"
      page = urllib2.urlopen(url)

      # Get temperature from page
      soup = BeautifulSoup(page)
      dayTemp = soup.body.nobr.b.string

      # Format month for timestamp
      if len(str(m)) < 2:
        mStamp = '0' + str(m)
      else:
        mStamp = str(m)

      # Format day for timestamp
      if len(str(d)) < 2:
        dStamp = '0' + str(d)
      else:
        dStamp = str(d)

      # Build timestamp
      timestamp = str(y) + mStamp + dStamp

      # Write timestamp and temperature to file
      f.write(timestamp + ',' + dayTemp + '\n')

# Done getting data! Close file.
f.close()

1 个答案:

答案 0 :(得分:3)

不要乱解HTML,它可能会在没有通知的情况下再次更改。 获取one of their CSV files(HTML页面底部有链接),并使用csv模块进行解析。