构建WebScraper时出错:TypeError:'NoneType'对象没有属性'__getitem__'

时间:2013-02-19 04:24:14

标签: python google-api web-scraping

正在建立一个网络浏览器,显示网络趋势的顶级网址。但是,我总是返回以下错误。

Traceback (most recent call last):
  File "D:\Ceryx\webSearch.py", line 21, in <module>
    topl=webScraper(m)
  File "D:\Ceryx\webSearch.py", line 12, in webScraper
    hot = data['results'][0]['url']
TypeError: 'NoneType' object has no attribute '__getitem__'

帮助!!

import re
import json
import urllib, urllib2

def webScraper(trends):
    query=urllib.urlencode({'q':trends})
    url = 'http://ajax.googleapis.com/ajax/services/search/web?v=1.0&%s' % query
    response = urllib.urlopen(url)
    extract = response.read()
    results = json.loads(extract)
    data = results['responseData']
    hot = data['results'][0]['url']
    return hot

response = urllib2.urlopen('http://www.google.com/trends/hottrends/atom/hourly')
html = response.read()
matchObj = re.findall(r'<a[^>]*?>(.*?)</a>', html)

print "Urls"
for m in matchObj:
    topl=webScraper(m)
    print m,topl

1 个答案:

答案 0 :(得分:1)

错误在这一行:

hot = data['results'][0]['url']

这意味着以下其中一项是None

data
data['results']
data['results'][0]

您可以通过连续打印找出哪一个:

print 'data',data
print 'data[results]',data['results']
print 'data[results][0]',data['results'][0]

那么百万美元的问题将是你在json中首先得到的结果 - 并弄清楚你需要做些什么来处理它(或者如果你控制了它们就可以阻止它事情)。 :)