ValueError:dict包含的字段不在字段名中,即使使用if语句也是如此

时间:2016-07-19 01:57:14

标签: python web screen-scraping

我正在尝试使用Times的API提取所有2016年纽约时报的文章,其中包含“经济”一词。我在代码末尾收到以下错误消息:

ValueError:dict包含不在字段名中的字段:'abstract'

这是我的代码:

from nytimesarticle import articleAPI
api = articleAPI('0282db2f333f4f4095edd19f0660c978')

articles = api.search( q = 'economy', 
 fq = {'headline':'economy', 'source':['Reuters','AP', 'The New  
 YorkTimes']}, 
 begin_date = 20151231)

def parse_articles(articles):

news = []
for i in articles['response']['docs']:
    dic = {}
    dic['id'] = i['_id']
if i['abstract'] is not None:
        dic['abstract'] = i['abstract'].encode("utf8")
    dic['headline'] = i['headline']['main'].encode("utf8")
    dic['desk'] = i['news_desk']
    dic['date'] = i['pub_date'][0:10] # cutting time of day.
    dic['section'] = i['section_name']
    if i['snippet'] is not None:
        dic['snippet'] = i['snippet'].encode("utf8")
    dic['source'] = i['source']
    dic['type'] = i['type_of_material']
    dic['url'] = i['web_url']
    dic['word_count'] = i['word_count']

    locations = []
    for x in range(0,len(i['keywords'])):
        if 'glocations' in i['keywords'][x]['name']:
            locations.append(i['keywords'][x]['value'])
    dic['locations'] = locations

    subjects = []
    for x in range(0,len(i['keywords'])):
        if 'subject' in i['keywords'][x]['name']:
            subjects.append(i['keywords'][x]['value'])
    dic['subjects'] = subjects   
    news.append(dic)
return(news) 

def get_articles(date,query):

all_articles = []
for i in range(0,100): 
    articles = api.search(q = query,
           fq = {'source':['Reuters','AP', 'The New York Times']},
           begin_date = 20151231,
           end_date = 20160715,
           sort='oldest',
           page = str(i))
    articles = parse_articles(articles)
    all_articles = all_articles + articles
return(all_articles)

econ_all = []
for i in range(2015,2016):
print 'Processing' + str(i) + '...'
econ_year =  get_articles(str(i),'economy')
econ_all = econ_all + econ_year


import csv
keys = econ_all[0].keys()
with open('econ-mentions.csv', 'wb') as output_file:

dict_writer = csv.DictWriter(output_file, keys)
dict_writer.writeheader()
dict_writer.writerows(econ_all)

似乎我的if语句应该可以防止错误。另外,如果我使用“writerow”,就像我在这里看到的那样,我会在不创建csv的情况下获得完整的详细列表。任何帮助将不胜感激!

1 个答案:

答案 0 :(得分:0)

我不确定您的问题是什么,但此代码会创建一个包含内容的文件econ-mentions.csv。

<div th:each="answer : ${question.answerList }">
    <input type="radio" name="how to do?" id="how to do?" value="how to do?" th:value="${answer.sequence}">
    <label th:text="${ans.answer}"></label>
</div>