Question

我已经成功地成功调用了News API，并将结果放入了DataFrame中，但仅适用于第1页。

def get_articles(keyword):

  all_articles = newsapi.get_everything(q=keyword, sources='abc-news-au, news-com-au',
                                      domains='http://www.abc.net.au/news, http://www.news.com.au',
                                      from_param='2018-12-28',
                                      to='2019-01-28',
                                      language='en',
                                      sort_by='popularity',
                                      page=1)

  all_articles = pd.DataFrame(all_articles)
  all_articles = pd.concat([all_articles.drop(['articles'], axis=1), all_articles['articles'].apply(pd.Series)], axis=1)

  return all_articles

它给了我想要的数据帧，但是，当我尝试遍历以下页面时，我变得不粘住了。

我尝试了以下

empty_list = []

for i in range(1,4,1):
  all_articles = all_articles = newsapi.get_everything(q=keyword, sources='abc-news-au, news-com-au',
                                  domains='http://www.abc.net.au/news, http://www.news.com.au',
                                  from_param='2018-12-28',
                                  to='2019-01-28',
                                  language='en',
                                  sort_by='popularity',
                                  page=i)
  empty_list.append(all_articles)

这将返回所有文章，但是它是存储在列表中的字典。

[{'articles': [{'author': None,
    'content': 'Updated \r\nJanuary 14, 2019 14:33:00\r\nANZ customers have lost access to banking services at their local post offices after the bank failed to reach an agreement with Australia Post on their Bank@Post service.\r\nThe change, which came into effect last night, wil… [+5084 chars]',
    'description': 'ANZ customers can no longer utilise banking services at their local post offices after the bank failed to reach an agreement with Australia Post on their Bank@Post service.',
    'publishedAt': '2019-01-14T03:14:57Z',
    'source': {'id': 'abc-news-au', 'name': 'ABC News (AU)'},
    'title': "ANZ customers 'furious' as access to Bank@Post cancelled",
    'url': 'https://www.abc.net.au/news/2019-01-14/anz-customers-lose-banking-service-at-australia-post/10713156',
    'urlToImage': 'https://www.abc.net.au/news/image/10710052-16x9-700x394.jpg'},
   {'author': 'Stephen Letts',
    'content': "Posted \r\nJanuary 26, 2019 06:20:15\r\nIf you think AMP's glum market update of an additional $200 million worth of costs to fix its various scandals rules a line under the sordid and sorry mess, think again.\r\nKey points:\r\nRemediation costs for Australia's scand… [+5019 chars]",
    'description': "Australia's six big wealth managers currently have provisions for about $2.6 billion to fix the scandals that have emerged from the banking royal commission. That could be be woefully inadequate.",
    'publishedAt': '2019-01-25T19:20:15Z',
    'source': {'id': 'abc-news-au', 'name': 'ABC News (AU)'},
    'title': "Wealth managers' remediation costs set to soar",
    'url': 'https://www.abc.net.au/news/2019-01-26/wealth-manager-remediation-costs-set-to-soar/10749810',
    'urlToImage': 'https://www.abc.net.au/news/image/1147126-16x9-700x394.jpg'}]

以前，这只是一本字典[没有列表]。

当我进行一些转换时（类似于上面的内容），我得到以下DataFrame

问题：

有人知道更好的方法吗？
如果要使用当前数据框，如何从每列中获取字典并进行显示，使其看起来像第一个数据框？

任何帮助将不胜感激。

PS：如果要复制，则可以复制我的代码-您只需要从以下位置获取自己的API密钥即可：https://newsapi.org/docs/client-libraries/python

Answer 1

您似乎想提取商品值和extend而不是append：

articles = []

for i in range(1,4,1):
    articles_page = newsapi.get_everything(
            q=keyword,
            sources='abc-news-au, news-com-au',
            domains='http://www.abc.net.au/news, http://www.news.com.au',
            from_param='2018-12-28',
            to='2019-01-28',
            language='en',
            sort_by='popularity',
            page=i)
    articles.extend(articles_page['articles'])

# outside of the loop, create the DataFrame
pd.DataFrame(articles)

新闻API-将输出输出到Pandas DataFrame

1 个答案: