Question

当我使用下面的代码时，我会收到字典，但是当我尝试通过panda将其转换为csv时，我遇到了麻烦：代码如下：

from selenium import webdriver
driver = webdriver.Chrome('C:\Users\zhang\Downloads\chromedriver_win32\chromedriver.exe')
driver.get( "http://quotes.toscrape.com/")
from scrapy.selector import Selector
sel = Selector(text=driver.page_source)
for quote in sel.css("div.quote"):
   text = quote.css("span.text::text").extract_first()
   author = quote.css("small.author::text").extract_first()
   tags = quote.css("div.tags a.tag::text").extract()
   print(dict(text=text, author=author, tags=tags))

运行上面的代码，我得到如下结果：

{'text': u'\u201cThe world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.\u201d', 'tags': [u'change', u'deep-thoughts', u'thinking', u'world'], 'author': u'Albert Einstein'}
{'text': u'\u201cIt is our choices, Harry, that show what we truly are, far more than our abilities.\u201d', 'tags': [u'abilities', u'choices'], 'author': u'J.K. Rowling'}
{'text': u'\u201cThere are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.\u201d', 'tags': [u'inspirational', u'life', u'live', u'miracle', u'miracles'], 'author': u'Albert Einstein'}
{'text': u'\u201cThe person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.\u201d', 'tags': [u'aliteracy', u'books', u'classic', u'humor'], 'author': u'Jane Austen'}
{'text': u"\u201cImperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.\u201d", 'tags': [u'be-yourself', u'inspirational'], 'author': u'Marilyn Monroe'}
{'text': u'\u201cTry not to become a man of success. Rather become a man of value.\u201d', 'tags': [u'adulthood', u'success', u'value'], 'author': u'Albert Einstein'}
{'text': u'\u201cIt is better to be hated for what you are than to be loved for what you are not.\u201d', 'tags': [u'life', u'love'], 'author': u'Andr\xe9 Gide'}
{'text': u"\u201cI have not failed. I've just found 10,000 ways that won't work.\u201d", 'tags': [u'edison', u'failure', u'inspirational', u'paraphrased'], 'author': u'Thomas A. Edison'}
{'text': u"\u201cA woman is like a tea bag; you never know how strong it is until it's in hot water.\u201d", 'tags': [u'misattributed-eleanor-roosevelt'], 'author': u'Eleanor Roosevelt'}
{'text': u'\u201cA day without sunshine is like, you know, night.\u201d', 'tags': [u'humor', u'obvious', u'simile'], 'author': u'Steve Martin'}

然后x = dict(text=text, author=author, tags=tags) 但打印x x仅显示最后一个字典：

{'text': u'\u201cA day without sunshine is like, you know, night.\u201d', 'tags': [u'humor', u'obvious', u'simile'], 'author': u'Steve Martin'}

所以任何人都可以告诉我是什么原因？

Answer 1

问题是你重写了x的内容，因为每次迭代时键（文本，作者，标签）是相同的，所以为了保存所有数据，你可以这样做：

dict_x = {}
index_nb = 0
for quote in sel.css("div.quote"):
   text = quote.css("span.text::text").extract_first()
   author = quote.css("small.author::text").extract_first()
   tags = quote.css("div.tags a.tag::text").extract()
   dict_x[index_nb] = dict(text=text, author=author, tags=tags)
   index_nb += 1

然后您的df可以创建如下：

import pandas as pd
df = pd.DataFrame.from_dict(dict_x, orient='index')

并且每行都有一个引用，以及列文本，作者和标签的数据

如果您想要输出的特定格式，可以df.to_csv('file_name.csv')添加一些parameters according to the documentation来创建csv

如何使字典成为csv

1 个答案: