Python:从Web scrape导出CSV

时间:2016-11-18 18:41:24

标签: python-2.7 csv selenium-webdriver web-scraping

下面的代码目前打印出我想要的所有网站;但是,我很难将其变成我可以使用的CSV格式。

from selenium import webdriver
import csv

driver = webdriver.PhantomJS()

#fetch top Amsterdam restaurants
driver.get('http://www.eater.com/maps/best-amsterdam-restaurants')

a=[]
b=[]
c=[]

for elem in driver.find_elements_by_xpath('.//h2[span[@class = "c-mapstack__card-index"]]'):
    restname = elem.text.encode('ascii', 'ignore')
    a.append(restname)

for address in driver.find_elements_by_class_name('c-mapstack__address'):
    restaddress = address.text.encode('ascii', 'ignore').strip()
    b.append(restaddress)
for content in driver.find_elements_by_class_name('c-entry-content'):
    restdescrip = content.text.encode('ascii', 'ignore').strip()
    c.append(restdescrip)

q=[(x,y) for x,y in zip(b, b[1:]) if '+31' in y]


q.insert(21,'Raadhuisstraat Amsterdam, Netherlands')
q.insert(25,'Leidsestraat 94 Amsterdam, North Holland 1017 PE, Netherlands')
d=c[1:]

new_dict= dict((a[i], (d[i],q[i])) for i in range(len(a)))
with open('EaterPull-Amsterdam.csv', 'a') as fd:
    writer = csv.writer(fd, 'excel')
    writer.writerow(new_dict.iteritems())
#commented this out to write csv
#for k, v in new_dict.iteritems():
    #print k, v

运行时出现此错误:

writer.writerow(new_dict.iteritems())
    _csv.Error: sequence expected

我觉得这很简单,但我无法完全了解正在发生的事情。

1 个答案:

答案 0 :(得分:0)

这最终解决了我的问题。 k和v是csv模块想要的序列。代码如下:

for k, v in new_dict.iteritems():
    print k, v
    eater = [k,v]
    with open('EaterPull-Amsterdam.csv', 'a') as fd:
        writer = csv.writer(fd, delimiter=",")
        writer.writerow(eater)