当我尝试使用csv.writer写入数据时出现问题

时间:2019-10-24 04:45:40

标签: python python-3.x web-crawler

import requests
import urllib as ulb
import re
import csv

temp= r"C:\Users\ATP\Desktop"+'\\'+"data"
with open(temp + '.csv', 'a', newline='') as f:
    csvwriter = csv.writer(f, dialect='excel')
    csvwriter.writerow(['ID', 'total', 'share', 'rate', 'employee', 'social', 'stake', 'environment'])
for i in range(1,184):
    url = (
        'http://stockdata.stock.hexun.com/zrbg/data/zrbList.aspx?date={}-12-31&count=20&pname=20&titType=null&page={}&callback=hxbase_json11571728040603'.format(
            2018, i))
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.62 Safari/537.36'}
    req = ulb.request.Request(url=url, headers=headers)
    response = ulb.request.urlopen(req)
    html1 = response.read().decode('gbk')

    id = re.findall(r'(?<=\()\d\d\d\d\d\d(?=\))',html1)
    total = re.findall(r'(?<=industryrate:\').+?(?=\')',html1)
    share = re.findall(r'(?<=stockNumber:\').+?(?=\')',html1)
    rate = re.findall(r'(?<=Pricelimit:\').+?(?=\')',html1)      
    employee = re.findall(r'(?<=lootingchips:\').+?(?=\')',html1)

    social = re.findall(r'(?<=Strongstock:\').+?(?=\')',html1)

    stake = re.findall(r'(?<=Scramble:\').+?(?=\')',html1)
    environment = re.findall(r'(?<=rscramble:\').+?(?=\')',html1)  

    for a,b,c,d,e,f,g,h in zip(id,total,share,rate,employee,social,stake,environment):

        with open(temp+'.csv','a',newline = '\n',encoding='gbk') as f:
            csvwriter = csv.writer(f,dialect='excel')
            csvwriter.writerow([a,b,c,d,e,f,g,h])

上面的代码。问题出在csv文件中的变量“ social”(即“ social”列)。
如果我运行'print(social)',则数据或结果可能会按预期显示。
但是,当将其写入csv文件时,“社交”列将显示“ <_ io.TextIOWrapper名称='C:\ Users \ ATP \ Desktop \ data.csv'mode ='a'encoding ='gbk' > ”,而不是预期的数据。
有谁知道如何解决这个问题?提前致谢。 enter image description here

1 个答案:

答案 0 :(得分:1)

您正在覆盖for循环中的变量f。 将打开的语句更改为:

with open(temp+'.csv','a',newline = '\n',encoding='gbk') as fp: # as f will override the f of the foor loop

显然将csvwriter = csv.writer(f,dialect='excel')更改为csvwriter = csv.writer(fp,dialect='excel')

请记住这个错误,向自己解释为什么键入额外的字符并使用有意义的变量名非常有用。