Question

我正在尝试解析从已发布的Google表格中获取的CSV数据。我正在通过一个简单的http请求获取CSV字符串。然后尝试使用csv python库解析它。解析器似乎无法识别每行中的列数，但我不知道如何解决。

我尝试了相同的代码从磁盘文件中读取csv，并且效果很好。

import csv
import requests

response = requests.get('https://docs.google.com/spreadsheets/d/e/2PACX-1vThbG2jdBjBWpcW0iarJD2TtW6olqygCvuQHRI-fwnLx6jkIFtuyiF7Bm4rEPLo_jpjLmZsr9sYGs8e/pub?output=csv&gid=784986591')
csv_bytestream = response.content.decode()
csv.register_dialect('GoogleDialect', delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
reader = csv.reader(csv_bytestream, dialect='GoogleDialect')
print(" **** NOT WORKING CODE **** ")
print(csv_bytestream)
num_row = 0
for row in reader:
    num_row += 1
    print("Row {num} has {nelem} columns".format(num=str(num_row) ,nelem=str(len(row))))
    print(row)

print(" **** WORKING CODE **** ")
with open('MyFile.csv','r', encoding='utf-8') as f:
    print(f.read())

with open('MyFile.csv', 'r', encoding='utf-8') as f:
    reader = csv.reader(f, dialect='GoogleDialect')
    num_row = 0
    for row in reader:
        num_row += 1
        print("Row {num} has {nelem} columns".format(num=str(num_row), nelem=str(len(row))))
        print(row)

我期望两个输出都一样：

Row 1 has 16 columns
['', 'Just insult, here and now!', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
Row 2 has 16 columns
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
...

从NON WORKING代码获得的输出如下：

Row 1 has 2 columns
['', '']
Row 2 has 1 columns
['Just insult, here and now!']
Row 3 has 2 columns
['', '']
...

print(csv_bytestream)和print(f.read())打印的内容相同。如果我知道该怎么做，我会附加csv文件。

Answer 1

问题在于编码！您要获取的csv响应具有ISO-8859-1

 >>> response.encoding
 >>>'ISO-8859-1'

这是可行的解决方案，可能对您有帮助

response = requests.get('https://docs.google.com/spreadsheets/d/e/2PACX-1vThbG2jdBjBWpcW0iarJD2TtW6olqygCvuQHRI-fwnLx6jkIFtuyiF7Bm4rEPLo_jpjLmZsr9sYGs8e/pub?output=csv&gid=784986591')
csv_bytestream = response.content.decode('utf-8')
cr = csv.reader(csv_bytestream.splitlines(), delimiter=',')
my_list = list(cr)
my_list = list(cr)
for row in my_list:
    print(row)

示例响应：

['', 'Just insult, here and now!', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', 'STAGIONE CONTINUA', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['', 'Rank', 'Adj', 'Player', 'Rating', 'Adj', 'Dist', 'Played', '# Pet', '% Pet/Played', '# Best', '# Worst', '(B-W)%', 'Wins', 'Losses', '']
['', '1', '⇔ (1)', 'JTsuited', '1674', '0', '0', '335', '4', '1,19%', '92', '52', '11,94%', '176', '159', 'HALL OF FAME']
['', '2', '⇔ (2)', 'Nameless', '1662', '2', '12', '293', '7', '2,39%', '74', '43', '10,58%', '171', '122', '']
['', '3', '⇑ (5)', 'Takysis', '1562', '55', '100', '688', '36', '5,23%', '145', '130', '2,18%', '348', '340', '']
['', '4', '⇓ (3)', 'Sfinks', '1528', '-19', '34', '696', '77', '11,06%', '134', '154', '-2,87%', '380', '316', '']
['', '5', '⇓ (4)', 'Neristelrooy', '1522', '-24', '6', '521', '37', '7,10%', '75', '103', '-5,37%', '282', '239', '']
['', '6', '⇔ (6)', 'Starmanzo', '1411', '-4', '111', '450', '24', '5,33%', '92', '78', '3,11%', '210', '240', '']
['', '7', '⇔ (7)', 'Bruno', '1386', '-13', '25', '446', '33', '7,40%', '74', '89', '-3,36%', '214', '232', 'HALL OF SHAME']
['', '8', '⇑ (9)', 'Biru', '1350', '-13', '36', '792', '132', '16,67%', '103', '159', '-7,07%', '414', '378', '']
['', '9', '⇓ (8)', 'Alino', '1340', '-32', '10', '33', '1', '3,03%', '3', '5', '-6,06%', '13', '20', '']
['', '10', '⇑ (12)', 'starpa', '1246', '48', '94', '796', '62', '7,79%', '177', '124', '6,66%', '337', '459', '']
['', '11', '⇓ (10)', 'Praxy', '1237', '0', '9', '223', '35', '15,70%', '30', '46', '-7,17%', '111', '112', '']
['', '12', '⇓ (11)', 'Beatren', '1218', '0', '19', '213', '19', '8,92%', '43', '43', '0,00%', '92', '121', '']
['', '13', '⇔ (13)', 'staratlas', '1169', '0', '49', '75', '9', '12,00%', '6', '22', '-21,33%', '34', '41', '']

Answer 2

问题是您正在将字符串传递给csv阅读器。相反，您应该传递类似文件的对象或Interator返回行。

您可以这样做：

import io
csv_bytestream = io.StringIO(response.content.decode())

或

csv_bytestream = response.content.decode().splitlines()

请注意，尽管变量csv_bytestream不能反映您所拥有的实际类型，但在第一种情况下它将返回文本，而不是字节，在第二种情况下将返回行。

还要注意，最好使用response.content.decode()而不是使用response.text，因为那样会使用响应的编码。

解析从Google表格获取的CSV

2 个答案: