我正在将pd.read_csv
与chunkszie
配合使用,以将1.6GB
文本文件加载为dataframe
格式。
典型的数据组如下所示:
beer/name: Sausa Weizen
review/aroma: 2
review/palate: 1.5
review/text: A lot of foam. But a lot. In the smell some banana, and then lactic and tart. Not a good start. Quite dark orange in color, with a lively carbonation (now visible, under the foam). Again tending to lactic sourness. Same for the taste. With some yeast and banana.
这是我的代码:
chunksize = 10**5
TextFileReader = pd.read_csv('beerreview.txt', error_bad_lines=False,
chunksize=chunksize, iterator=True)
beerdata = pd.concat(TextFileReader, ignore_index=True)
但是,运行完代码后,我得到除'review/text'
以外的所有字段。我可以知道为什么会这样吗?