使用zipfile将文件压缩到内存不足

时间:2019-06-08 06:49:59

标签: python pandas

我想压缩CSV文件并将其存储,但是当我尝试在总行数设置为10000000左右但其记录较少时却出现错误提示

import pandas as pd
import numpy as np
from io import BytesIO, StringIO
import zipfile
s_buf = StringIO()
s_buf.name='my_filename.csv'
zipped_file = BytesIO()
df = pd.DataFrame(np.random.randn(10000000, 4), columns=list('ABCD'))
df.to_csv(s_buf,index=False)
filepath =  "D:/bere_zip.zip"
s_buf.seek(0)
with zipfile.ZipFile(zipped_file, mode="w",compression=zipfile.ZIP_DEFLATED) as f:
    f.writestr(s_buf.name, s_buf.getvalue())
zipped_file.seek(0)
with open(filepath, "wb") as f:
     f.write(zipped_file.getvalue())

错误日志

Traceback (most recent call last):
  File "C:/Users/ASCB/PycharmProjects/dfZip/dfZiptest.py", line 13, in <module>
    f.writestr(s_buf.name, s_buf.getvalue())
MemoryError


0 个答案:

没有答案