如何打开嵌套的zip存档并将其附加到数据框中而不进行提取?

时间:2019-04-17 12:01:16

标签: python-3.x pandas dataframe zipfile

我正在尝试打开大量在zip文件的多层中找到的csv文件。考虑到该项目的性质,我尝试打开它们,将它们read_csv放入数据框,将该数据附加到聚合数据框,然后继续循环。

示例:文件夹目录/第一个Zip /第二个Zip /第三个Zip / csv file.csv

我现有的代码可以遍历第二个和第三个zip文件的内容,并获取每个csv文件的名称。我知道可以通过导入glob来使此代码更简单,但是我并不熟悉。

import os
import pandas as pd 
import zipfile, re, io
directory = 'C:/Test/'
os.chdir(directory)
fname = "test" + ".zip"
with zipfile.ZipFile(fname, 'r') as zfile:
    # second level of zip files
    for zipname in zfile.namelist():
        if re.search(r'\.zip$', zipname) != None:
            zfiledata = io.BytesIO(zfile.read(zipname))
            # third level of zip files
            with zipfile.ZipFile(zfiledata) as zfile2:
                for zipname2 in zfile2.namelist():
                    # this zipfile contains xml and csv contents. This filters out the xmls
                    if zipname2.find("csv") > 0:
                        zfiledata2 = io.BytesIO(zfile2.read(zipname2))
                        with zipfile.ZipFile(zfiledata2) as zfile3:
                            fullpath = directory + fname + "/" + zipname + "/" + zipname2 + "/"
                            # csv file names are always the same as their zips. this cleans the string.
                            csvf = zipname2.replace('_csv.zip',".csv")
                            filehandle = open(fullpath, 'rb')
                            # the above statement is erroring: FileNotFoundError: [Errno 2] No such file or directory:
                            zfilehandle = zipfile.ZipFile(filehandle)
                            data = []
                            csvdata = StringIO.StringIO(zfilehandle.read(csvf))
                            df = pd.read_csv(csvdata)
                            data.append(df)
print(data.head())

0 个答案:

没有答案