使用python 3合并大二进制文件

时间:2014-10-31 17:46:33

标签: python-3.x

如何使用python 3将几个大文件合并为1? 它应该像bash命令一样工作

cat * > outfile

但它应该适用于Linux,Windows和OS X.

如果我使用

outfile = open("outfile", "wb")
for file in os.listdir():
    outfile.write(file.read())

它使用太多RAM

1 个答案:

答案 0 :(得分:1)

对于大型二进制文件,不是读取行,而是读取磁盘块大小的倍数的块。像(未经测试的)

之类的东西
BLOCKSIZE = 4096  # typical, I believe
BLOCKS = 1024  # somewhat arbitrary
chunk = BLOCKS * BLOCKSIZE
with open("outfile", "wb") as outfile:
    for fname in os.listdir():
        with open('fname', "rb") as infile
            outfile.write(infile.read(chunk))