将Python中的大文件拆分为多个大文件

时间:2019-07-11 20:31:32

标签: python file python-3.7 large-files filesplitting

我正在尝试分割一个622 MB的文件,该文件最终将被导入到数据库中,但是每次导入最多只能导入100 MB。该代码和622 MB逗号分隔文件都在同一文件夹中。一段时间后,我将返回Python ...

下面来自github的代码给我一个错误:

NameError: name 'current_out_writer' is not defined

似乎csv.writer的调用未初始化current_out_writer

代码在这里:https://gist.github.com/jrivero/1085501

import csv
    reader = csv.reader(filehandler, delimiter=delimiter)
    current_piece = 1
    current_out_path = os.path.join(
         output_path,
         output_name_template  % current_piece
    )

    current_out_writer = csv.writer(open(current_out_path, 'w'), delimiter=delimiter)

    current_limit = row_limit
    if keep_headers:
        headers = reader.next()
        current_out_writer.writerow(headers)
    for i, row in enumerate(reader):
        if i + 1 > current_limit:
            current_piece += 1
            current_limit = row_limit * current_piece
            current_out_path = os.path.join(
               output_path,
               output_name_template  % current_piece
            )
            current_out_writer = csv.writer(open(current_out_path, 'w'), delimiter=delimiter)
            if keep_headers:
                current_out_writer.writerow(headers)
current_out_writer.writerow(row)

0 个答案:

没有答案