Question

我有一个目录说/var/work/X/我有数百个tsv文件这些文件名为call1.tsv, call2.tsv ,call3.tsv等。

一个tsv看起来像这样（称之为call1.tsv）：

field1 field2 field3 field4 field5
abc   command text  media  'hello'
xyz  tts  reply  sms  'hi'
mno  server reply sms 'done'
pqr command text media 'search'

我想通过每一行只选择第3列的行作为回复或field3 =回复并将其保存在另一个目录/var/work/processed/中具有相同名称的文件中最后我需要进入/var/work/processed/call1.tsv

field1 field2 field3 field4 field5
xyz  tts  reply  sms  'hi'
mno  server reply sms 'done'

我需要经历所有这样的tsv。请帮我解释一下代码

import  os, sys,glob,codecs

import csv
csv.field_size_limit(sys.maxint)
input_dir = "/var/work/X"
print input_dir

output_dir= "/var/work/processed"
print output_dir
# Get names of all tsv files
tsvs= glob.glob(os.path.join(input_dir,'*.tsv'))
for tsvfile in tsvs:
    outtsvfile=str(tsvfile).split('/')[-1]
    print outtsvfile

    data=csv.reader(open(tsvfile,'rb'),delimiter = "\t")

    try:
        with open(os.path.join(output_dir, outtsvfile)) as outputfile:
           csvwriter=csv.writer(outputfile,delimiter='\t')
        for row in data:

            if "reply" in row[2]:
                  csvwriter.writerow(row)
    except csv.Error as e:
            print "%s" %e
            print "%s" %traceback.format_exc()

我明白了：值错误：我？关闭文件的操作

Answer 1

使用csv模块：

import csv

with open('output.tsv') as output_file:
    csv_writer = csv.writer(output_file,  delimiter='\t')

with open('call1.tsv') as input_file:
    csv_reader = csv.reader(input_file, delimiter='\t')
    for row in csv_reader:
        if 'reply' == row[2]:  # I think you meant field 3
            csv_writer.writerow(row)

csv_reader.close()
csv_writer.close()

对输入目录中的所有文件执行此操作。要列出并循环输入目录中的所有tsv文件，您可以使用os模块＆＃39; listdir方法。

Answer 2

我认为你的第一个问题是

    with open(os.path.join(output_dir, outtsvfile)) as outputfile:
       csvwriter=csv.writer(outputfile,delimiter='\t')
    for row in data:    # <= bad indentation!
        if "reply" in row[2]:
              csvwriter.writerow(row)

你的for循环必须在with语句中（即缩进一个级别）;否则文件会在你尝试阅读之前关闭。

以下是一些迭代所需文件的代码：

IN_DIR = "/var/work/X/"
OUT_DIR = "/var/work/processed/"

def get_file_names(dir, ext=""):
    for fname in os.listdir(dir):
        if fname.endswith(ext) and os.path.isfile(fname):
            yield fname

def process_file(in_file, out_file):
    # your file-processing code goes here
    print("{} => {}".format(in_file, out_file))

def main():
    for fname in get_file_names(IN_DIR, ".tsv"):
        process_file(
            os.path.join(IN_DIR, fname),
            os.path.join(OUT_DIR, fname)
        )

if __name__=="__main__":
    main()

并且您的文件处理代码应该类似于

import csv
import sys

if sys.hexversion < 0x3000000:
    READ_MODE = "rb"   # Python 2.x
    WRITE_MODE = "wb"
else:
    READ_MODE = "rU"   # Python 3.x
    WRITE_MODE = "wU"

def read_csv(fname, skip_header=False, **kwargs):
    with open(fname, READ_MODE) as inf:
        incsv = csv.reader(inf, **kwargs)
        if skip_header:
            next(incsv, None)
        for row in incsv:    # or 'yield from incsv' if your Python supports it
            yield row        #

def write_csv(fname, rows, header=None, **kwargs):
    with open(fname, WRITE_MODE) as outf:
        outcsv = csv.writer(outf, **kwargs)
        if header:
            outcsv.writerow(header)
        outcsv.writerows(rows)

def process_file(in_file, out_file):
    data = read_csv(in_file, delimiter="\t")
    header = next(data, [])
    filtered = (row for row in data if row[2] == "reply")
    write_csv(out_file, filtered, header=header, delimiter="\t")

根据条件将每行csv复制到另一个tsv

2 个答案: