使用一个代码在Python中处理多个文件

时间:2017-04-03 10:41:25

标签: python file csv pandas

我的代码遇到了一些麻烦。我想执行以下操作:我需要处理大约30个csv文件,我希望将结果放在一个单独的csv文件中。到目前为止,我有以下代码(示例):

=SUMPRODUCT(((C2:C22="Incurred")+(C2:C22="Expected")+(C2:C22="Deferred"))*B2:B22)

所以这段代码可以完成我想要的单个文件。我写了另一个脚本来浏览所有文件:

import pandas as pd
import csv as csv

df = pd.read_csv ("file.csv", 
                  delimiter=",",skiprows=1)

datamean10=df[61:240].mean()
datamean15=df[241:420].mean()

list10=[]
list15=[]

list10.append(datamean10.clip(0))
list15.append(datamean15.clip(0))

csvfile = "C:/Users/bla/bla/list10.csv"
with open(csvfile, 'w') as output:
    writer = csv.writer(output, lineterminator='\n')
    writer.writerows(list10)

csvfile = "C:/Users/bla/bla/list15.csv"
with open(csvfile, 'w') as output:
    writer = csv.writer(output, lineterminator='\n')
    writer.writerows(list15)

毋庸置疑,此脚本会再次删除列表,并以空白列表开头,而我想将结果附加到同一列表中。因此,生成的文件对于每个处理的文件都有一行平均值。有人能告诉我怎么做吗?非常感谢你提前。

2 个答案:

答案 0 :(得分:0)

这应该是一个有效的组合。

import pandas as pd
import csv as csv

df = pd.read_csv ("file.csv", 
                  delimiter=",",skiprows=1)

datamean10=df[61:240].mean()
datamean15=df[241:420].mean()

list10=[]
list15=[]

list10.append(datamean10.clip(0))
list15.append(datamean15.clip(0))

csvfile = "C:/Users/bla/bla/list10.csv"
with open(csvfile, 'w') as output:
    writer = csv.writer(output, lineterminator='\n')
    writer.writerows(list10)

import sys, os
for root, dir, files in os.walk('C:/Users/bla/bla/bla'):
    for file in files:
        if file.endswith('.csv'):
            #csvfile = "C:/Users/bla/bla/list15.csv"
            with open(file, 'w') as output:
                writer = csv.writer(output, lineterminator='\n')
                writer.writerows(list15)

答案 1 :(得分:0)

只需在内部脚本中以附加模式打开输出文件:

with open(csvfile, 'a') as output:

然后仅在主脚本中截断:

with open("C:/Users/bla/bla/list10.csv", 'w') as output:
    pass
with open("C:/Users/bla/bla/list15.csv", 'w') as output:
    pass


for root, dir, files in os.walk('C:/Users/bla/bla/bla'):
    ...

但是使用system从另一个Python脚本调用Python脚本是不好的。我最好在函数中编写内部脚本的工作部分,然后在导入后从外部脚本中使用它:

内部脚本(averagelist.py)

import pandas as pd
import csv as csv

def average(file):

    df = pd.read_csv (file, 
                      delimiter=",",skiprows=1)
    ...    
    with open(csvfile, 'w') as output:
        writer = csv.writer(output, lineterminator='\n')
        writer.writerows(list15)

外部脚本:

import os
import averagelist

with open("C:/Users/bla/bla/list10.csv", 'w') as output:
    pass
with open("C:/Users/bla/bla/list15.csv", 'w') as output:
    pass
for root, dir, files in os.walk('C:/Users/bla/bla/bla'):
    for file in files:
        if file.endswith('.csv'):
            averagelist.average(root + '\\' + file))