Python中多个文件的求和和平均值

时间:2014-03-25 18:33:41

标签: python add mean

我有一个关于用Python输出的问题。

我有以下3个文件作为输入数据:

文件A

abc with-1-rosette-n    2
abc with-1-tyre-n   1
abc with-1-weight-n 2

档案B

def with-1-rosette-n 1
def with-1-tyre-n   2
def about-bit-n 1

文件C

ghi with-1-rosette-n  2
ghi as+n-produce-v   1
ghi then-damage-v  1

我首先尝试创建一个脚本,在其中我将考虑Col 2交叉点的值(Col 3)的总和。

这很好 - 正确输出所有行。

我尝试修改脚本以考虑Col 2交叉的Col 3值的平均值,这就是我遇到麻烦的地方。

基本上,脚本不输出交集的行。

脚本A

def sumVectors(classA_infile, classB_infile, outfile):

        class_dictA = {}

        with open(classA_infile, "rb") as opened_infile_A:
                for line in opened_infile_A:
                        items = line.split()
                        classA, feat, valuesA = items[:3]
                        class_dictA[feat] = float(valuesA)


        class_dictB = {}

        with open(classB_infile, "rb") as opened_infile_B:
                for line in opened_infile_B:
                        items = line.split()
                        classB, feat, valuesB = items[:3]
                        class_dictB[feat] = float(valuesB)

        with open(outfile, "wb") as output_file:
                for key in class_dictA:
                        if key in class_dictB:
                                weight = (class_dictA[key] + class_dictB[key])/2
                                outstring = "\t".join([classA + "-" +  classB, key, str(weight)])
                                print outstring
                        else:
                                weight = class_dictA[key]
                                outstring = "\t".join([classA + "-" +  classB, key, str(weight)])
                output_file.write(outstring + "\n")

                for key in class_dictB:
                        if key not in class_dictA:
                                weight = class_dictB[key]
                                outstring = "\t".join([classA + "-" + classB, key, str(weight)])
                                output_file.write(outstring + "\n")

当我尝试合并第三个文件时:我遇到了一个关键问题。在这里,我试图查看文件C中的密钥是否也在文件A和B中,如果是,我们取这三个文件的平均值。在这种情况下,它正在给我一个关键错误,就在它进入第一个if块时,所以我很难解决这个问题。

以下是考虑3个文件的脚本示例。

脚本B

def sumVectors(classA_infile, classB_infile, classC_infile, outfile):

        class_dictA = {}

        with open(classA_infile, "rb") as opened_infile_A:
                for line in opened_infile_A:
                        items = line.split()
                        classA, feat, valuesA = items[:3]
                        class_dictA[feat] = float(valuesA)


        class_dictB = {}

        with open(classB_infile, "rb") as opened_infile_B:
                for line in opened_infile_B:
                        items = line.split()
                        classB, feat, valuesB = items[:3]
                        class_dictB[feat] = float(valuesB)

        class_dictC = {}

        with open(classC_infile, "rb") as opened_infile_C:
                for line in opened_infile_C:
                        items = line.split()
                        classC, feat, valuesC = items[:3]
                        class_dictC[feat] = float(valuesC)

        with open(outfile, "wb") as output_file:
                for key in class_dictC:
                        if key in class_dictA and class_dictB:
                                weight = (class_dictA[key] + class_dictB[key]+ class_dictC[key])/3
                                outstring = "\t".join([classA + "-" +  classB + "-" +  classC, key, str(weight)])
                                print outstring
                        else:
                                weight = class_dictC[key]
                                outstring = "\t".join([classA + "-" +  classB + "-" +  classC,  key, str(weight)])
                                output_file.write(outstring + "\n")

在脚本A的情况下,所需的输出将是:

(我们考虑Col 2中共同元素的平均值):

abc-def with-1-rosette-n    1.5
abc-def with-1-tyre-n   1
abc-def with-1-weight-n 2
def with-1-tyre-n   2
def about-bit-n 1

并且在脚本B的情况下,所需的输出将是:

档案B. (我们考虑Col 2中所有3个文件的共同元素的平均值):

abc-def-ghi with-1-rosette-n    1.667
abc-def-ghi with-1-tyre-n   1.5
abc-def-ghi with-1-weight-n 2
abc-def-ghi with-1-rosette-n 1.5
abc-def-ghi about-bit-n 1
abc-def-ghi as+n-produce-v   1
abc-def-ghi then-damage-v  1

任何人都可以帮我看看我哪里出错了,我不确定解决它的最好的pythonic路线...... 感谢。

1 个答案:

答案 0 :(得分:2)

from collections import defaultdict

# Because you are looking for a union of files, we can treat
#  the input data as a simple concatenation of all input files;
# If you were after intersection, we would have to deal with
#  each input file separately.
def chain_from_files(*filenames):
    for fname in filenames:
        with open(fname, "rb") as inf:
            for line in inf:
                yield line

# get the key and all related data for each line
def get_item(line):
    row = line.split()
    return row[1], (row[0], int(row[2]))    # <= returns a tuple ('abc', 2)

# iterate through the input,
# collect a list of related values for each key
def collect_items(lines, get_item):
    result = defaultdict(list)
    for line in lines:
        key, value = get_item(line)
        result[key].append(value)
    return result

# make an output-string for each key
# and its list of related values
def show_item(key, values):
    classes, nums = zip(*values)          # <= unpacks the tuples
    classes = '-'.join(sorted(set(classes)))
    average = float(sum(nums)) / len(nums)
    return "{} {} {}\n".format(classes, key, average)

def main():
    lines = chain_from_files(classA_infile, classB_infile, classC_infile)
    data  = collect_items(lines, get_item)

    with open(outputfile, "wb") as outf:
        for key,value in data.items():
            outf.write(show_item(key, value))

if __name__=="__main__":
    main()

作为输出

ghi then-damage-v 1.0
abc-def with-1-tyre-n 1.5
abc-def-ghi with-1-rosette-n 1.66666666667
ghi as+n-produce-v 1.0
abc with-1-weight-n 2.0
def about-bit-n 1.0