Question

我有.csv文件形式的一系列随机模拟的输出，看起来像这样：

Run,ID,Var
1,1,7
1,2,9
1,3,4
2,1,3
2,2,4
2,3,8

等

除此之外，我还有另一个数据文件，也就是.csv，格式如下：

ID, Var2, Var3
1,0.89,0.10
2,0.45,0.98
3,0.27,0.05
4,0.98,0.24

注意：数据文件中有一些不出现在模拟文件中的值。我希望忽略这些。

我想要做的是编写一个脚本，从第一个.csv文件中获取每个值ID，并找到Var2和Var3并将它们放在一起，最终得到如下内容：

Run, ID, Var, Var2, Var3
1,1,7,0.89,0.10
1,2,9,0.45,0.98
1,3,4,0.27,0.05
2,1,3,0.89,0.10
2,2,4,0.45,0.98
2,3,8,0.27,0.05

有关方法的任何建议吗？我承认这是我对Python中数据处理理解的极限。我对在SAS中如何做到这一点有了一个公平的认识，但我更喜欢将它作为一种单语言任务，以便将它们作为单个脚本进行处理。

Answer 1

ouput.csv：

Run, ID, Var
1, 1, 7
1, 2, 9
1, 3, 4
2, 1, 3
2, 2, 4
2, 3, 8

data.csv：

ID, Var2, Var3
1, 0.89, 0.10
2, 0.45, 0.98
3, 0.27, 0.05
8, 0.4, 0.5

注意即使我们在data.csv中有条目，而不是ouput.csv中的条目，它也不会影响最终结果，因为当我们解析output.csv时，我们只查找ID的那个我们从output.csv知道了，虽然相反的情况并非如此在minimun的data.csv必须包含output.csv中的所有ID，但如果需要，可以很容易地处理。

代码：

import csv
from pprint import pprint 

data = dict([(row['ID'], row) for row in csv.DictReader(open('data.csv', 'rb'), skipinitialspace = True)])
values = []
for row in csv.DictReader(open('output.csv', 'rb'), skipinitialspace = True):
    values.append(row)
    values[-1].update(data[row['ID']])

>>> pprint(values)
[{'ID': '1', 'Run': '1', 'Var': '7', 'Var2': '0.89', 'Var3': '0.10'},
 {'ID': '2', 'Run': '1', 'Var': '9', 'Var2': '0.45', 'Var3': '0.98'},
 {'ID': '3', 'Run': '1', 'Var': '4', 'Var2': '0.27', 'Var3': '0.05'},
 {'ID': '1', 'Run': '2', 'Var': '3', 'Var2': '0.89', 'Var3': '0.10'},
 {'ID': '2', 'Run': '2', 'Var': '4', 'Var2': '0.45', 'Var3': '0.98'},
 {'ID': '3', 'Run': '2', 'Var': '8', 'Var2': '0.27', 'Var3': '0.05'}]
>>>

现在要保存回csv文件。

fieldnames = ['Run', 'ID', 'Var', 'Var2', 'Var3']
f = open('combined.csv', 'wb')
csvwriter = csv.DictWriter(f, fieldnames = fieldnames)
csvwriter.writerow(dict((fn,fn) for fn in fieldnames)) # 2.7 has writeheader, which is cleaner
[csvwriter.writerow(row) for row in values]
f.close()


$ cat combined.csv 
Run,ID,Var,Var2,Var3
1,1,7,0.89,0.10
1,2,9,0.45,0.98
1,3,4,0.27,0.05
2,1,3,0.89,0.10
2,2,4,0.45,0.98
2,3,8,0.27,0.05

我希望这会有所帮助。

Answer 2

不使用csv模块的解决方案：

with open('data.txt') as f1,open('data1.txt') as f2,open('data3.txt','w') as f3:
    header1=f1.readline().strip().split(',') #header from file 1 i.e Run,ID,Var

    header2=f2.readline().strip().split(',')[1:] #header from file 2 ,i.e Var2, Var3

    dic={x.strip().split(',')[0]:x.strip().split(',')[1:] for x in f2 if x.strip()} #use dict to save data as per ID from file 2

    f3.write(','.join((header1+header2))+'\n') #write the new header(header1+header2) to file 3

    for x in f1: 
        f3.write(x.strip()+','+','.join(dic[x.split(',')[1]])+'\n') #fetch results from dic as per the ID obtained from the current line in data.txt

<强>输出： data3.txt包含

Run,ID,Var, Var2, Var3
1,1,7,0.89,0.10
1,2,9,0.45,0.98
1,3,4,0.27,0.05
2,1,3,0.89,0.10
2,2,4,0.45,0.98
2,3,8,0.27,0.05

Answer 3

简单易行：

f = open('one.csv', 'r')
one = f.read()
f.close()

f = open('two.csv', 'r')
two = f.read()
f.close()

one = one.split('\n')[1:-1]
two = two.split('\n')[1:-1]
output = 'Run, ID, Var, Var2, Var3\n'

for o in one:
  for t in two:
    row = t.split(',')
    if o.split(',')[1] == row[0]:
      output += '%s,%s,%s\n' % (o, row[1], row[2])

# or save it to a file
print output

在Python中合并一个多CSV文件

3 个答案: