shell脚本如何按行合并三个文件并计算一些值,满足一些条件

时间:2015-12-23 19:27:51

标签: shell awk

while read line1
do
    while read line2
    do
        while read line3 
        do  echo "$line1, $line2, $line3" | awk -F , ' $1==$5 && $6==$11 && $10==$12 {print $1,",",$2,",",$3,",",$4,",",$6,",",$7,",",$8,",",$9,",",$10,",",$13,",",$14,",",$15}' >>out.txt
    done < grades.csv
    done < subjects.csv
done < students.csv

在这段代码中,我按行合并三个文件(交叉产品),如果任何合并的行符合条件“$ 1 == $ 5&amp;&amp; $ 6 == $ 11&amp;&amp; $ 10 == $ 12”,我就是将它们打印在输出文件中。 现在我的问题是,如果符合条件,我想继续为每次迭代添加“$ 13”字段值。

我该怎么做?请帮忙。 这是示例文件。

gardes.csv包含行:

1,ARCH,1,90,very good,80   
1,ARCH,2,70,good,85    
1,PLNG,1,89,very good,85

subjects.csv包含以下行:

1,ARCH,Computer Architecture,A,K.Gose   
1,PLNG,Programming Languages,A,P.Yang    
1,OS,Operating System,B,K.Gopalan    
2,ARCH,Computer Architecture,A,K.Gose

students.csv包含以下行:

1,pankaj,vestal,986-654-32   
2,satadisha,binghamton,879-876-54    
5,pankaj,vestal,986-654-32    
6,pankaj,vestal,986-654-31

这是预期的输出:

ARCH  1  pankaj     vestal      986-654-32  Computer Architecture  A  K.Gose  1  1  90  very good  80
ARCH  1  pankaj     vestal      986-654-32  Computer Architecture  A  K.Gose  1  2  70  good       85
ARCH  2  satadisha  binghamton  879-876-54  Computer Architecture  A  K.Gose  1  1  90  very good  80
ARCH  2  satadisha  binghamton  879-876-54  Computer Architecture  A  K.Gose  1  2  70  good       85
PLNG  1  pankaj     vestal      986-654-32  Programming Languages  A  P.Yang  1  1  89  very good  85

另外我需要在另一个可以写入文件的shell变量中加上(90 + 70 + 90 + 70 + 89)的总和。

2 个答案:

答案 0 :(得分:1)

您可以使用join创建扩展数据,并使用awk进行操作。

$ join -t, -1 5 -2 2 <(join -t, -j 1 file3 file2 | sort -t, -k5,5) file1 | column -s, -t

ARCH  1  pankaj     vestal      986-654-32  Computer Architecture  A  K.Gose  1  1  90  very good  80
ARCH  1  pankaj     vestal      986-654-32  Computer Architecture  A  K.Gose  1  2  70  good       85
ARCH  2  satadisha  binghamton  879-876-54  Computer Architecture  A  K.Gose  1  1  90  very good  80
ARCH  2  satadisha  binghamton  879-876-54  Computer Architecture  A  K.Gose  1  2  70  good       85
PLNG  1  pankaj     vestal      986-654-32  Programming Languages  A  P.Yang  1  1  89  very good  85

或者,您也可以在awk中进行联接,从而消除while循环。

如果您想以$ 11添加值。

$ join -t, -1 5 -2 2 <(join -t, -j 1 file3 file2 
    | sort -t, -k5,5) file1 | awk -F, '{sum+=$11} END{print sum}'

将结果分配给shell变量

$ sum=$(join ... )

答案 1 :(得分:1)

假设您已加入列以形成TSV(制表符分隔值)文件或流,并且列$ k1,$ k2和$ k3(在该文件或流中)构成键,并且您想要要在连接中对$ s进行求和,这里是awk命令,您可以使用它来形成密钥和总和的TSV列表:

awk -F\\t -v k1=$k1 -v k2=$k2 -v k3=$k3 '
  BEGIN{t=OFS="\t"}
  { key=$k1 t $k2 t $k3; sum[key]+=$s }
  END {for (key in sum) {print key, sum[key] } }'

(使用awk处理可能包含逗号的CSV文件会遇到麻烦,所以我已经说明了如何将awk与标签一起使用。)