Question

没有Excel可以使用awk为csv文件做多个列的小计

File1.txt

Group,Host,1-Dec-14,2-Dec-14,3-Dec-14,4-Dec-14,5-Dec-14,6-Dec-14
GroupA,HostA,10,20,30,40,50,60
GroupB,HostB,10,20,30,40,50,60
GroupA,HostC,10,20,30,40,50,60
GroupC,HostD,10,20,30,40,50,60
GroupB,HostE,10,20,30,40,50,60
GroupC,HostF,10,20,30,40,50,60

部门档案：

Group,Host,Dept,1-Dec-14,2-Dec-14,3-Dec-14,4-Dec-14,5-Dec-14,6-Dec-14
GroupA,HostA,Finance,10,20,30,40,50,60
GroupB,HostB,HR,10,20,30,40,50,60
GroupA,HostC,Finance,10,20,30,40,50,60
GroupC,HostD,HR,10,20,30,40,50,60
GroupB,HostE,Finance,10,20,30,40,50,60
GroupC,HostF,HR,10,20,30,40,50,60

结果是

Group,Host ,1-Dec-14,2-Dec-14,3-Dec-14,4-Dec-14,5-Dec-14,6-Dec-14
GroupA Total,,20,40,60,80,100,120
GroupB Total,,20,40,60,80,100,120
GroupC Total,,20,40,60,80,100,120
GrandTotal,,60,120,180,240,300,360

Answer 1

这个程序就像你问的那样。它使用通常的策略，以适当的形式将数据读入内存，然后将数据打印到输出。

目前它从DATA读取输入并将输出发送到STDOUT。我相信你知道如何打开自己的输入和输出文件？通过在命令行上使用重定向>，可以将输出保存到您喜欢的任何文件中，但请说明是否需要更多帮助。

use strict;
use warnings;

print scalar <>;  # Copy and ignore the header

my (%data, @groups, @grand);

while (<>) {
  chomp;
  my ($group, $host, @vals) = split /,/;
  push @groups, $group unless $data{$group};
  for my $i (0 .. $#vals) {
    $data{$group}[$i] += $vals[$i];
    $grand[$i] += $vals[$i];
  }
}

for my $group (@groups) {
  print join(',', $group, '', @{ $data{$group} }), "\n";
}

print join(',', 'GrandTotal', '', @grand), "\n";

<强>输出

Group,Host,1-Dec-14,2-Dec-14,3-Dec-14,4-Dec-14,5-Dec-14,6-Dec-14
GroupA,,20,40,60,80,100,120
GroupB,,20,40,60,80,100,120
GroupC,,20,40,60,80,100,120
GrandTotal,,60,120,180,240,300,360

Answer 2

使用GNU awk实现真正的2D数组：

$ cat tst.awk
BEGIN{ FS=OFS="," }
NR==1 { print; next }
{
    for (i=3; i<=NF; i++) {
        subtot[$1][i] += $i
    }
}
END {
    for (group in subtot) {
        printf "%s%s", group, OFS
        for (i=3; i<=NF; i++) {
            printf "%s%s", OFS, subtot[group][i]
        }
        print ""
    }
}
$
$ awk -f tst.awk file
Group,Host,1-Dec-14,2-Dec-14,3-Dec-14,4-Dec-14,5-Dec-14,6-Dec-14
GroupA,,20,40,60,80,100,120
GroupB,,20,40,60,80,100,120
GroupC,,20,40,60,80,100,120

Answer 3

以下是可用作模板的Perl脚本：

use strict;
use warnings;

use Text::CSV;

my $fn = 'File1.txt';
my $csv = Text::CSV->new();
open(my $fh, '<', $fn) or die "Could not open file '$fn': $!\n";
my $header = <$fh>;
my %grp; 
my @tot;
while (my $line = <$fh>) {
    chomp $line;
    if ($csv->parse($line)) {
        my @fields = $csv->fields();
        my $key = $fields[0];
        $grp{$key} //= [(0) x ( scalar(@fields) - 2 )];
        for my $i (2..$#fields) {
            $grp{$key}->[$i - 2] += $fields[$i];
            $tot[$i - 2] += $fields[$i];
        }
    } else {
        warn "Line could not be parsed: $line\n";
    }
}
close($fh);

print $header;
for (sort keys %grp) {
    print $_ . " Total,," . join( ",", @{$grp{$_}}) . "\n";
}
print "GrandTotal,," . join( ",", @tot ) . "\n";

Answer 4

使用tee + mkfifo复制管道输出。适用于大型文件

$>cat foo 
foo,10,foo_a
bar,10,bar_a
foo,20,foo_b
bar,20,bar_b
foo,69,foo_c
bar,69,bar_c

$>mkfifo fifotmp

$>awk -F',' ' 
BEGIN {T="\t" }

pass==1 {
    sum_first_col[ $1 ] += $2;
    sum_all += $2;
} 

pass==2 { 
    percentage_itm=100 * $2 / sum_first_col[ $1 ];
    percentage_all=100 * $2 / sum_all;

    print $1 T $2 T $3 T percentage_itm T percentage_all;

}' pass=1 <(cat foo | tee -a fifotmp) pass=2 <(cat fifotmp ) 

OR 

$>cat foo | tee -a fifotmp | awk -F',' ' 
    BEGIN {T="\t" }

    pass==1 {
        sum_first_col[ $1 ] += $2;
        sum_all += $2;
    } 

    pass==2 { 
        percentage_itm=100 * $2 / sum_first_col[ $1 ];
        percentage_all=100 * $2 / sum_all;

        print $1 T $2 T $3 T percentage_itm T percentage_all;

    }' pass=1 - pass=2 <(cat fifotmp ) 

foo 10  foo_a   10.101  5.05051
bar 10  bar_a   10.101  5.05051
foo 20  foo_b   20.202  10.101
bar 20  bar_b   20.202  10.101
foo 69  foo_c   69.697  34.8485
bar 69  bar_c   69.697  34.8485

使用awk为多列的小计

4 个答案: