我如何解析linux中的大文件

时间:2014-06-17 20:00:22

标签: linux bash awk

我是Linux的初学者。我有以下平面文件test.txt

Iteration 1   
Telephony   


Pass/Fail

5.1.1.1   voiceCallPhoneBook   50   45
5.1.1.4   voiceCallPhoneHistory   50   49
5.1.1.7   receiveCall   100   100
5.1.1.8   deleteContacts   20   19
5.1.1.9   addContacts   20   20
Telephony   16:47:42
Messaging   


Pass/Fail

5.1.2.3   openSMS   50   49
5.1.2.1   smsManuallyEntryOption   50   50
5.1.2.2   smsSelectContactsOption   50   50
Messaging   03:26:31
Email   


Pass/Fail

Email   00:00:48
Email   


Pass/Fail

Email   00:00:40
PIM   


Pass/Fail

5.1.6.1   addAppointment   5   0
5.1.6.2   setAlarm   1   0
5.1.6.3   deleteAppointment   5   0
5.1.6.4   deleteAlarm   1   0
5.1.6.5   addTask   1   0
5.1.6.6   openTask   1   0
5.1.6.7   deleteTask   1   0
PIM   00:03:06
Multi-Media   

teration 2   
Telephony   


Pass/Fail

5.1.1.1   voiceCallPhoneBook   50   47
5.1.1.4   voiceCallPhoneHistory   50   50
5.1.1.7   receiveCall   100   100
5.1.1.8   deleteContacts   20   20
5.1.1.9   addContacts   20   20
Telephony   04:02:05
Messaging   


Pass/Fail

5.1.2.3   openSMS   50   50
5.1.2.1   smsManuallyEntryOption   50   50
5.1.2.2   smsSelectContactsOption   50   50
Messaging   03:20:01
Email   


Pass/Fail

Email   00:00:47
Email   


Pass/Fail

Email   00:00:40
PIM   


Pass/Fail

5.1.6.1   addAppointment   5   5
5.1.6.2   setAlarm   1   1
5.1.6.3   deleteAppointment   5   5
5.1.6.4   deleteAlarm   1   1
5.1.6.5   addTask   1   1
5.1.6.6   openTask   1   1
5.1.6.7   deleteTask   1   1
PIM   00:09:20
Multi-Media   

我想计算文件中特定单词的出现次数例如:如果我使用" voiceCallPhoneBook"它显示为2次。

我可以使用

cat reports.txt | grep "5.1.1.4" | cut -d' ' -f1,4,7,10 |

运行此脚本后,我得到如下输出

5.1.1.4 voiceCallPhoneBook  50  45
5.1.1.4 voiceCallPhoneBook  50  47

这是一个非常大的文件,我想利用bash / awk脚本的循环,并找到第3和第4列值的平均值。我正在努力用bash脚本编写。可以理解,有人可以为它提供解决方案。

由于

2 个答案:

答案 0 :(得分:1)

#!/usr/bin/awk -f
BEGIN{
c3 = 0
c4 = 0
count = 0
}

/voiceCallPhoneBook/{
    c3 = c3 + $3;
    c4 = c4 + $4;
    count++;
}

END{
 print "column 3 avg: " c3/count
 print "column 4 avg: " c4/count
}

1)将其保存在文件中,例如countVoiceCall.awk

2)awk -f countVoiceCall.awk sample.txt

输出:

column 3 avg: 50
column 4 avg: 46

简要解释一下:

a.    BEGIN{...} block uses for variables initialization
b.    /PATTERN/{...}  blocks uses to search your keyword, for example "voiceCallPhoneBook"
c.    END{...} block uses for print the results

答案 1 :(得分:0)

这将搜索包含5.1.1.4的行 制作第3和第4列的记录 然后将它们全部打印出来

awk '/^5\.?\.?\.?/ {a[$1" " $2] +=$3 ; b[$1" " $2] +=$4 }
END{ for (k in a){
      printf("%-50s%-10i%-10i\n",k,a[k],b[k])}
}' $1

今天早些时候复制的是Parse the large test files using awk

标题avg和Occurence计数并且格式化得更整齐以便于阅读:)

awk 'BEGIN{
            printf("%-50s%-10s%-10s%-10s\n","Name","Col3 Tot","Col4 Tot","Ocurr")
}

/^5\.?\.?\.?/ {
        count++
        c3 = c3 + $3
        c4 = c4 + $4
        a[$1" " $2] +=$3
        b[$1" " $2] +=$4
        c[$1" " $2]++
}

END{
        for (k in a)
                {printf("%-50s%-10i%-10i%-10i\n",k,a[k],b[k],c[k])}

                print "col3 avg: " c3/count "\ncol4 avg: " c4/count
}' $1