将文件中的特定列与指定次数的另一个文件进行比较

时间:2017-07-05 04:54:52

标签: shell file awk scripting

我有2个文件说file1.txt& FILE2.TXT。

FILE1.TXT:

2017042100000000010000000943678177700000000819900000026572
2017042100000000010000000943678177700000003500000000026581
2017042100000000010000000943678177700000013450000000026591
2017042100000000010000000943678177700000011500000000026601
2017042100000000010000000943678177700000010000000000026611
2017042100000000010000000943678177700000010000000000026622
2017042100000000010000000943678177700000012855000000026632

FILE2.TXT

20170421,0000000001,00000009436781777,000000008199,0000002657,F,3,img_F1_1.tiff
20170421,0000000001,00000009436781777,000000008199,0000002657,B,3,img_F1_1.tiff
20170421,0000000001,00000009436781777,000000035000,0000002658,F,8,img_F1_2.tiff
20170421,0000000001,00000009436781777,000000134500,0000002659,F,1,img_F1_3.tiff
20170421,0000000001,00000009436781777,000000115000,0000002660,F,2,img_F1_4.tiff
20170421,0000000001,00000009436781777,000000100000,0000002661,F,1,img_F1_5.tiff
20170421,0000000001,00000009436781777,000000100000,0000002662,F,8,img_F1_6.tiff

我必须将file1.txt(最后一个字符除外)的条目与file2.txt的前5列进行比较。如果它匹配,那么我必须将file2.txt的条目存储到另一个文件,例如matched.txt。如果它没有,那么我必须将file1.txt的条目存储在另一个文件中,例如unmatched.txt。以下命令对我有用。

awk -F',' 'FILENAME=="file1.txt" {A[substr($1, 1, length($1)-1)]=substr($1, 1, length($1)-1)} FILENAME=="file2.txt"{if(A[$1$2$3$4$5]){print}}' file1.txt file2.txt > matched.txt

现在,我还有另外一个问题:

如果file1.txt(最后一个字符除外)的条目与file2.txt的前5列匹配,则必须检查file1.txt的最后一个字符(即1或2)。如果最后一个数字/字符是2,那么它必须在file2.txt中搜索2个相同的条目(前5列),其中第6列必须具有' F'对于第一次参赛和' B'第二次参赛。 例如: FILE1.TXT

2017042100000000010000000943678177700000000819900000026572

这里的最后一位是2,那么我们必须在file2.txt中找到2个条目

FILE2.TXT

20170421,0000000001,00000009436781777,000000008199,0000002657,F,3,img_F1_1.tiff
20170421,0000000001,00000009436781777,000000008199,0000002657,B,3,img_F1_1.tiff

有两个条目' F' &安培; ' B'

如果我们找到少于2个条目,那么我们必须将丢失的条目存储到文件中,名为missing.txt。我的命令适用于2个条目或0个条目,但只有一个条目不起作用。

预期输出:

missing.txt

2017042100000000010000000943678177700000010000000000026622 'B'
2017042100000000010000000943678177700000012855000000026632 'F'
2017042100000000010000000943678177700000012855000000026632 'B'

1 个答案:

答案 0 :(得分:1)

仅解决另一个问题

$ cat program.awk
BEGIN { FS="," }                      
NR==FNR {                              # file1
    if(substr($0,length($0),1)==2) {   # process only records ending in 2
        a[$0 " B"]                     # create B and ...
        a[$0 " F"]                     # ... F entries to a hash
    }
    next
}
{                                      # file2
    delete a[$1 $2 $3 $4 $5 2 " " $6]  # delete the ones we meet (* below)
}
END {                                  # in the end
    for(i in a)                        # the leftovers (in no order particular)
        print i                        # shall be outputed
}

*样本数据是这样的,file2中的每条记录都将从哈希中删除,而不只是那些包含B F记录的记录。

运行它:

$ awk -f program.awk file1 file2
2017042100000000010000000943678177700000010000000000026622 B
2017042100000000010000000943678177700000012855000000026632 B
2017042100000000010000000943678177700000012855000000026632 F