Question

EG。档案1

"ACCOUNT_ID","CTN","NAME","GATEWAY_GUID","DEVICE_GUID","CATALOG_ID","FW_VERSION","DATE_CREATED","STATUS_ID","LOCATION_CODE","BAN","Market_Area","State","IMEI","HW_MODEL"
"306875",="9404653975","14-052917 14-052917","313A0B72E3E440DD8687BD681E55FB03","0",="000010000010004","FW: 1.04.122, JVM: Oracle Corporation 1.7.0_72-ea, OS: Linux 2.6.33.5","06/24/2014 14:32:38","0",="0003013034",="177046772949","DLS","TX",="351612051721824","Cisco DLC-100"
"306875",="9404653975","14-052917 14-052917","7EED6EE61F0949EE99554D4D4F09E4FE","ACFF000001",="000010901000004","1.2.14","06/24/2014 21:28:17","0",="",="177046772949","DLS","TX",="351612051721824",""
"306875",="9404653975","14-052917 14-052917","D57DAE988A1C482EA3217312EDC7466E ","ACFF010904",="000010907000004","","12/16/2015 23:39:21","0",="",="177046772949","DLS","TX",="351612051721824",""

文件2

account,ban,ctn,first_name,last_name,device_gateway_guid,device_id,device_cat_id,IMEI,device_fw_vrsn,date_created,device_status,subscription_created,subscription_name,subscription_market,date
DL!813269 , 418069632891 , undefined , MUHAMMAD , ANJUM , 313A0B72E3E440DD8687BD681E55FB03, ACFF010904 , 00010907000004 , 351612054025777 ,  , 2015-12-18 19:45:31 , 0 , undefined , [object Object] , WAS , undefined
DL!782477 , 523266997720 , undefined , SAM , MAURER , 7EED6EE61F0949EE99554D4D4F09E4FE , 0 , 00010000010004 , 351612053801194 , FW: 1.04.122, JVM: Oracle Corporation 1.7.0_72-ea, OS: Linux 2.6.33.5 , 2015-12-18 19:02:27 , 0 , undefined , [object Object] , FLP , 2015-07-29 09:07:22
DL!926875 , 578172109430 , undefined , TRACY , BUSH , D57DAE988A1C482EA3217312EDC7466E , 0 , 00010000010004 , 351612054481798 , FW: 1.04.122, JVM: Oracle Corporation 1.7.0_72-ea, OS: Linux 2.6.33.5 , 2016-01-23 16:09:21 , 0 , undefined , [object Object] , GLF , 2015-11-06 02:26:31
"306875",="9404653975","14-052917 14-052917","313A0B72E3E440DD8687BD681E55FB03","0",="313A0B72E3E440DD8687BD681E55FB03","FW: 1.04.122, JVM: Oracle Corporation 1.7.0_72-ea, OS: Linux 2.6.33.5","06/24/2014 14:32:38","0",="0003013034",="177046772949","DLS","TX",="351612051721824","Cisco DLC-100"

我想将file1的“GATEWAY_GUID”列与file2的device_gateway_guid列进行比较。输出应该是file1中看不到的file2的所有记录。

例如：如果文件1有10条记录而文件2有5000条记录，其中5条记录与文件1相同。那么我的输出文件应显示缺少文件1中5个未在file2中看到的列值。

到目前为止，我尝试了以下脚本但没有用。任何帮助表示赞赏。

awk 'NR==FNR{c[$6]++;next};c[$4] == 0' s2_1.csv s1_1.csv > compares1s2.csv

Answer 1

第一步：解析file1，以便只获得相关信息：

grep -v "GATEWAY_GUID" file1 | cut -d'"' -f8 
# or more difficult to read
sed -n '2,$ s/\([^,]*,\)\{3\}"\([^"]*\).*/\2/p' file1

根据您输入的可能格式，您可能需要更改此信息输出第一步

313A0B72E3E440DD8687BD681E55FB03
7EED6EE61F0949EE99554D4D4F09E4FE
D57DAE988A1C482EA3217312EDC7466E

现在你想做类似

的事情

grep -Ev "313A0B72E3E440DD8687BD681E55FB03|7EED6EE61F0949EE99554D4D4F09E4FE|D57DAE988A1C482EA3217312EDC7466E" file2

grep可以选择从文件中读取搜索字符串，因此可以将其更改为

grep -EvFxf tempfile_with_search_keys file2

您可以将临时文件＆＃34;保留在内存中＃34;使用＆＃34;流程替换＆＃34;：

grep -EvFxf <(some_command) file2

对于some_command，我可以使用第一个命令解析file1

grep -EvFxf <(grep -v "GATEWAY_GUID" file1 | cut -d'"' -f8) file2

Answer 2

你需要这样的东西：

awk '
    BEGIN   { FS="[,\"]";}
    FNR==NR {  
      a[ $11 ] ++;
      next;
    }

    { tmp = gensub(/[ ]+/, "", "g", $6); 
      if( !( tmp in a)  ) print tmp; 
    }
' ff1.csv ff2.csv

假设文件名是ff1.csv和ff2.csv
问题中提供的示例在第二个文件中有空格，有些行有引号，有些没有引号
引号在FS
使用gensub
在linux上用GNU awk 4测试

Answer 3

有点一次，但它适用于您的示例数据：

 awk -F, 'NR==FNR {
              s=$4; 
              gsub("\"", "", s); 
              gsub(/[[:space:]]/,"", s); 
              arr[s]++} 
          NR>FNR && FNR>1 {
              s=$6; 
              gsub("\"", "", s); 
              gsub(/[[:space:]]/,"", s); 
              sub(/=/,"",s); 
              if (!(s in arr))
                  print s;
              }' file1 file2

Answer 4

如果这是你想要的或者没有，你没有发布你想要的输出，但是你还是去了：

$ awk -F'[ "]*,[ "]*' 'NR==FNR{a[$4];next} (FNR==1) || ($6 in a)' file1 file2
account,ban,ctn,first_name,last_name,device_gateway_guid,device_id,device_cat_id,IMEI,device_fw_vrsn,date_created,device_status,subscription_created,subscription_name,subscription_market,date
DL!813269 , 418069632891 , undefined , MUHAMMAD , ANJUM , 313A0B72E3E440DD8687BD681E55FB03, ACFF010904 , 00010907000004 , 351612054025777 ,  , 2015-12-18 19:45:31 , 0 , undefined , [object Object] , WAS , undefined
DL!782477 , 523266997720 , undefined , SAM , MAURER , 7EED6EE61F0949EE99554D4D4F09E4FE , 0 , 00010000010004 , 351612053801194 , FW: 1.04.122, JVM: Oracle Corporation 1.7.0_72-ea, OS: Linux 2.6.33.5 , 2015-12-18 19:02:27 , 0 , undefined , [object Object] , FLP , 2015-07-29 09:07:22
DL!926875 , 578172109430 , undefined , TRACY , BUSH , D57DAE988A1C482EA3217312EDC7466E , 0 , 00010000010004 , 351612054481798 , FW: 1.04.122, JVM: Oracle Corporation 1.7.0_72-ea, OS: Linux 2.6.33.5 , 2016-01-23 16:09:21 , 0 , undefined , [object Object] , GLF , 2015-11-06 02:26:31

awk比较两个文件中的列，如果在file1中没有看到，则打印文件2列

4 个答案: