比较两个CSV文件并导出Python中的差异和相似之处?

时间:2018-02-10 21:52:25

标签: python pandas csv automation analysis

嘿,所有这些因为我尝试使用两个单独的CSV文件来完成我需要的工作时遇到了一些麻烦。我发现在网络上分散了一些脚本,这些脚本可以做我想要的,但不完全正确。我已经不再使用我已经尝试过的代码,因为我已经删除了不同的代码很多次以至于我已经盯着空白的py文件很长一段时间了。首先是CSV文件。

netscan.csv(包含计算机名称和序列号,具有正确的数据,具有模型)

Name        Serial   Models
computer1   serial1  model1
computer2   serial2  model2
computer3   serial3  model3
computer4   serial4  model4
...         ...

computer_list.csv(包含计算机名称和序列号,名称正确,名称不在netscan.csv中,没有型号,序列号错误)

Name        Serial    Models
computer1   serialZ
computerH   serialN/A
computer3   serialQ
computer4   serialX
computer2   serialM
computerP   serialN/A

所以我想做的是查看这两个文件,如果Name列中的值匹配,我希望它将netscan.csv中的行打印到新文件中并执行此操作每一行。之后我希望它能把所有不存在的东西(例如,计算机H不存在于netscan.csv中)并在更新的正确信息下将它们添加到新的csv中。像这样:

Name        Serial   Models
computer1   serial1  model1
computer2   serial2  model2
computer3   serial3  model3
computer4   serial4  model4
computerH   serialN/A
computerP   serialN/A

我尝试过合并,循环,写行等等,而且我现在无法完成这项工作。任何帮助将不胜感激。

编辑:@unutbu我从你的代码中获得的内容基本上是

Name     Serial    Models
computer1   serial1  model1
computer2   serial2  model2
computer3   serial3  model3
computer4   serial4  model4
computerH   serialN/A
computerP   serialN/A
computer2   serialN/A
computer3   serialN/A
computer4   serialN/A

因此,虽然几乎所有内容都是正确的,但Name中仍有重复的computer_list.csv行,如果被正确的信息替换,则需要删除这些行。所以我想查找具有重复名称的行,如果Serial是serialN / A,则删除它们。希望这更有意义。

3 个答案:

答案 0 :(得分:0)

这可能有助于各种比较

import numpy as np
import pandas as pd
#file_name = "list.xlsx"
df = pd.DataFrame({'List1':[1,2,3,4,5,5,11,4],'List 2':[3,5,6,8,9,3,4,9]}, columns=['List1', 'List 2'])#pd.read_excel(file_name, sheetname=0)
print(df)
#df.to_excel("list1.xlsx", header=True, index=False)
df['Intersect']=pd.DataFrame(np.intersect1d(df['List1'], df['List 2'])) #unique common in both
df['commonin1']=df['List1'][np.in1d(df['List1'], df['List 2'])] #non unique common items of list 1
df['commonin2']=df['List 2'][np.in1d(df['List 2'], df['List1'])] #non unique common items of list 2
df['1not2']=pd.DataFrame(np.setdiff1d(df['List1'], df['List 2'])) #in list1 but not in list 2
df['2not1']=pd.DataFrame(np.setdiff1d(df['List 2'], df['List1'])) #in list 2 but not in list1
df['1not2NU']=df['List1'][~np.in1d(df['List1'], df['List 2'])] #in list1 but not in list 2 non unique
df['2not1NU']=df['List 2'][~np.in1d(df['List 2'], df['List1'])] #in list 2 but not in list1 non unique
df['exclusive']=pd.DataFrame(np.setxor1d(df['List1'], df['List 2'])) # in a and not b + in b but not a
df=pd.concat([df,pd.DataFrame(np.union1d(df['List1'], df['List 2']), columns=['Union'])], axis=1) # unique all
df

答案 1 :(得分:0)

看看这个:

import pandas as pd
netscan = pd.read_csv('netscan.csv', header=0) # read netscan.csv and columns names are from the first row of your csv
computer_list = pd.read_csv('computer_list.csv', header=0)

# An inner merge keeps only row found in both pandas.DataFrame 
computer_match = netscan.merge(right=computer_list, how='inner', on='Name', suffixes=('netscan_', 'computer_list_'))

# Get list of Name of computers that matches
match_list = computer_match.Name.unique().tolist()

# Get characteristics of not matched computers
computer_no_match = computer_list.loc[computer_list.Name.isin(match_list), :]

# Finally, save everything to CSV
computer_match.to_csv('computer_match.csv', index=False)
computer_no_match.to_csv('computer_no_match.csv', index=False)

答案 2 :(得分:0)

您可以合并netscancomputer数据框,然后使用Serial填充SerialN/A列中的缺失值。

import pandas as pd
netscan = pd.read_csv('netscan.csv')
computer = pd.read_csv('computer_list.csv', usecols=['Name'])
for df in [netscan, computer]:
    df['Name'] = df['Name'].str.rstrip()
result = pd.merge(netscan, computer, on='Name', how='outer')
result['Serial'] = result['Serial'].fillna('SerialN/A')
result.to_csv('result.csv', index=False)
print(result)

生成包含

的CSV文件(result.csv
Name,Serial,Models
computer1,serial1,model1
computer2,serial2,model2
computer3,serial3,model3
computer4,serial4,model4
computerH,SerialN/A,
computerP,SerialN/A,