比较python中两个csv文件中的两列

时间:2019-07-15 15:48:40

标签: python python-3.x pandas csv

我有两个具有相同列名的csv文件:

  • 在file1中,我得到了所有进行测试的人员以及所有状态(通过/未通过)
  • 在file2中,我只有那些错过考试的人

我想比较file1.column1和file2.column1

  • 如果它们匹配,则比较file1.column4和file2.column4
  • 如果它们不同,请从file2删除项目行

我不知道该怎么做。 我看着大熊猫的东西,但没有做任何有用的事情

我所拥有的是:

file1.csv:

name;DOB;service;test status;test date
Smith;12/12/2012;compta;Missed;01/01/2019
foo;02/11/1989;office;Passed;01/01/2019
bar;03/09/1972;sales;Passed;02/03/2018
Doe;25/03/1958;garage;Missed;02/04/2019
Smith;12/12/2012;compta;Passed;04/05/2019

file2.csv:

name;DOB;service;test status;test date
Smith;12/12/2012;compta;Missed;01/01/2019
Doe;25/03/1958;garage;Missed;02/04/2019

我想要得到的是:

file1.csv:

name;DOB;service;test status;test date
Smith;12/12/2012;compta;Missed;01/01/2019
foo;02/11/1989;office;Passed;01/01/2019
bar;03/09/1972;sales;Passed;02/03/2018
Doe;25/03/1958;garage;Missed;02/04/2019
Smith;12/12/2012;compta;Passed;04/05/2019

file2.csv:

name;DOB;service;test status;test date
Doe;25/03/1958;garage;Missed;02/04/2019

1 个答案:

答案 0 :(得分:0)

所以首先您必须打开:

    import pandas as pd
    df1 = pd.read_csv('file1.csv',delimiter=';')
    df2 = pd.read_csv('file2.csv',delimiter=';')

由于发现空白而处理数据框

    df1.columns= df1.columns.str.strip()
    df2.columns= df2.columns.str.strip()
    # Assuming only strings
    df1 = df1.apply(lambda column: column.str.strip())
    df2 = df2.apply(lambda column: column.str.strip())

预期的解决方案,假设您的名字是UNIQUE。

合并文件

    new_merged_df = df2.merge(df1[['name','test status']],'left',on=['name'],suffixes=('','file1'))

DataFrame结果:

    name         DOB service test status   test date test statusfile1
0  Smith  12/12/2012  compta      Missed  01/01/2019           Missed
1  Smith  12/12/2012  compta      Missed  01/01/2019           Passed
2    Doe  25/03/1958  garage      Missed  02/04/2019           Missed

根据要求进行过滤,并删除名称具有不同测试状态的行。

    filter = new_merged_df['test status'] != new_merged_df['test statusfile1']
    # Check if there is different values
    if len(new_merged_df[filter]) > 0:
       drop_names = list(new_merged_df[filter]['name'])
       # Removing the values that we don't want
       new_merged_df = new_merged_df[~new_merged_df['name'].isin(drop_names)]

删除列并存储

    # Saving as a file with the same schema as file2
    new_merged_df.drop(columns=['test statusfile1'],inplace=True)
    new_merged_df.to_csv('file2.csv',delimiter=';',index=False)

结果

  name         DOB service test status   test date
2  Doe  25/03/1958  garage      Missed  02/04/2019