将CSV文件导入数据框

Question

在此方面寻求帮助。我不知道为什么，但是删除重复项不起作用，尝试了lambda循环。我仍然无能为力，将删除输出中的多个重复项。

# Import files for use in the program:
import pandas as pd
import os
import matplotlib.pyplot as plt
import matplotlib.font_manager as fm
import matplotlib.dates as mdates
import numpy as np
import csv

将CSV文件导入数据框

Crash_Data_df = pd.read_csv("crash_data.csv",encoding='UTF-8')

#split date column
Crash_Data_df[["Day", "Month", "DayNum","Time","Zone","Year"]] = 
Crash_Data_df["Date"].str.split(" ", n = 6, expand = True) 

#define max and min dates
d_max=Crash_Data_df["Date"].min()
d_min=Crash_Data_df["Date"].max()

#split name column
Crash_Data_df[["A","B"]] = Crash_Data_df["Name"].str.split("_|2018100", n = 
2, expand = True) 

#Drop time zone
Crash_Data_df.drop(['Zone'], axis = 1, inplace = True)
Crash_Data_df.reset_index(drop=True)

# group by unnamed column
Crash_Data_df = Crash_Data_df.loc[Crash_Data_df['Unnamed: 0'] == 0, :]

#del columns 
del Crash_Data_df['Unnamed: 0']
del Crash_Data_df['Name']
del Crash_Data_df['A']

Crash_Data_df = Crash_Data_df.loc[Crash_Data_df['B'] != 9954815, :]

Crash_Data_df = Crash_Data_df.dropna(how='any')
Crash_Data_df.drop_duplicates(subset=['Time'], keep=False)
Crash_Data_df.sort_values(by=['B'])
Crash_Data_df.reset_index(drop=True)
Crash_Data_df = Crash_Data_df.rename(columns=    
{'B':'ID','Date':'DATE','Direction':'DIRECTION','Road':'ROAD',
'Location':'LOCATION','Event':'EVENT','Day':'DAY',
'Month':'MONTH','DayNum':'DAYNUM','Time':'TIME','Year':'YEAR'})

Crash_Data_df.set_index('ID', inplace=True,drop=True)

Crash_Data_df.to_csv("crash_data_check.csv", index=False, header=True)
Crash_Data_df.drop_duplicates()

Crash_Data_df.groupby("ID").filter(lambda x: len(x) > 1)
Crash_Data_df.head()

ID重复项不会被删除。尝试过不同的列没关系..输出看起来像：

    DATE    DIRECTION   ROAD    LOCATION    EVENT   DAY MONTH   DAYNUM  TIME    YEAR
ID                                      
9954815 Sun Oct 07 03:35:22 CDT 2018    WB  T.H.62  @ T.H.100 NB    CRASH   Sun Oct 07  03:35:22    2018
9954815 Sun Oct 07 03:35:22 CDT 2018    WB  T.H.62  @ T.H.100 NB    CRASH   Sun Oct 07  03:35:22    2018
9954815 Sun Oct 07 03:35:22 CDT 2018    WB  T.H.62  @ T.H.100 NB    CRASH   Sun Oct 07  03:35:22    2018
9954815 Sun Oct 07 03:35:22 CDT 2018    WB  T.H.62  @ T.H.100 NB    CRASH   Sun Oct 07  03:35:22    2018
9954815 Sun Oct 07 03:35:22 CDT 2018    WB  T.H.62  @ T.H.100 NB    CRASH   Sun Oct 07  03:35:22    2018
9954815 Sun Oct 07 03:35:22 CDT 2018    WB  T.H.62  @ T.H.100 NB    CRASH   Sun Oct 07  03:35:22    201

DATE         object
DIRECTION    object
ROAD         object
LOCATION     object
EVENT        object
DAY          object
MONTH        object
DAYNUM       object
TIME         object
YEAR         object
dtype: object

Answer 1

由于.drop_duplicates返回了df的副本，因此您想要更新df变量或使用inplace=True进行删除。

尝试：

Crash_Data_df = Crash_Data_df.drop_duplicates(subset=['Time'], keep=False)

或

Crash_Data_df.drop_duplicates(subset=['Time'], keep=False, inplace=True)

两者都应该起作用。

顺便说一句，其他drop_duplicates调用也一样。

删除重复项对我导入的csv文件不起作用

将CSV文件导入数据框

1 个答案: