Question

name    date   
a       [01-01,01-01,01-03]
b       [02-01.03-03.03-03,03-05]
..       ..
..       ..

这是我的数据帧
数据具有重复的ID和日期，因此我使用groupby id

df=DataFrame(data)
uid=df['uid']
dt=df['dt']

df1=pd.Series(uid,name='uid')
df3=pd.Series(dt,name='dt')

df=pd.concat([df1,df3], axis=1,ignore_index=True)
df.groupby(uid, as_index=False).agg(lambda x: x.tolist())

我想要的输出是这样的

 name    date   
a       [01-01,01-03]
b       [02-01,03-03,03-05]
..       ..
..       ..

Answer 1

尝试：

df.date = df.date.apply(lambda x: list(set(x)))

Answer 2

如果要删除重复项，还要根据初始顺序对它们进行排序。见下面的例子：

df = pd.DataFrame.from_dict({'name':['a','b'], 'date': [['01-01','01-01','01-03'],['02-01','03-03','03-03','03-05']]})
print 'before removing duplicates'
print df

print 'after removing duplicates and sorting based on initial order'
df['date'] = df['date'].apply(lambda x: sorted(list(set(x)), key = x.index))
print df

结果

before removing duplicates
                           date name
0         [01-01, 01-01, 01-03]    a
1  [02-01, 03-03, 03-03, 03-05]    b

after removing duplicates and sorting based on initial order
                    date name
0         [01-01, 01-03]    a
1  [02-01, 03-03, 03-05]    b

pandas如何删除重复值

2 个答案: