Question

如何删除DataFrame中的连续/连续/相邻重复项？

我正在以CSV格式操作数据，按日期排序，然后按识别号码排序。识别号码可以出现在不同的日期，但我只想删除每日重复项。 drop_duplicates保留一个唯一的实例，但随后在所有其他日期删除该标识符。我尝试了this，但收到了错误：

localhost:~/Desktop/Public$ python3 test.py 
Traceback (most recent call last):
  File "test.py", line 31, in <module>
    df2.loc[df2.shift(1) != df2]
  File "/usr/lib/python3/dist-packages/pandas/core/indexing.py", line 1028, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/usr/lib/python3/dist-packages/pandas/core/indexing.py", line 1148, in _getitem_axis
    raise ValueError('Cannot index with multidimensional key')
ValueError: Cannot index with multidimensional key

编辑原始帖子添加：

我尝试使用index_reset（）来删除任何多索引。以下是数据集的示例：

,DATE,REC,NAME
0,07/02/2009,682566,"Schmoe, Joe"
1,07/02/2009,244828,"Doe, Joe"
2,07/11/2009,325640,"Black, Joe"
3,07/11/2009,544440,"Dirt, Joe"
4,07/11/2009,544440,"Dirt, Joe"
5,07/16/2009,200560,"White, Joe"
6,07/16/2009,685370,"Purple, Joe"
7,07/16/2009,685370,"Purple, Joe"
8,07/16/2009,635400,"Red, Joe"
9,07/16/2009,348562,"Blue, Joe

Answer 1

使用.loc进行索引的方式仅在df2为Series而不是DataFrame时才有效。您实际上是在尝试使用boleens的数据框进行索引，并且.loc不知道该怎么做（它会尝试将其用作多索引）：

>>> df

        DATE     REC         NAME
0 2009-07-02  682566  Schmoe, Joe
1 2009-07-02  244828     Doe, Joe
2 2009-07-11  325640   Black, Joe
3 2009-07-11  544440    Dirt, Joe
4 2009-07-11  544440    Dirt, Joe
5 2009-07-16  200560   White, Joe
6 2009-07-16  685370  Purple, Joe
7 2009-07-16  685370  Purple, Joe
8 2009-07-16  635400     Red, Joe
9 2009-07-16  348562    Blue, Joe

>>> df.shift() != df

    DATE    REC   NAME
0   True   True   True
1  False   True   True
2   True   True   True
3  False   True   True
4  False  False  False
5   True   True   True
6  False   True   True
7  False  False  False
8  False   True   True
9  False   True   True

相反，您希望执行以下操作：

>>> df.loc[df.DATE.shift() != df.DATE]

        DATE     REC         NAME
0 2009-07-02  682566  Schmoe, Joe
2 2009-07-11  325640   Black, Joe
5 2009-07-16  200560   White, Joe

.loc在这里工作，因为我们只为索引创建了一个boleen系列：

>>> df.DATE.shift() != df.DATE

0     True
1    False
2     True
3    False
4    False
5     True
6    False
7    False
8    False
9    False

当然，那不是你想要的数据。要等同于df.drop_duplicates(['REC','DATE'])，您需要以下内容：

>>>  df.loc[(df.DATE != df.DATE.shift(1)) | (df.REC != df.REC.shift(1))]

        DATE     REC         NAME
0 2009-07-02  682566  Schmoe, Joe
1 2009-07-02  244828     Doe, Joe
2 2009-07-11  325640   Black, Joe
3 2009-07-11  544440    Dirt, Joe
5 2009-07-16  200560   White, Joe
6 2009-07-16  685370  Purple, Joe
8 2009-07-16  635400     Red, Joe
9 2009-07-16  348562    Blue, Joe

与drop_duplicates的比较：

>>> df.drop_duplicates(['REC','DATE'])

        DATE     REC         NAME
0 2009-07-02  682566  Schmoe, Joe
1 2009-07-02  244828     Doe, Joe
2 2009-07-11  325640   Black, Joe
3 2009-07-11  544440    Dirt, Joe
5 2009-07-16  200560   White, Joe
6 2009-07-16  685370  Purple, Joe
8 2009-07-16  635400     Red, Joe
9 2009-07-16  348562    Blue, Joe

pandas DataFrame删除连续的重复项

编辑原始帖子添加：

1 个答案: