使用字典过滤熊猫

时间:2018-10-18 14:59:06

标签: python pandas

我在一个自助服务公司工作,我们正在寻找UI更新是否有任何作用。每台机器在不同的日期/时间都有更新。我建立了machine_ids字典和安装新UI的时间戳。然后,我想用它来过滤结果,所以只返回在字典中有machine_id且存放日期大于字典中相应日期的行

uidict= {
 14.0: Timestamp('2018-10-12 17:48:57'),
 16.0: Timestamp('2018-10-12 13:38:00'),
 19.0: Timestamp('2018-10-17 20:17:33'),
 20.0: Timestamp('2018-10-15 12:15:34'),
 27.0: Timestamp('2018-09-26 11:50:01'),
 29.0: Timestamp('2018-10-03 13:38:17'),
 31.0: Timestamp('2018-10-17 10:06:23'),
 33.0: Timestamp('2018-09-21 15:17:14'),
 34.0: Timestamp('2018-10-17 11:42:21'),
 42.0: Timestamp('2018-10-16 12:36:32'),
 45.0: Timestamp('2018-09-23 13:23:37'),
 53.0: Timestamp('2018-09-27 12:18:39'),
 60.0: Timestamp('2018-10-15 15:27:46'),
 62.0: Timestamp('2018-08-30 17:26:27'),
 63.0: Timestamp('2018-09-25 17:44:04'),
 64.0: Timestamp('2018-09-23 14:19:57'),
 65.0: Timestamp('2018-08-31 19:07:47'),
 66.0: Timestamp('2018-09-08 14:12:20'),
 67.0: Timestamp('2018-09-11 08:18:31'),
 69.0: Timestamp('2018-09-20 17:12:37'),
 70.0: Timestamp('2018-09-24 12:56:45'),
 71.0: Timestamp('2018-08-27 09:37:17'),
 72.0: Timestamp('2018-09-05 19:07:34'),
 73.0: Timestamp('2018-09-10 14:42:52'),
 74.0: Timestamp('2018-09-25 16:36:05'),
 75.0: Timestamp('2018-08-27 10:09:02'),
 76.0: Timestamp('2018-09-13 07:20:40'),
 77.0: Timestamp('2018-09-02 14:10:22'),
 78.0: Timestamp('2018-09-26 15:06:51'),
 79.0: Timestamp('2018-08-31 15:52:49'),
 81.0: Timestamp('2018-10-05 10:05:11')}

我尝试了此过滤以使其起作用:

df[(df.machine_id.isin(uidict.keys()))&(df.deposited_at>uidict[df.machine_id])]

但这返回

TypeError: 'Series' objects are mutable, thus they cannot be hashed

所以我以为我会忘记字典,而只使用我制作的groupby系列。.

 ---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-90-10d8db20a295> in <module>()
----> 1 df[(df.machine_name.isin(newuidict.index))&(df.deposited_at>newuidict[df.machine_name])]

~/anaconda3/lib/python3.6/site-packages/pandas/core/ops.py in wrapper(self, other, axis)
    816             if not self._indexed_same(other):
    817                 msg = 'Can only compare identically-labeled Series objects'
--> 818                 raise ValueError(msg)
    819             return self._constructor(na_op(self.values, other.values),
    820                                      index=self.index, name=name)

ValueError: Can only compare identically-labeled Series objects

使用函数运行该函数并应用它需要花费很多时间,而我将不得不经常运行此代码,是否有任何办法可以使这种过滤器起作用?

小数据示例:

 machine_id deposited_at
12  2018-10-04 14:49:38
56  2018-09-20 14:41:59
24  2018-08-25 14:50:07
56  2018-08-04 15:33:09
12  2018-08-01 18:18:44
24  2018-09-24 12:34:35
35  2018-10-01 17:09:38
21  2018-09-27 11:32:02
21  2018-09-27 11:33:55
23  2018-08-30 10:03:01

2 个答案:

答案 0 :(得分:1)

[答案需要Python 3和Pandas]

如果更改自己的uidict并不太麻烦,您可以将其转换为数据框并使用联接。我将在下面说明该过程:

首先,重新创建您的uidict:

import pandas as pd
from pandas import Timestamp

uidict= {
 14.0: Timestamp('2018-10-12 17:48:57'),
 16.0: Timestamp('2018-10-12 13:38:00'),
 19.0: Timestamp('2018-10-17 20:17:33'),
 20.0: Timestamp('2018-10-15 12:15:34'),
 27.0: Timestamp('2018-09-26 11:50:01'),
 29.0: Timestamp('2018-10-03 13:38:17'),
 31.0: Timestamp('2018-10-17 10:06:23'),
 33.0: Timestamp('2018-09-21 15:17:14'),
 34.0: Timestamp('2018-10-17 11:42:21'),
 42.0: Timestamp('2018-10-16 12:36:32'),
 45.0: Timestamp('2018-09-23 13:23:37'),
 53.0: Timestamp('2018-09-27 12:18:39'),
 60.0: Timestamp('2018-10-15 15:27:46'),
 62.0: Timestamp('2018-08-30 17:26:27'),
 63.0: Timestamp('2018-09-25 17:44:04'),
 64.0: Timestamp('2018-09-23 14:19:57'),
 65.0: Timestamp('2018-08-31 19:07:47'),
 66.0: Timestamp('2018-09-08 14:12:20'),
 67.0: Timestamp('2018-09-11 08:18:31'),
 69.0: Timestamp('2018-09-20 17:12:37'),
 70.0: Timestamp('2018-09-24 12:56:45'),
 71.0: Timestamp('2018-08-27 09:37:17'),
 72.0: Timestamp('2018-09-05 19:07:34'),
 73.0: Timestamp('2018-09-10 14:42:52'),
 74.0: Timestamp('2018-09-25 16:36:05'),
 75.0: Timestamp('2018-08-27 10:09:02'),
 76.0: Timestamp('2018-09-13 07:20:40'),
 77.0: Timestamp('2018-09-02 14:10:22'),
 78.0: Timestamp('2018-09-26 15:06:51'),
 79.0: Timestamp('2018-08-31 15:52:49'),
 81.0: Timestamp('2018-10-05 10:05:11')
}

然后我们可以使用此行创建一个熊猫数据框,为方便起见,我将字典的键命名为“ machine_id”。

uidf = pd.DataFrame(list(uidict.items()),columns=['machine_id','ui_date'])

这将导致:

    machine_id  ui_date 
0   64.0    2018-09-23 14:19:57
1   65.0    2018-08-31 19:07:47
2   66.0    2018-09-08 14:12:20
3   67.0    2018-09-11 08:18:31
4   69.0    2018-09-20 17:12:37
5   70.0    2018-09-24 12:56:45
6   71.0    2018-08-27 09:37:17
7   72.0    2018-09-05 19:07:34
8   73.0    2018-09-10 14:42:52
9   74.0    2018-09-25 16:36:05
10  75.0    2018-08-27 10:09:02
11  76.0    2018-09-13 07:20:40
12  77.0    2018-09-02 14:10:22
13  14.0    2018-10-12 17:48:57
14  79.0    2018-08-31 15:52:49
15  16.0    2018-10-12 13:38:00
16  81.0    2018-10-05 10:05:11
17  19.0    2018-10-17 20:17:33
18  20.0    2018-10-15 12:15:34
19  78.0    2018-09-26 15:06:51
20  27.0    2018-09-26 11:50:01
21  29.0    2018-10-03 13:38:17
22  31.0    2018-10-17 10:06:23
23  33.0    2018-09-21 15:17:14
24  34.0    2018-10-17 11:42:21
25  42.0    2018-10-16 12:36:32
26  45.0    2018-09-23 13:23:37
27  53.0    2018-09-27 12:18:39
28  60.0    2018-10-15 15:27:46
29  62.0    2018-08-30 17:26:27
30  63.0    2018-09-25 17:44:04

然后重新创建示例数据,但是我在底部添加了两个测试用例行,因为您提供的示例在uidict上似乎没有任何匹配项。具体来说,一行的machine_id = 81,但日期早于uidict中的日期,而日期早于uidict中的日期。

data_sample = pd.DataFrame(
    [
        {'machine_id': 12, 'deposited_at' : Timestamp('2018-10-04 14:49:38')},
        {'machine_id': 56, 'deposited_at' : Timestamp('2018-09-20 14:41:59')},
        {'machine_id': 24, 'deposited_at' : Timestamp('2018-08-25 14:50:07')},
        {'machine_id': 56, 'deposited_at' : Timestamp('2018-08-04 15:33:09')},
        {'machine_id': 12, 'deposited_at' : Timestamp('2018-08-01 18:18:44')},
        {'machine_id': 24, 'deposited_at' : Timestamp('2018-09-24 12:34:35')},
        {'machine_id': 35, 'deposited_at' : Timestamp('2018-10-01 17:09:38')},
        {'machine_id': 21, 'deposited_at' : Timestamp('2018-09-27 11:32:02')},
        {'machine_id': 21, 'deposited_at' : Timestamp('2018-09-27 11:33:55')},
        {'machine_id': 23, 'deposited_at' : Timestamp('2018-08-30 10:03:01')},
        {'machine_id': 81, 'deposited_at' : Timestamp('2018-09-01 10:03:01')},
        {'machine_id': 81, 'deposited_at' : Timestamp('2018-10-06 10:03:01')}
    ]
)

    deposited_at    machine_id
0   2018-10-04 14:49:38 12
1   2018-09-20 14:41:59 56
2   2018-08-25 14:50:07 24
3   2018-08-04 15:33:09 56
4   2018-08-01 18:18:44 12
5   2018-09-24 12:34:35 24
6   2018-10-01 17:09:38 35
7   2018-09-27 11:32:02 21
8   2018-09-27 11:33:55 21
9   2018-08-30 10:03:01 23
10  2018-09-01 10:03:01 81
11  2018-10-06 10:03:01 81

然后我们使用“ machine_id”作为键在这两个DataFrame上进行内部联接,然后在日期上跟踪一个简单的过滤条件。最后 这行是为了简单地清理列以使其类似于原始输入。

filtered_dataframe = data_sample.merge(uidf, on=['machine_id'], how='inner')

filtered_dataframe = filtered_dataframe[
    filtered_dataframe['deposited_at'] > filtered_dataframe['ui_date']
]

filtered_dataframe = filtered_dataframe[['machine_id', 'deposited_at']]

有效地确保1)数据样本中的计算机ID在您的UI表中,以及2)存放日期大于UI表中的日期:

    machine_id  deposited_at
1   81  2018-10-06 10:03:01

希望这就是您想要的!

答案 1 :(得分:1)

df ['flag'] = df.apply(lambda x:如果x ['deposited_at']。iloc [0]> = uidict(x ['machine_id']。iloc [0])则为True,否则为False)< / p>

df [df ['flag'] ==真]