Question

我有多个数据帧，这些数据帧可以具有相同的时间戳（也为+ -1秒），其中包含毫秒。因此，当它们一起放在新的数据框中时，我想过滤出彼此之间相差1秒以上的行

是否存在类似于dftogether['unique'] = np.ediff1d(dftogether['DateTime']的可用于时间戳记的功能？

我当前的解决方案有效，但是我正在寻找一种合适的方法来实现。假设我有3个数据帧，df1，df2和df3。对于每个数据框，我都这样做：

df1['DateTime'] = df1['DateTime'].apply(lambda 
x: x.strftime('%Y%d%m%H%M%S'))
df1['DateTime']= df1['DateTime'].astype(np.int64)

将我的DateTime变成int，这样我就可以做到：

dftogether= pd.concat(z, sort=True)
dftogether= dftogether.sort_values('DateTime')
dftogether['unique'] = np.ediff1d(dftogether['DateTime'], to_begin=20181211150613411) <1
dftogether= dftogether[dftogether.unique == False]

然后我将int转换回datetime

 dftogether['DateTime'] = dftogether['DateTime'].apply(lambda x: pd.to_datetime(str(x), format='%Y%d%m%H%M%S'))

我不知道如何为时间戳创建样本数据，因此我只复制粘贴数据框的一部分。

df1

737    2018-12-18 12:37:19.717
738    2018-12-18 12:37:21.936
739    2018-12-18 12:37:22.841
740    2018-12-18 12:37:23.144
877    2018-12-18 12:40:53.268
878    2018-12-18 12:40:56.597
879    2018-12-18 12:40:56.899
880    2018-12-18 12:40:57.300
968    2018-12-18 12:43:31.411
969    2018-12-18 12:43:36.150
970    2018-12-18 12:43:36.452

df2

691    2018-12-18 12:35:23.612
692    2018-12-18 12:35:25.627
788    2018-12-18 12:38:33.248
789    2018-12-18 12:38:33.553
790    2018-12-18 12:38:34.759
866    2018-12-18 12:40:29.487
867    2018-12-18 12:40:31.199
868    2018-12-18 12:40:32.206

df3

699    2018-12-18 12:35:42.452
701    2018-12-18 12:35:45.081
727    2018-12-18 12:36:47.466
730    2018-12-18 12:36:51.796
741    2018-12-18 12:37:23.448
881    2018-12-18 12:40:57.603
910    2018-12-18 12:42:02.904
971    2018-12-18 12:43:37.361

我希望dftogether看起来像这样，但是带有时间戳而不是整数

   Unique  DateTime
 737    False  20181812123719
 738    False  20181812123721
 739    False  20181812123722
 741    False  20181812123723
 742     True  20181812123723
 740     True  20181812123723
 785    False  20181812123830
 786    False  20181812123831
 787    False  20181812123832
 787     True  20181812123832
 788    False  20181812123833

所以我可以将那些放在Unique == True

 785    False 2018-12-18 12:38:30
 786    False 2018-12-18 12:38:31
 787    False 2018-12-18 12:38:32
 788    False 2018-12-18 12:38:33
 790    False 2018-12-18 12:38:34
 812    False 2018-12-18 12:39:10
 813    False 2018-12-18 12:39:11

还有其他事情：对于新的stackoverflow，我在哪里可以发表自己的看法？ IMO这真是太糟糕了，它一直在滚动，输入/复制代码现在真的很混乱，所有示例程序都使人分心。我花了30多分钟来写这个问题

Answer 1

我将您的df1和df2加入了df，并创建了这样的日期列表：

df = pd.concat([df1,df2]).sort_values('DateTime').reset_index(drop=True)

date_list = [datetime.strptime(i, '%Y-%m-%d %H:%M:%S.%f') for i in df.DateTime.tolist()]

然后我得到1衬里的所需输出：

df[[x>1 for x in [0]+[(j-i).total_seconds() for i,j in zip(date_list, date_list[1:])]]]

要了解其工作原理，请首先检查以下内容的输出：

[x>1 for x in [0]+[(j-i).total_seconds() for i,j in zip(date_list, date_list[1:])]]

希望这会有所帮助。干杯。

Answer 2

我这样做了，您的初始列是a和b-这是您需要的吗？

from datetime import timedelta
df = pd.concat([df1, df2, df3])
df = df.sort_values('b')
df['s'] = df['b'].shift()
df['d'] = df['b'] - df['s'] 
df['f'] = df['d'] < timedelta(0, 1)
print(df)

结果：

      a                       b                       s               d      f
0   691 2018-12-18 12:35:23.612                     NaT             NaT  False
1   692 2018-12-18 12:35:25.627 2018-12-18 12:35:23.612 00:00:02.015000  False
0   699 2018-12-18 12:35:42.452 2018-12-18 12:35:25.627 00:00:16.825000  False
1   701 2018-12-18 12:35:45.081 2018-12-18 12:35:42.452 00:00:02.629000  False
2   727 2018-12-18 12:36:47.466 2018-12-18 12:35:45.081 00:01:02.385000  False
3   730 2018-12-18 12:36:51.796 2018-12-18 12:36:47.466 00:00:04.330000  False
0   737 2018-12-18 12:37:19.717 2018-12-18 12:36:51.796 00:00:27.921000  False
1   738 2018-12-18 12:37:21.936 2018-12-18 12:37:19.717 00:00:02.219000  False
2   739 2018-12-18 12:37:22.841 2018-12-18 12:37:21.936 00:00:00.905000   True
3   740 2018-12-18 12:37:23.144 2018-12-18 12:37:22.841 00:00:00.303000   True
4   741 2018-12-18 12:37:23.448 2018-12-18 12:37:23.144 00:00:00.304000   True
2   788 2018-12-18 12:38:33.248 2018-12-18 12:37:23.448 00:01:09.800000  False
3   789 2018-12-18 12:38:33.553 2018-12-18 12:38:33.248 00:00:00.305000   True
4   790 2018-12-18 12:38:34.759 2018-12-18 12:38:33.553 00:00:01.206000  False
5   866 2018-12-18 12:40:29.487 2018-12-18 12:38:34.759 00:01:54.728000  False
6   867 2018-12-18 12:40:31.199 2018-12-18 12:40:29.487 00:00:01.712000  False
7   868 2018-12-18 12:40:32.206 2018-12-18 12:40:31.199 00:00:01.007000  False
4   877 2018-12-18 12:40:53.268 2018-12-18 12:40:32.206 00:00:21.062000  False
5   878 2018-12-18 12:40:56.597 2018-12-18 12:40:53.268 00:00:03.329000  False
6   879 2018-12-18 12:40:56.899 2018-12-18 12:40:56.597 00:00:00.302000   True
7   880 2018-12-18 12:40:57.300 2018-12-18 12:40:56.899 00:00:00.401000   True
5   881 2018-12-18 12:40:57.603 2018-12-18 12:40:57.300 00:00:00.303000   True
6   910 2018-12-18 12:42:02.904 2018-12-18 12:40:57.603 00:01:05.301000  False
8   968 2018-12-18 12:43:31.411 2018-12-18 12:42:02.904 00:01:28.507000  False
9   969 2018-12-18 12:43:36.150 2018-12-18 12:43:31.411 00:00:04.739000  False
10  970 2018-12-18 12:43:36.452 2018-12-18 12:43:36.150 00:00:00.302000   True
7   971 2018-12-18 12:43:37.361 2018-12-18 12:43:36.452 00:00:00.909000   True

有找到日期时间之间差异的功能吗？

2 个答案: