Question

我有一个大数据框，其中包含600天的数据。每天都有100个时间戳。我有一个单独的30天列表，要从中开始记录数据。如何从数据框中删除这30天的数据？我尝试了for循环，但没有成功。我知道有一个简单的方法。但是我不知道如何实现。

df #is main dataframe which has many columns and rows. Index is a timestamp. 

df['dates'] = df.index.strftime('%Y-%m-%d') # date part of timestamp is sliced and  
#a new column is created. Instead of index, I want to use this column for comparing with bad list. 
bad_list # it is a list of bad dates   
for i in range(0,len(df)):
    for j in range(0,len(bad_list)):
        if str(df['dates'][i])== bad_list[j]:
            df.drop(df[i].index,inplace=True)

Answer 1

您可以执行以下操作

df['dates'] = df.index.strftime('%Y-%m-%d') 
#badlist should be in date format too. 

newdf = df[~df['dates'].isin(badlist)]
# the ~ is used to denote "not in" the list.

#if Jan 1, 2000 is a bad date, it should be in the list as datetime(2000,1,1)

Answer 2

您可以执行简单的比较：

>>> dates = pd.Series(pd.to_datetime(np.random.randint(int(time()) - 60 * 60 * 24 * 5, int(time()), 12), unit='s'))
>>> dates
0    2019-03-19 05:25:32
1    2019-03-20 00:58:29
2    2019-03-19 01:03:36
3    2019-03-22 11:45:24
4    2019-03-19 08:14:29
5    2019-03-21 10:17:13
6    2019-03-18 09:09:15
7    2019-03-20 00:14:16
8    2019-03-21 19:47:02
9    2019-03-23 06:19:35
10   2019-03-23 05:42:34
11   2019-03-21 11:37:46

>>> start_date = pd.to_datetime('2019-03-20')
>>> end_date = pd.to_datetime('2019-03-22')
>>> dates[(dates > start_date) & (dates < end_date)]
1    2019-03-20 00:58:29
5    2019-03-21 10:17:13
7    2019-03-20 00:14:16
8    2019-03-21 19:47:02
11   2019-03-21 11:37:46

如果源Series的格式不是datetime，那么您将需要使用pd.to_datetime进行转换。

如何从大数据框中删除特定的日期时间戳

2 个答案: