Question

我有一个数据集

BDate,Snum,ArrTime,OID,TDate,TTime,VID
1/1/2018,72,05:59:01,7214,1/1/2018,12:06:20 AM ,7206
1/1/2018,72,06:04:33,7208,1/1/2018,12:36:31 AM,7205
1/1/2018,72,06:21:07,7216,1/1/2018,5:53:49 AM,7220
1/1/2018,80,06:29:01,8026,1/1/2018,5:59:10 AM,7214
1/1/2018,72,06:30:54,7218,1/1/2018,6:04:55 AM,7208
1/1/2018,72,06:33:54,7221,1/1/2018,06:21:17 AM,7216
1/1/2018,80,06:35:26,8018,1/1/2018,06:31:04 AM,7218
1/1/2018,72,09:38:34,7211,1/1/2018,1:40:38 PM,7209
1/1/2018,72,13:39:45,7209,,,

我正在寻找的目标是将ArrTime上的列与TTime中最接近的时间进行匹配，这是我在其他帖子中已经实现的。

我正在尝试通过基于ArrTime列创建一个时间范围来改善分析。从上面的数据集可以看出，ArrTime的第一个计时是0:59:01，最后一个ArrTime是13:39:45。我想使用这2个计时（但最后一个计时+ 1分钟）作为时间边界，以删除TTime中任何不在范围内的时间。

如下所示是我的代码

mydataset = pd.read_csv("Test.csv", error_bad_lines=False, engine ='python', index_col= False,header = 0, sep = ",")
mydataset['Date1'] = pd.to_datetime(mydataset['BDate'] + ' ' + mydataset['ArrTime'], format='%d/%m/%Y %H:%M:%S')
datesAM = pd.to_datetime(mydataset['TDate'] + ' ' + mydataset['TTime'], format='%d/%m/%Y %I:%M:%S %p')
datesPM = pd.to_datetime(mydataset['TDate'] + ' ' + mydataset['TTime'], format='%d/%m/%Y %H:%M:%S %p')
mydataset['Date2'] = datesAM.mask(mydataset['TTime'].str.endswith('AM',na=False), datesPM)
#print(mydataset)

df1 = mydataset[['Date1','Snum', 'OID']].sort_values('Date1').dropna(subset=['I'])
df1['OID'] = df1['OID'].astype(np.int64)

a = df1['Date1'].iloc[0]
a1 = a.time().strftime('%H:%M:%S') 
print(a1)
b = df1['Date1'].iloc[-1]
b1 = b.time().strftime('%H:%M:%S') 
print(b1)

df2 = mydataset[['Date2','VID']].sort_values('Date2').dropna(subset=['VID'])
df2['VID'] = df2['VID'].astype(np.int64)

df2[df2['Date2'].indexer_between_time(a1,b1)]

#df2['Date2'] = pd.date_range(start = a1, end = b1)
#print(df2)

我尝试使用iloc识别第一个datetime和lastdate时间，然后将其剥离为timeformat。我曾尝试使用pd.date_range和indexer_between_time，但这都给了我错误，例如“'系列'对象没有属性'indexer_between_time'”和“值的长度与索引的长度不匹配”

我的最终目标是删除不在范围内的详细信息（不是整行，而是TDate，TTime，VID），然后进行最近时间的匹配（匹配时间已经实现）

BDate,Snum,ArrTime,OID,TDate,TTime,VID
1/1/2018,72,05:59:01,7214,,,
1/1/2018,72,06:04:33,7208,,,
1/1/2018,72,06:21:07,7216,,,
1/1/2018,80,06:29:01,8026,1/1/2018,5:59:10 AM,7214
1/1/2018,72,06:30:54,7218,1/1/2018,6:04:55 AM,7208
1/1/2018,72,06:33:54,7221,1/1/2018,06:21:17 AM,7216
1/1/2018,80,06:35:26,8018,1/1/2018,06:31:04 AM,7218
1/1/2018,72,09:38:34,7211,1/1/2018,1:40:38 PM,7209  
1/1/2018,72,13:39:45,7209,,,

Answer 1

我可以通过将datetime列转换为unix时间戳来做到这一点，以便我们轻松比较和过滤出范围内的datetime。

这就是我要做的：

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
    <style>

        img:nth-child (8) , img:nth-child (9) , img:nth-child (10) img:nth-child (11)  { /* It is not working because img is not first-element, it's from 8th to 11th elements. */
            position: absolute;
        }

        a:nth-child(4) {

            background-color: red;
        }

        p:nth-child(1) {
            background-color: red;
        }

    </style>
</head>
<body>
<div class="box">
    <p>abc</p>
    <p>abc</p>
    <p>abc</p>
    <a>adwddw</a>
    <a>qwer</a>
    <a>qwer</a>
    <a>qwer</a>
    <img src="4.jpg" alt="">
    <img src="1.png" alt="">
    <img src="2.jpg" alt="">
    <img src="3.jpeg" alt="">
</div>
</body>
</html>

这是输出：

mydataset = pd.read_csv("data.csv", error_bad_lines=False, engine ='python', index_col= False,header = 0, sep = ",")
mydataset['Date1'] = pd.to_datetime(mydataset['BDate'] + ' ' + mydataset['ArrTime'], format='%d/%m/%Y %H:%M:%S')

# Function to clean dates because the format is not consistent. For example: We have *6:04:55 AM* and *06:21:17 AM* 
def cleanDate(x):
    if str(x) == 'nan':
        return np.nan
    else:
        temp = ''
        if int(x.split(':')[0]) < 10:
            temp += '0' + str(int(x.split(':')[0])) +':'
        else:
            temp += x.split(':')[0] + ':'
        temp += x.split(':',1)[1]
        return temp

mydataset['TTime'] = mydataset['TTime'].apply(lambda x: cleanDate(x))
mydataset['Date2'] = pd.to_datetime(mydataset['TDate'] + ' ' + mydataset['TTime'], format='%d/%m/%Y %I:%M:%S %p', errors='ignore')
mydataset['Date2'] = pd.to_datetime(mydataset['Date2'])

# Convert Datetime to unix timestamp and create a new column
mydataset['tsArrTime'] = mydataset['Date1'].apply(lambda x: time.mktime(x.timetuple()))
mydataset['tsTTime'] = mydataset['Date2'].apply(lambda x: time.mktime(x.timetuple()) if str(x) != 'NaT' else 0)

# Get min and max timestamp from tsArrTime column
minTime = mydataset['tsArrTime'].min() 
maxTime = mydataset['tsArrTime'].max() + 60  # End datetime + 1 min

# Check if tsTTime is within the range else replace with empty string (Change it to whatever you want)
mydataset.loc[(mydataset['tsTTime'] < minTime) | (mydataset['tsTTime'] > maxTime), 'TTime'] = ''
mydataset.loc[(mydataset['tsTTime'] < minTime) | (mydataset['tsTTime'] > maxTime), 'TDate'] = ''
mydataset.loc[(mydataset['tsTTime'] < minTime) | (mydataset['tsTTime'] > maxTime), 'VID'] = ''
mydataset['TTime'] = mydataset['TTime'].fillna('')
mydataset['TDate'] = mydataset['TDate'].fillna('')
mydataset['VID'] = mydataset['VID'].fillna('')

mydataset = mydataset.drop(columns=['Date1','Date2','tsArrTime','tsTTime'])

使用python将日期时间保持在相对于数据的范围内

1 个答案: