创建数据帧中不存在的时间间隔

时间:2019-09-09 11:48:37

标签: python pandas datetime time intervals

我有-机器错误/机器停止-详细的工厂,工作站,机器,开始日期时间和结束日期时间的数据。

我想在机器与python / pandas一起正常运行时创建时间间隔

因此,我希望有24小时的时间表,并且每个时间间隔都标记为有效(如果没有发生错误)或无效。

数据帧如下所示,用于1个站点(总共17个),1个机器类型(总计10个)和1天;

Stat.  Mac.   start_date          end_date            start_no   end_no  status
 A     B    2019-01-03 00:00:00  2019-01-03 01:30:00     1         90     pause
 A     B    2019-01-03 09:35:00  2019-01-03 10:20:00    575        620    pause
 A     B    2019-01-03 20:20:00  2019-01-03 20:40:00    1220       1240   pause
 A     B    2019-01-03 21:45:00  2019-01-03 22:45:00    1305       1365   pause

对于相同的工作站-机器-天对,请求的数据帧应如下所示;

    Stat.  Mac.   start_date          end_date            start_no   end_no  status
     A     B    2019-01-03 00:00:00  2019:01:03 00:00:01     0         1      working
     A     B    2019-01-03 00:00:00  2019-01-03 01:30:00     1         90     pause
     A     B    2019-01-03 01:30:00  2019-01-03 09:35:00     90        575    working
     A     B    2019-01-03 09:35:00  2019-01-03 10:20:00    575        620    pause
     A     B    2019-01-03 10:20:00  2019-01-03 20:20:00    620        1220   working
     A     B    2019-01-03 20:20:00  2019-01-03 20:40:00    1220       1240   pause
     A     B    2019-01-03 20:40:00  2019-01-03 21:45:00    1240       1305   working
     A     B    2019-01-03 21:45:00  2019-01-03 22:45:00    1305       1365   pause
     A     B    2019-01-03 22:45:00  2019-01-03 23:59:00    1365       1439   working

我在下面的链接中上传了示例数据帧(1000rows-〜80kb);

https://gofile.io/?c=tKA8Qj

我应该如何解决这个问题?

预先感谢

2 个答案:

答案 0 :(得分:1)

一种快速但缓慢的方法可能是循环遍历所有行并检查当前+下一行。您只有1000行,所以暂时就可以了。这看起来像这样:

import pandas as pd
df = pd.read_excel("sample_2.xlsx")

df['status'] = 'pause'

df = df.sort_values(['Workcenter','Machine','Error_Reason','Class','start_date','start_time', 'end_date','end_time']).reset_index()
new_df = df.copy()

number_rows = len(df)-1
for i in range(number_rows):
    row = df.loc[i]
    next_row = df.loc[i+1]

    new_row = row
    new_row['status'] = 'working'
    new_row['start_date'] = row['end_date']
    new_row['end_date'] = next_row['start_date']
    new_row['start_number'] = row['end_number']
    new_row['end_number'] = next_row['start_number']
    new_df = new_df.append(new_row)

答案 1 :(得分:1)

在此问题中,我们有一个顺序模式,可以将“ start_no”和“ end_no”列转换为所需数据帧的列。 当我们采用(start_no0, end_no0, start_no1, end_no1, ...)之类的值时,实际上得到了“ start_no”和“ end_no”所需列的最大部分。通过简单的修复,我们可以获得完全相同的列。可以将相同的逻辑应用于start_date和end_date,因为它们表示相同的事物。

由于站和机器的值不同,因此我们可以通过用Stat。,Mac。,start_date,end_date编制索引将问题分为几类。在代码中,我尝试通过忽略原始数据集中的时间字段来做到这一点,以获取当天的所有值。基本上,我只是将数据分组,然后对每个组进行迭代以创建一个包含所需信息的新数据框。

对于您共享的案例,代码如下:

import numpy as np
import pandas as pd

data = pd.read_excel("sample_2.xlsx")

# transform (start|end)_date as only date without time
data["_sDate"] = data.start_date.apply(lambda x: x.strftime("%Y-%m-%d"))
data["_eDate"] = data.end_date.apply(lambda x: x.strftime("%Y-%m-%d"))

# group the data by following columns
grouped = data.groupby(["Station","Machine","_sDate","_eDate"])
# container for storing result of each group
container = []

# iterate the groups
for name, group in grouped:
    # sort them by start_number
    group = group.sort_values("start_number")
    # get (start|end)_numbers into a flatten array
    nums = group[["start_number", "end_number"]].values.flatten()
    # get (start|end)_date into a flatten array
    dates = group[["start_date", "end_date"]].values.flatten()
    ## insert required values to nums and dates
    # we add the first pause time at index 1 to show first working interval
    dates = np.insert(dates, 1 , dates[0] + nums[0]*10**9)
    # we add 0 in the beginning of the array to show first working interval
    nums = np.insert(nums, 0, 0)
    # create df
    nrow = nums.size-1      # decrement, because we add one additional element
    newdf = pd.DataFrame({
        "Station": np.tile(("A"),nrow),
        "Machine": np.tile(("B"),nrow),
        "start_date": dates[:-1],
        "end_date": dates[1:],
        "start_no": nums[:-1],
        "end_no": nums[1:],
        "status": np.tile(["working", "pause"], nrow//2)
    })
    container.append(newdf)

df_final = pd.concat(container)
df_final.index = range(0,df_final.shape[0])