在pandas df

时间:2018-10-10 00:04:47

标签: python pandas numpy dataframe assign

此问题与名册或人员配置有关。我正在尝试将各种工作分配给个人(员工)。使用下面的df

`[Person]` = Individuals (employees)
`[Area]` and `[Place]` = unique jobs
`[On]` = How many unique jobs are occurring at each point in time

因此[Area][Place]共同构成了unique个值,它们是不同的工作。这些值将分配给个人,其总体目标是使用尽可能少的个人。每个人的唯一值assigned是3。[On]显示unique[Place]的当前[Area]值发生了多少。因此,这为我需要多少个人提供了具体的指导。例如,

1-3 unique values occurring = 1 individual
4-6 unique values occurring = 2 individuals
7-9 unique values occurring = 3 individuals etc

问题: unique[Area][Place]值的数量大于3的地方给我带来麻烦。我无法进行groupby,其中assign的第一个3 unique valuesindividual 1,接下来的三个uniqueindividual 2等。我想要按[Area][Place][Area]中的唯一值进行分组。因此,请向个人(最多3个)assign中的[Area]相同值。然后,如果有剩余的 值(<3),则应将它们合并为一组(如果可能),为3。

我设想的工作方式是:hour 展望未来。对于每个新的row值,script应该看到多少个值[On](这表明需要多少个人)。如果unique的值> 3,则它们应为assigned的{​​{1}}与grouping中的相同值。如果有剩余个值,则应将它们组合起来以组成3个一组。

将其逐步进行:

1)使用[Area] [On]来确定需要多少人,方法是展望未来寻找Column

2)如果出现超过3个hour值,请首先在unique中分配相同的值。

3)如果有任何剩余值,则无论如何都希望合并。

对于下面的[Area]dfunique中有[Place]的位置出现9个[Area]值。因此,我们应该有3个人hour。当assigned的值> 3时,应由unique分配,并查看是否出现相同的值。 剩余值应与值小于3 [Area]的其他个人组合。

unique

这是我的尝试:

import pandas as pd
import numpy as np

d = ({
    'Time' : ['8:03:00','8:17:00','8:20:00','8:28:00','8:35:00','08:40:00','08:42:00','08:45:00','08:50:00'],                 
    'Place' : ['House 1','House 2','House 3','House 4','House 5','House 1','House 2','House 3','House 2'],                 
    'Area' : ['A','B','C','D','E','D','E','F','G'],     
    'On' : ['1','2','3','4','5','6','7','8','9'], 
    'Person' : ['Person 1','Person 2','Person 3','Person 4','Person 5','Person 4','Person 5','Person 6','Person 7'],   
     })

df = pd.DataFrame(data=d)

这是输出:

def reduce_df(df):
    values = df['Area'] + df['Place']
    df1 = df.loc[~values.duplicated(),:] # ignore duplicate values for this part..
    person_count = df1.groupby('Person')['Person'].agg('count')
    leftover_count = person_count[person_count < 3] # the 'leftovers'

    # try merging pairs together
    nleft = leftover_count.shape[0]
    to_try = np.arange(nleft - 1)
    to_merge = (leftover_count.values[to_try] + 
                leftover_count.values[to_try + 1]) <= 3
    to_merge[1:] = to_merge[1:] & ~to_merge[:-1]
    to_merge = to_try[to_merge]
    merge_dict = dict(zip(leftover_count.index.values[to_merge+1], 
                leftover_count.index.values[to_merge]))
    def change_person(p):
        if p in merge_dict.keys():
            return merge_dict[p]
        return p
    reduced_df = df.copy()
    # update df with the merges you found
    reduced_df['Person'] = reduced_df['Person'].apply(change_person)
    return reduced_df

df1 = (reduce_df(reduce_df(df)))

这是我的预期输出:

       Time    Place Area On    Person
0   8:03:00  House 1    A  1  Person 1
1   8:17:00  House 2    B  2  Person 1
2   8:20:00  House 3    C  3  Person 1
3   8:28:00  House 4    D  4  Person 4
4   8:35:00  House 5    E  5  Person 5
5   8:40:00  House 1    D  6  Person 4
6   8:42:00  House 2    E  7  Person 5
7   8:45:00  House 3    F  8  Person 5
8   8:50:00  House 2    G  9  Person 7

关于如何获取此输出的说明:

       Time    Place Area On    Person
0   8:03:00  House 1    A  1  Person 1
1   8:17:00  House 2    B  2  Person 1
2   8:20:00  House 3    C  3  Person 1
3   8:28:00  House 4    D  4  Person 2
4   8:35:00  House 5    E  5  Person 3
5   8:40:00  House 6    D  6  Person 2
6   8:42:00  House 2    E  7  Person 3
7   8:45:00  House 3    F  8  Person 2
8   8:50:00  House 2    G  9  Person 3

示例2:

Index 0: One `unique` value occurring. So `assign` to individual 1
Index 1: Two `unique` values occurring. So `assign` to individual 1
Index 2: Three `unique` values occurring. So `assign` to individual 1
Index 3: Four `unique` values on. So `assign` to individual 2
Index 4: Five `unique` values on. This one is a bit tricky and hard to conceptualise. But there is another `E` within an `hour`. So `assign` to a new individual so it can be combined with the other `E`
Index 5: Six `unique` values on. Should be `assigned` with the other `D`. So individual 2
Index 6: Seven `unique` values on. Should be `assigned` with other `E`. So individual 3
Index 7: Eight `unique` values on. New value in `[Area]`, which is a _leftover_. `Assign` to either individual 2 or 3
Index 8: Nine `unique` values on. New value in `[Area]`, which is a _leftover_. `Assign` to either individual 2 or 3

我遇到错误:

d = ({
    'Time' : ['8:03:00','8:17:00','8:20:00','8:28:00','8:35:00','8:40:00','8:42:00','8:45:00','8:50:00'],                 
    'Place' : ['House 1','House 2','House 3','House 1','House 2','House 3','House 1','House 2','House 3'],                 
    'Area' : ['X','X','X','X','X','X','X','X','X'],     
    'On' : ['1','2','3','3','3','3','3','3','3'], 
    'Person' : ['Person 1','Person 1','Person 1','Person 1','Person 1','Person 1','Person 1','Person 1','Person 1'],   
    })

    df = pd.DataFrame(data=d)

在这一行:

 IndexError: index 1 is out of bounds for axis 1 with size 1

但是,如果我将Person更改为1,2,3重复,它将返回以下内容:

df.loc[:,'Person'] = df['Person'].unique()[assignedPeople]

预期输出:

'Person' : ['Person 1','Person 2','Person 3','Person 1','Person 2','Person 3','Person 1','Person 2','Person 3'], 

      Time    Place Area On    Person
0  8:03:00  House 1    X  1  Person 1
1  8:17:00  House 2    X  2  Person 1
2  8:20:00  House 3    X  3  Person 1
3  8:28:00  House 1    X  3  Person 2
4  8:35:00  House 2    X  3  Person 2
5  8:40:00  House 3    X  3  Person 2
6  8:42:00  House 1    X  3  Person 3
7  8:45:00  House 2    X  3  Person 3
8  8:50:00  House 3    X  3  Person 3

示例2的主要内容是:

      Time    Place Area On    Person
0  8:03:00  House 1    X  1  Person 1
1  8:17:00  House 2    X  2  Person 1
2  8:20:00  House 3    X  3  Person 1
3  8:28:00  House 1    X  3  Person 1
4  8:35:00  House 2    X  3  Person 1
5  8:40:00  House 3    X  3  Person 1
6  8:42:00  House 1    X  3  Person 1
7  8:45:00  House 2    X  3  Person 1
8  8:50:00  House 3    X  3  Person 1

3 个答案:

答案 0 :(得分:4)

更新

There's a live version of this answer online that you can try for yourself.

以下是allocatePeople函数形式的答案。它基于预先计算一个小时内重复的所有索引:

from collections import Counter
import numpy as np
import pandas as pd

def getAssignedPeople(df, areasPerPerson):
    areas = df['Area'].values
    places = df['Place'].values
    times = pd.to_datetime(df['Time']).values
    maxPerson = np.ceil(areas.size / float(areasPerPerson)) - 1
    assignmentCount = Counter()
    assignedPeople = []
    assignedPlaces = {}
    heldPeople = {}
    heldAreas = {}
    holdAvailable = True
    person = 0

    # search for repeated areas. Mark them if the next repeat occurs within an hour
    ixrep = np.argmax(np.triu(areas.reshape(-1, 1)==areas, k=1), axis=1)
    holds = np.zeros(areas.size, dtype=bool)
    holds[ixrep.nonzero()] = (times[ixrep[ixrep.nonzero()]] - times[ixrep.nonzero()]) < np.timedelta64(1, 'h')

    for area,place,hold in zip(areas, places, holds):
        if (area, place) in assignedPlaces:
            # this unique (area, place) has already been assigned to someone
            assignedPeople.append(assignedPlaces[(area, place)])
            continue

        if assignmentCount[person] >= areasPerPerson:
            # the current person is already assigned to enough areas, move on to the next
            a = heldPeople.pop(person, None)
            heldAreas.pop(a, None)
            person += 1

        if area in heldAreas:
            # assign to the person held in this area
            p = heldAreas.pop(area)
            heldPeople.pop(p)
        else:
            # get the first non-held person. If we need to hold in this area, 
            # also make sure the person has at least 2 free assignment slots,
            # though if it's the last person assign to them anyway 
            p = person
            while p in heldPeople or (hold and holdAvailable and (areasPerPerson - assignmentCount[p] < 2)) and not p==maxPerson:
                p += 1

        assignmentCount.update([p])
        assignedPlaces[(area, place)] = p
        assignedPeople.append(p)

        if hold:
            if p==maxPerson:
                # mark that there are no more people available to perform holds
                holdAvailable = False

            # this area recurrs in an hour, mark that the person should be held here
            heldPeople[p] = area
            heldAreas[area] = p

    return assignedPeople

def allocatePeople(df, areasPerPerson=3):
    assignedPeople = getAssignedPeople(df, areasPerPerson=areasPerPerson)
    df = df.copy()
    df.loc[:,'Person'] = df['Person'].unique()[assignedPeople]
    return df

请注意在df['Person'].unique()中使用allocatePeople。这可以解决输入中重复人员的情况。假定输入中人员的顺序是应该分配这些人员的所需顺序。

我针对OP的示例输入(allocatePeopleexample1)以及我想到的一些边缘案例(我认为(?)符合OP的要求)测试了example2算法:

ds = dict(
example1 = ({
    'Time' : ['8:03:00','8:17:00','8:20:00','8:28:00','8:35:00','08:40:00','08:42:00','08:45:00','08:50:00'],                 
    'Place' : ['House 1','House 2','House 3','House 4','House 5','House 1','House 2','House 3','House 2'],                 
    'Area' : ['A','B','C','D','E','D','E','F','G'],     
    'On' : ['1','2','3','4','5','6','7','8','9'], 
    'Person' : ['Person 1','Person 2','Person 3','Person 4','Person 5','Person 4','Person 5','Person 6','Person 7'],   
    }),
example2 = ({
    'Time' : ['8:03:00','8:17:00','8:20:00','8:28:00','8:35:00','8:40:00','8:42:00','8:45:00','8:50:00'],                 
    'Place' : ['House 1','House 2','House 3','House 1','House 2','House 3','House 1','House 2','House 3'],                 
    'Area' : ['X','X','X','X','X','X','X','X','X'],     
    'On' : ['1','2','3','3','3','3','3','3','3'], 
    'Person' : ['Person 1','Person 1','Person 1','Person 1','Person 1','Person 1','Person 1','Person 1','Person 1'],   
    }),

long_repeats = ({
    'Time' : ['8:03:00','8:17:00','8:20:00','8:25:00','8:30:00','8:31:00','8:35:00','8:45:00','8:50:00'],                 
    'Place' : ['House 1','House 2','House 3','House 4','House 1','House 1','House 2','House 3','House 2'],                 
    'Area' : ['A','A','A','A','B','C','C','C','B'],  
    'Person' : ['Person 1','Person 1','Person 1','Person 2','Person 3','Person 4','Person 4','Person 4','Person 3'],   
    'On' : ['1','2','3','4','5','6','7','8','9'],                      
    }),
many_repeats = ({
    'Time' : ['8:03:00','8:17:00','8:20:00','8:28:00','8:35:00','08:40:00','08:42:00','08:45:00','08:50:00'],                 
    'Place' : ['House 1','House 2','House 3','House 4','House 1','House 1','House 2','House 1','House 2'],                 
    'Area' : ['A', 'B', 'C', 'D', 'D', 'E', 'E', 'F', 'F'],     
    'On' : ['1','2','3','4','5','6','7','8','9'], 
    'Person' : ['Person 1','Person 1','Person 1','Person 2','Person 3','Person 4','Person 3','Person 5','Person 6'],   
    }),
large_gap = ({
    'Time' : ['8:03:00','8:17:00','8:20:00','8:28:00','8:35:00','08:40:00','08:42:00','08:45:00','08:50:00'],                 
    'Place' : ['House 1','House 2','House 3','House 4','House 1','House 1','House 2','House 1','House 3'],                 
    'Area' : ['A', 'B', 'C', 'D', 'E', 'F', 'D', 'D', 'D'],     
    'On' : ['1','2','3','4','5','6','7','8','9'], 
    'Person' : ['Person 1','Person 1','Person 1','Person 2','Person 3','Person 4','Person 3','Person 5','Person 6'],   
    }),
different_times = ({
    'Time' : ['8:03:00','8:17:00','8:20:00','8:28:00','8:35:00','08:40:00','09:42:00','09:45:00','09:50:00'],                 
    'Place' : ['House 1','House 2','House 3','House 4','House 1','House 1','House 2','House 1','House 1'],                 
    'Area' : ['A', 'B', 'C', 'D', 'D', 'E', 'E', 'F', 'G'],     
    'On' : ['1','2','3','4','5','6','7','8','9'], 
    'Person' : ['Person 1','Person 1','Person 1','Person 2','Person 3','Person 4','Person 3','Person 5','Person 6'],   
    })
)

expectedPeoples = dict(
    example1 = [1,1,1,2,3,2,3,2,3],
    example2 = [1,1,1,1,1,1,1,1,1],
    long_repeats = [1,1,1,2,2,3,3,3,2],
    many_repeats = [1,1,1,2,2,3,3,2,3],
    large_gap = [1,1,1,2,3,3,2,2,3],
    different_times = [1,1,1,2,2,2,3,3,3],
)

for name,d in ds.items():
    df = pd.DataFrame(d)
    expected = ['Person %d' % i for i in expectedPeoples[name]]
    ap = allocatePeople(df)

    print(name, ap, sep='\n', end='\n\n')
    np.testing.assert_array_equal(ap['Person'], expected)

assert_array_equal语句通过,并且输出与OP的预期输出匹配:

example1
       Time    Place Area On    Person
0   8:03:00  House 1    A  1  Person 1
1   8:17:00  House 2    B  2  Person 1
2   8:20:00  House 3    C  3  Person 1
3   8:28:00  House 4    D  4  Person 2
4   8:35:00  House 5    E  5  Person 3
5  08:40:00  House 1    D  6  Person 2
6  08:42:00  House 2    E  7  Person 3
7  08:45:00  House 3    F  8  Person 2
8  08:50:00  House 2    G  9  Person 3

example2
      Time    Place Area On    Person
0  8:03:00  House 1    X  1  Person 1
1  8:17:00  House 2    X  2  Person 1
2  8:20:00  House 3    X  3  Person 1
3  8:28:00  House 1    X  3  Person 1
4  8:35:00  House 2    X  3  Person 1
5  8:40:00  House 3    X  3  Person 1
6  8:42:00  House 1    X  3  Person 1
7  8:45:00  House 2    X  3  Person 1
8  8:50:00  House 3    X  3  Person 1

我的测试用例的输出也符合我的期望:

long_repeats
      Time    Place Area    Person On
0  8:03:00  House 1    A  Person 1  1
1  8:17:00  House 2    A  Person 1  2
2  8:20:00  House 3    A  Person 1  3
3  8:25:00  House 4    A  Person 2  4
4  8:30:00  House 1    B  Person 2  5
5  8:31:00  House 1    C  Person 3  6
6  8:35:00  House 2    C  Person 3  7
7  8:45:00  House 3    C  Person 3  8
8  8:50:00  House 2    B  Person 2  9

many_repeats
       Time    Place Area On    Person
0   8:03:00  House 1    A  1  Person 1
1   8:17:00  House 2    B  2  Person 1
2   8:20:00  House 3    C  3  Person 1
3   8:28:00  House 4    D  4  Person 2
4   8:35:00  House 1    D  5  Person 2
5  08:40:00  House 1    E  6  Person 3
6  08:42:00  House 2    E  7  Person 3
7  08:45:00  House 1    F  8  Person 2
8  08:50:00  House 2    F  9  Person 3

large_gap
       Time    Place Area On    Person
0   8:03:00  House 1    A  1  Person 1
1   8:17:00  House 2    B  2  Person 1
2   8:20:00  House 3    C  3  Person 1
3   8:28:00  House 4    D  4  Person 2
4   8:35:00  House 1    E  5  Person 3
5  08:40:00  House 1    F  6  Person 3
6  08:42:00  House 2    D  7  Person 2
7  08:45:00  House 1    D  8  Person 2
8  08:50:00  House 3    D  9  Person 3

different_times
       Time    Place Area On    Person
0   8:03:00  House 1    A  1  Person 1
1   8:17:00  House 2    B  2  Person 1
2   8:20:00  House 3    C  3  Person 1
3   8:28:00  House 4    D  4  Person 2
4   8:35:00  House 1    D  5  Person 2
5  08:40:00  House 1    E  6  Person 2
6  09:42:00  House 2    E  7  Person 3
7  09:45:00  House 1    F  8  Person 3
8  09:50:00  House 1    G  9  Person 3

让我知道它是否可以完成您想要的一切,或者是否仍需要进行一些调整。我认为每个人都渴望看到您实现您的愿景。

答案 1 :(得分:1)

好吧,在我们深入研究问题的逻辑之前,值得做一些整理工作以整理数据并将其转换为更有用的格式:

#Create table of unique people
unique_people = df[['Person']].drop_duplicates().sort_values(['Person']).reset_index(drop=True)

#Reformat time column
df['Time'] = pd.to_datetime(df['Time'])

现在,了解问题的逻辑,将问题分解为多个阶段很有用。首先,我们将要基于“区域”及其之间的时间来创建单个作业(带有作业编号)。即一个小时内位于同一区域的工作可以共享相同的工作编号。

#Assign jobs
df= df.sort_values(['Area','Time']).reset_index(drop=True)
df['Job no'] = 0
current_job = 1   
df.loc[0,'Job no'] = current_job
for i in range(rows-1):
    prev_row = df.loc[i]
    row = df.loc[i+1]
    time_diff = (row['Time'] - prev_row['Time']).seconds //3600
    if (row['Area'] == prev_row['Area'])  & (time_diff == 0):
        pass
    else:
        current_job +=1
    df.loc[i+1,'Job no'] = current_job

此步骤现已完成,只需将“人员”分配给单个作业即可。

df= df.sort_values(['Job no']).reset_index(drop=True)
df['Person'] = ""
df_groups = df.groupby('Job no')
for group in df_groups:
    group_size = group[1].count()['Time']
    for person_idx in range(len(unique_people)):
        person = unique_people.loc[person_idx]['Person']
        person_count = df[df['Person']==person]['Person'].count()
        if group_size <= (3-person_count):
            idx = group[1].index.values
            df.loc[idx,'Person'] = person
            break

最后

df= df.sort_values(['Time']).reset_index(drop=True)
print(df)

我试图以一种更容易取消选择的方式对此进行编码,因此在这里可能会提高效率。但是,目的是阐明所使用的逻辑。

此代码在两个数据集上均提供了预期的结果,因此希望它能回答您的问题。

答案 2 :(得分:0)

在写我的other answer时,我慢慢地想到了一种想法,即OP的算法可能更易于实现,该算法的重点是工作(可以不同)而不是人(全部都一样)。这是一个以工作为中心的解决方案:

from collections import Counter
import numpy as np
import pandas as pd

def assignJob(job, assignedix, areasPerPerson):
    for i in range(len(assignedix)):
        if (areasPerPerson - len(assignedix[i])) >= len(job):
            assignedix[i].extend(job)
            return True
    else:
        return False

def allocatePeople(df, areasPerPerson=3):
    areas = df['Area'].values
    times = pd.to_datetime(df['Time']).values
    peopleUniq = df['Person'].unique()
    npeople = int(np.ceil(areas.size / float(areasPerPerson)))

    # search for repeated areas. Mark them if the next repeat occurs within an hour
    ixrep = np.argmax(np.triu(areas.reshape(-1, 1)==areas, k=1), axis=1)
    holds = np.zeros(areas.size, dtype=bool)
    holds[ixrep.nonzero()] = (times[ixrep[ixrep.nonzero()]] - times[ixrep.nonzero()]) < np.timedelta64(1, 'h')

    jobs =[]
    _jobdict = {}
    for i,(area,hold) in enumerate(zip(areas, holds)):
        if hold:
            _jobdict[area] = job = _jobdict.get(area, []) + [i]
            if len(job)==areasPerPerson:
                jobs.append(_jobdict.pop(area))
        elif area in _jobdict:
            jobs.append(_jobdict.pop(area) + [i])
        else:
            jobs.append([i])
    jobs.sort()

    assignedix = [[] for i in range(npeople)]
    for job in jobs:
        if not assignJob(job, assignedix, areasPerPerson):
            # break the job up and try again
            for subjob in ([sj] for sj in job):
                assignJob(subjob, assignedix, areasPerPerson)

    df = df.copy()
    for i,aix in enumerate(assignedix):
        df.loc[aix, 'Person'] = peopleUniq[i]
    return df

此版本的allocatePeople也经过了广泛的测试,并通过了我其他答案中所述的所有相同检查。

与我的其他解决方案相比,它确实具有更多的循环,因此它的效率可能略低(尽管仅当您的数据帧非常大(例如,1e6行以上)时才重要)。另一方面,它更短一些,我认为更简单易懂。