如何在此数据集中插入缺失的行?

时间:2018-01-04 19:21:31

标签: python pandas numpy

Data set

我想要做的是每当缺少一行时将记录插入数据集。

如果查看上面的数据集,它包含3列属性,然后是2个数值。第三列TTF是增量的,不应跳过任何值。在这个例子中,它缺少2行,显示在底部。所以我想让我的代码做的就是将这两行插入到结果集中(即计算机 - 显示器缺少TTF为5,电视 - 电源缺少TTF为6.我会将修复值设置为0,并且运行总值与上一行相同。)

我以为我会通过分割列名并递归遍历前2个来接近它,然后是第1个到第8个。

for i in range(len(Product)):
    for j in range(len(Module)):
        for k in range(1, 8):  
            # Check if the Repair value is there if not make it 0
            # If Repair value is missing, look up previous Running Total

这看起来是最好的方法吗?任何帮助实现这一目标的实际代码都将非常感激。

编辑:这是DF中的代码读取,因为基于excel屏幕截图,这似乎令人困惑。

>>> import pandas as pd
>>> 
>>> df = pd.read_csv('minimal.csv')
>>> 
>>> df
       Product         Module   TTF   Repair   Running Total
0     Computer        Display     1        3               3
1     Computer        Display     2        2               5
2     Computer        Display     3        1               6
3     Computer        Display     4        5              11
4     Computer        Display     6        4              15
5     Computer        Display     7        3              18
6     Computer        Display     8        2              20
7   Television   Power Supply     1        7               7
8   Television   Power Supply     2        6              13
9   Television   Power Supply     3        4              17
10  Television   Power Supply     4        5              22
11  Television   Power Supply     5        6              28
12  Television   Power Supply     7        7              35
13  Television   Power Supply     8        8              43

1 个答案:

答案 0 :(得分:2)

让我们使用reindexnp.arange顺序为缺失的号码创建新的TTF:

df = pd.DataFrame({'Product':['Computer']*7 + ['Television']*7,'Module':['Display']*7 + ['Power Supply']*7,
                 'TTF':[1,2,3,4,6,7,8,1,2,3,4,5,7,8],'Repair':np.random.randint(1,8,14)})

df['Running Total'] = df['Repair'].cumsum()

print(df)

输入数据框:

          Module     Product  Repair  TTF  Running Total
0        Display    Computer       6    1              6
1        Display    Computer       2    2              8
2        Display    Computer       2    3             10
3        Display    Computer       4    4             14
4        Display    Computer       2    6             16
5        Display    Computer       3    7             19
6        Display    Computer       6    8             25
7   Power Supply  Television       3    1             28
8   Power Supply  Television       3    2             31
9   Power Supply  Television       5    3             36
10  Power Supply  Television       6    4             42
11  Power Supply  Television       4    5             46
12  Power Supply  Television       2    7             48
13  Power Supply  Television       2    8             50


df_out = df.set_index('TTF').groupby(['Product','Module'], group_keys=False).apply(lambda x: x.reindex(np.arange(1,9)))

df_out['repair'] = df_out['Repair'].fillna(0)

df_out = df_out.ffill().reset_index()

print(df_out)

输出:

    TTF        Module     Product  Repair  Running Total  repair
0     1       Display    Computer     6.0            6.0     6.0
1     2       Display    Computer     2.0            8.0     2.0
2     3       Display    Computer     2.0           10.0     2.0
3     4       Display    Computer     4.0           14.0     4.0
4     5       Display    Computer     4.0           14.0     0.0
5     6       Display    Computer     2.0           16.0     2.0
6     7       Display    Computer     3.0           19.0     3.0
7     8       Display    Computer     6.0           25.0     6.0
8     1  Power Supply  Television     3.0           28.0     3.0
9     2  Power Supply  Television     3.0           31.0     3.0
10    3  Power Supply  Television     5.0           36.0     5.0
11    4  Power Supply  Television     6.0           42.0     6.0
12    5  Power Supply  Television     4.0           46.0     4.0
13    6  Power Supply  Television     4.0           46.0     0.0
14    7  Power Supply  Television     2.0           48.0     2.0
15    8  Power Supply  Television     2.0           50.0     2.0