Question

我有一个pandas数据框，我在Python中从defaultdict读取，但有些列的长度不同。以下是数据的外观：

Date      col1    col2    col3    col4    col5
01-01-15  5       12      1      -15      10
01-02-15  7       0       9       11      7
01-03-15          6       1       2       18
01-04-15          9       8       10
01-05-15         -4               7
01-06-15         -11             -1
01-07-15          6

我可以像NaN那样用空白填充空白：

pd.DataFrame.from_dict(pred_dict, orient='index').T

给出了：

Date      col1    col2    col3    col4    col5
01-01-15  5       12      1      -15      10
01-02-15  7       0       9       11      7
01-03-15  NaN     6       1       2       18
01-04-15  NaN     9       8       10      NaN
01-05-15  NaN    -4       NaN     7       NaN
01-06-15  NaN    -11      NaN    -1       NaN
01-07-15  NaN     6       NaN     NaN     NaN

然而，我真正想要的是一种预先添加NaN而不是将它们附加到最后的方法，以便数据看起来像这样：

Date      col1    col2    col3    col4    col5
01-01-15  NaN     12      NaN     NaN     NaN
01-02-15  NaN     0       NaN    -15      NaN
01-03-15  NaN     6       NaN     11      NaN
01-04-15  NaN     9       1       2       NaN
01-05-15  NaN    -4       9       10      10
01-06-15  5      -11      1       7       7
01-07-15  7       6       8      -1       18

有一种简单的方法吗？

您可以使用以下代码重新创建字典：

import pandas as pd
from collections import defaultdict

d = defaultdict(list)
d["Date"].extend([
    "01-01-15", 
    "01-02-15", 
    "01-03-15", 
    "01-04-15", 
    "01-05-15",
    "01-06-15",
    "01-07-15"
])
d["col1"].extend([5, 7])
d["col2"].extend([12, 0, 6, 9, -4, -11, 6])
d["col3"].extend([1, 9, 1, 8])
d["col4"].extend([-15, 11, 2, 10, 7, -1])
d["col5"].extend([10, 7, 18])

Answer 1

您可以使用Series.shift来演示Series / DataFrame。遗憾的是，您无法传递句点数组 - 您必须将每列移动一个整数值。

s = df.isnull().sum()
for col, periods in s.iteritems():
    df[col] = df[col].shift(periods)

Answer 2

对您的earlier question：

的itertools解决方案进行了一些修改

pd.DataFrame(list(itertools.zip_longest(*[reversed(i) for i in d.values()]))[::-1], columns=d.keys()).sort_index(axis=1)
Out[143]: 
       Date  col1  col2  col3  col4  col5
0  01-01-15   NaN    12   NaN   NaN   NaN
1  01-02-15   NaN     0   NaN -15.0   NaN
2  01-03-15   NaN     6   NaN  11.0   NaN
3  01-04-15   NaN     9   1.0   2.0   NaN
4  01-05-15   NaN    -4   9.0  10.0  10.0
5  01-06-15   5.0   -11   1.0   7.0   7.0
6  01-07-15   7.0     6   8.0  -1.0  18.0

Answer 3

反转字典中的每个列表：

pf32bit

Answer 4

这是一种矢量化方法，使用pd.DataFrame.from_dict来获取通常情况下的数据帧。一旦我们获得了常规的2D数据，就可以轻松地以矢量化的方式翻转和屏蔽并获得所需的输出数据帧。

下面列出了实施 -

# Get the normal case output
df = pd.DataFrame.from_dict(d, orient='index').T

# Use masking to flip and select flipped elements to re-create expected df
colmask = df.columns!='Date'
arr = np.array(df.ix[:,colmask].values, dtype=np.float).T
mask = ~np.isnan(arr)
out_arr = np.full(mask.shape,np.nan)
out_arr[mask[:,::-1]] = arr[mask]
df.ix[:,colmask] = out_arr.T

示例运行 -

In [209]: d.values()
Out[209]: 
[[-15, 11, 2, 10, 7, -1],
 [10, 7, 18],
 [12, 0, 6, 9, -4, -11, 6],
 [1, 9, 1, 8],
 [5, 7],
 ['01-01-15',
  '01-02-15',
  '01-03-15',
  '01-04-15',
  '01-05-15',
  '01-06-15',
  '01-07-15']]

In [210]: df
Out[210]: 
  col4 col5 col2 col3 col1      Date
0  NaN  NaN   12  NaN  NaN  01-01-15
1  -15  NaN    0  NaN  NaN  01-02-15
2   11  NaN    6  NaN  NaN  01-03-15
3    2  NaN    9    1  NaN  01-04-15
4   10   10   -4    9  NaN  01-05-15
5    7    7  -11    1    5  01-06-15
6   -1   18    6    8    7  01-07-15

使用from_dict预先添加而不是在熊猫中附加NaN

4 个答案: