Question

我有一个列表的字典（具有可变长度），我期待从它创建一个有效的方法来创建一个数据帧。
假设我有最小的列表长度，所以我可以截断更大的列表创建Dataframe时列出。
这是我的虚拟代码

data_dict = {'a': [1,2,3,4], 'b': [1,2,3], 'c': [2,45,67,93,82,92]}
min_length = 3

我可以拥有10k或20k密钥的字典，因此寻找一种有效的方法来创建像下面这样的DataFrame

>>> df
   a  b   c
0  1  1   2
1  2  2  45
2  3  3  67

Answer 1

您可以在values中过滤dict dict comprehension，然后DataFrame完美无缺：

print ({k:v[:min_length] for k,v in data_dict.items()})
{'b': [1, 2, 3], 'c': [2, 45, 67], 'a': [1, 2, 3]}


df = pd.DataFrame({k:v[:min_length] for k,v in data_dict.items()})
print (df)
   a  b   c
0  1  1   2
1  2  2  45
2  3  3  67

如果min_length添加Series：

，则某些长度可能会减少

data_dict = {'a': [1,2,3,4], 'b': [1,2], 'c': [2,45,67,93,82,92]}
min_length = 3

df = pd.DataFrame({k:pd.Series(v[:min_length]) for k,v in data_dict.items()})
print (df)
   a    b   c
0  1  1.0   2
1  2  2.0  45
2  3  NaN  67

<强>计时：

In [355]: %timeit (pd.DataFrame({k:v[:min_length] for k,v in data_dict.items()}))
The slowest run took 5.32 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 520 µs per loop

In [356]: %timeit (pd.DataFrame({k:pd.Series(v[:min_length]) for k,v in data_dict.items()}))
The slowest run took 4.50 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 937 µs per loop

#Allen's solution
In [357]: %timeit (pd.DataFrame.from_dict(data_dict,orient='index').T.dropna())
1 loop, best of 3: 16.7 s per loop

时间安排的代码：

np.random.seed(123)
L = list('ABCDEFGH')
N = 500000
min_length = 10000

data_dict = {k:np.random.randint(10, size=np.random.randint(N)) for k in L}

Answer 2

单线解决方案：

#Construct the df horizontally and then transpose. Finally drop rows with nan.
pd.DataFrame.from_dict(data_dict,orient='index').T.dropna()
Out[326]: 
     a    b     c
0  1.0  1.0   2.0
1  2.0  2.0  45.0
2  3.0  3.0  67.0

从不等长度列表的dict创建一个DataFrame

2 个答案: