我有一个数据框(带有DateTime索引),其中一些列包含列表,每个列都有6个元素。
In: dframe.head()
Out:
A B \
timestamp
2017-05-01 00:32:25 30 [-3512, 375, -1025, -358, -1296, -4019]
2017-05-01 00:32:55 30 [-3519, 372, -1026, -361, -1302, -4020]
2017-05-01 00:33:25 30 [-3514, 371, -1026, -360, -1297, -4018]
2017-05-01 00:33:55 30 [-3517, 377, -1030, -363, -1293, -4027]
2017-05-01 00:34:25 30 [-3515, 372, -1033, -361, -1299, -4025]
C D
timestamp
2017-05-01 00:32:25 [1104, 1643, 625, 1374, 5414, 2066] 49.93
2017-05-01 00:32:55 [1106, 1643, 622, 1385, 5441, 2074] 49.94
2017-05-01 00:33:25 [1105, 1643, 623, 1373, 5445, 2074] 49.91
2017-05-01 00:33:55 [1105, 1646, 620, 1384, 5438, 2076] 49.91
2017-05-01 00:34:25 [1104, 1645, 613, 1374, 5431, 2082] 49.94
我有一个字典dict_of_dfs
,我希望用6个数据框填充
dict_of_dfs = {1: df1, 2:df2, 3:df3, 4:df4, 5:df5, 6:df6}
其中 ith 数据框包含每个列表中的 ith 项,因此dict中的第一个数据帧将是:
In:df1
Out:
A B C D
timestamp
2017-05-01 00:32:25 30 -3512 1104 49.93
2017-05-01 00:32:55 30 -3519 1106 49.94
2017-05-01 00:33:25 30 -3514 1105 49.91
2017-05-01 00:33:55 30 -3517 1105 49.91
2017-05-01 00:34:25 30 -3515 1104 49.94
等等。 实际的数据框有比这更多的列和数千行。 什么是最简单,最python的转换方式?
答案 0 :(得分:3)
您可以将词典理解与assign
一起使用,并将lists
的选择值用于str[0]
,str[1]
:
N = 6
dfs = {i:df.assign(B=df['B'].str[i-1], C=df['C'].str[i-1]) for i in range(1,N + 1)}
print(dfs[1])
timestamp A B C D
0 2017-05-01 00:32:25 30 -3512 1104 49.93
1 2017-05-01 00:32:55 30 -3519 1106 49.94
2 2017-05-01 00:33:25 30 -3514 1105 49.91
3 2017-05-01 00:33:55 30 -3517 1105 49.91
4 2017-05-01 00:34:25 30 -3515 1104 49.94
另一种解决方案:
dfs = {i:df.apply(lambda x: x.str[i-1] if type(x.iat[0]) == list else x) for i in range(1,7)}
print(dfs[1])
timestamp A B C D
0 2017-05-01 00:32:25 30 -3512 1104 49.93
1 2017-05-01 00:32:55 30 -3519 1106 49.94
2 2017-05-01 00:33:25 30 -3514 1105 49.91
3 2017-05-01 00:33:55 30 -3517 1105 49.91
4 2017-05-01 00:34:25 30 -3515 1104 49.94
<强>计时强>:
df = pd.concat([df]*10000).reset_index(drop=True)
In [185]: %timeit {i:df.assign(B=df['B'].str[i-1], C=df['C'].str[i-1]) for i in range(1,N+1)}
1 loop, best of 3: 420 ms per loop
In [186]: %timeit {i:df.apply(lambda x: x.str[i-1] if type(x.iat[0]) == list else x) for i in range(1,7)}
1 loop, best of 3: 447 ms per loop
In [187]: %timeit {(i+1):df.applymap(lambda x: x[i] if type(x) == list else x) for i in range(6)}
1 loop, best of 3: 881 ms per loop
答案 1 :(得分:3)
df = pd.DataFrame({'A': {'2017-05-01 00:32:25': 30,
'2017-05-01 00:32:55': 30,
'2017-05-01 00:33:25': 30,
'2017-05-01 00:33:55': 30,
'2017-05-01 00:34:25': 30},
'B': {'2017-05-01 00:32:25': [-3512, 375, -1025, -358, -1296, -4019],
'2017-05-01 00:32:55': [-3519, 372, -1026, -361, -1302, -4020],
'2017-05-01 00:33:25': [-3514, 371, -1026, -360, -1297, -4018],
'2017-05-01 00:33:55': [-3517, 377, -1030, -363, -1293, -4027],
'2017-05-01 00:34:25': [-3515, 372, -1033, -361, -1299, -4025]},
'C': {'2017-05-01 00:32:25': [1104, 1643, 625, 1374, 5414, 2066],
'2017-05-01 00:32:55': [1106, 1643, 622, 1385, 5441, 2074],
'2017-05-01 00:33:25': [1105, 1643, 623, 1373, 5445, 2074],
'2017-05-01 00:33:55': [1105, 1646, 620, 1384, 5438, 2076],
'2017-05-01 00:34:25': [1104, 1645, 613, 1374, 5431, 2082]},
'D': {'2017-05-01 00:32:25': 49.93,
'2017-05-01 00:32:55': 49.94,
'2017-05-01 00:33:25': 49.1,
'2017-05-01 00:33:55': 49.91,
'2017-05-01 00:34:25': 49.94}})
使用dict理解构造df dict。 sub df是使用applymap函数生成的。它可以使用6个元素的列表转换所有列:
dict_of_dfs = {(i+1):df.applymap(lambda x: x[i] if type(x) == list else x) for i in range(6)}
print(dict_of_dfs[1])
A B C D
2017-05-01 00:32:25 30 -3512 1104 49.93
2017-05-01 00:32:55 30 -3519 1106 49.94
2017-05-01 00:33:25 30 -3514 1105 49.10
2017-05-01 00:33:55 30 -3517 1105 49.91
2017-05-01 00:34:25 30 -3515 1104 49.94
print(dict_of_dfs[2])
A B C D
2017-05-01 00:32:25 30 375 1643 49.93
2017-05-01 00:32:55 30 372 1643 49.94
2017-05-01 00:33:25 30 371 1643 49.10
2017-05-01 00:33:55 30 377 1646 49.91
2017-05-01 00:34:25 30 372 1645 49.94