我有一个像这样的文本文件:
444537110 3 11112111022002200022022111121222002...
输入文件中的最后一个字段的长度为50k个字符,并且只能是0或1。2。我想要这个最终字段的一个热编码版本。所以我的预期结果是这样的数据框:
id chip g1_0 g1_1 g1_2 g2_0 g2_1 g2_2 g3_0 g3_1 g3_2 g4_0 ...
444537110 3 0 1 0 0 1 0 0 1 0 0
我通过读取输入文件创建了一个初始数据框:
df = pd.read_csv('test.txt', index_col=0, sep='\s+', header=None, names = ['chip', 'genos'])
这将创建一个包含3列的数据框:
id chip genos
444537110 3 1111211102200220000022022111121222000200022002...
我认为我可以使用如下所示的方法创建初始的单个列,然后使用pandas get_dummies函数进行一次热编码,但是我无法创建单个列。我尝试过
[c for c in df['genos'].str]
但这不会分隔字符
我在这里查看过类似的问题和答案:How can I one hot encode in Python?
但这仅处理一种热编码,而不处理拆分非常大的列所带来的复杂性
答案 0 :(得分:1)
首先创建DataFrame
,并将字符串转换为列表,然后调用get_dummies
:
df1 = pd.DataFrame([list(x) for x in df['genos']], index=df.index).add_prefix('g')
df2 = pd.get_dummies(df1)
如果需要将新的columnt添加到原始列中(如果可能,则缺少某些组合),将DataFrame.reindex
用于带有_
的拆分列以及MultiIndex.from_product
创建的所有组合:
df1 = pd.DataFrame([list(x) for x in df.pop('genos')], index=df.index).add_prefix('g')
df2 = pd.get_dummies(df1)
splitted = df2.columns.str.split('_')
df2.columns = [splitted.str[0].astype(int) + 1, splitted.str[1].astype(int)]
#
mux = pd.MultiIndex.from_product([df2.columns.get_level_values(0), [0,1,2]])
df2 = df2.reindex(mux, axis=1, fill_value=0)
df2.columns = [f'g{a}_{b}' for a, b in df2.columns]
print (df2)
g1_0 g1_1 g1_2 g2_0 g2_1 g2_2 g3_0 g3_1 g3_2 g4_0 ... g32_2 \
0 0 1 0 0 1 0 0 1 0 0 ... 1
g33_0 g33_1 g33_2 g34_0 g34_1 g34_2 g35_0 g35_1 g35_2
0 1 0 0 1 0 0 0 0 1
[1 rows x 105 columns]
答案 1 :(得分:1)
考虑到@Dan对您将要以50k * 3列结尾这一事实的评论,您可以这样做来获得所需的输出:
string ="444537110 3 11112111022002200022022111121222002"
df = pd.DataFrame([string.split(" ")],columns=['id','chip','genos'])
max_number_of_genes = int(df.genos.apply(lambda x : len([y for y in x])).max())
#Create columns
for gene in range(1,max_number_of_genes+1):
for y in range(4):
df['g{}_{}'.format(gene, y)] = 0
#Iterating over genos values
for row_number, row in df.iterrows():
genos = [int(x) for x in df.iloc[row_number, 2]]
for gene_number, gene in enumerate(genos):
df.loc[row_number, 'g{}_{}'.format(gene_number+1, gene)] = 1
print(df)
输出
+----+------------+-------+--------------------------------------+-------+-------+-------+-------+-------+-------+-------+------+--------+--------+--------+--------+--------+--------+--------+--------+--------+-------+
| | id | chip | genos | g1_0 | g1_1 | g1_2 | g1_3 | g2_0 | g2_1 | g2_2 | ... | g33_2 | g33_3 | g34_0 | g34_1 | g34_2 | g34_3 | g35_0 | g35_1 | g35_2 | g35_3 |
+----+------------+-------+--------------------------------------+-------+-------+-------+-------+-------+-------+-------+------+--------+--------+--------+--------+--------+--------+--------+--------+--------+-------+
| 0 | 444537110 | 3 | 11112111022002200022022111121222002 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | ... | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
+----+------------+-------+--------------------------------------+-------+-------+-------+-------+-------+-------+-------+------+--------+--------+--------+--------+--------+--------+--------+--------+--------+-------+
答案 2 :(得分:0)
如果仅拆分5万个字符,则可以使用原始Python(出于可读性考虑):
>>> a,b,c = zip(*[{0:(1,0,0),1:(0,1,0),2:(0,0,1)}[int(c)] for c in df['genos'][0]])
>>> a,b,c
((0, 0, 0, 0, 0, 0, ...), (1, 1, 1, 1, 0, 1, ...), (0, 0, 0, 0, 1, 0, ...))