我有一本字典,其中每个词都作为键和对应的整数值,例如:
{'me': 41, 'are': 21, 'the': 0}
我有一个数据框,其中有一列已被标记的单词列表,例如:
['I', 'liked', 'the', 'color', 'of', 'this', 'top']
['Just', 'grabbed', 'this', 'today', 'great', 'find']
如何将这些单词中的每个单词编码为字典中相应的值。例如:
[56, 78, 5, 1197, 556, 991, 40]
答案 0 :(得分:5)
怎么办
word2key = {'me': 41, 'are': 21, 'the': 0}
words = ['Just', 'grabbed', 'this', 'today', 'great', 'find']
default = 'unknown'
output = [word2key.get(x, default) for x in words]
如果要将x.lower()
和'Just'
映射到相同的值,则可能要使用'just'
。
答案 1 :(得分:1)
以下内容使用字典(final_dictionary
)确定单词的ID。如果您具有预设的ID字典,那就太好了。
def encode_tokens(tokens):
encoded_tokens = tokens[:]
for i, token in enumerate(tokens):
if token in final_dictionary:
encoded_tokens[i] = final_dictionary[token]
return encoded_tokens
print(encode_tokens(tokens))
如果您正在动态分配ID,则可以实现一个类(请参见下面的内容)。但是,如果您有一个预先定义的id字典,则可以传入关键字参数di
:
token_words_1 = ['I', 'liked', 'the', 'color', 'of', 'this', 'top']
token_words_2 = ['I', 'liked', 'to', 'test', 'repeat', 'words']
class AutoId:
def __init__(self, **kwargs):
self.di = kwargs.get("di", {})
self.loc = 0
def get(self, value):
if value not in self.di:
self.di[value] = self.loc
self.loc += 1
return self.di[value]
def get_list(self, li):
return [*map(self.get, li)]
encoding = AutoId()
print(encoding.get_list(token_words_1))
print(encoding.get_list(token_words_2))
答案 2 :(得分:1)
假设您的字典位于名为d
的变量中,并且列表名为l
:
d = {'me': 41, 'are': 21, 'the': 0}
l = ['I', 'liked', 'the', 'color', 'of', 'this', 'top']
print(l)
c = 0
while c < len(l):
try:
l[c] = d[l[c]]
except:
l[c] = None
c += 1
print(l)
答案 3 :(得分:1)
from itertools import chain
import numpy as np
# d = {'me': 41, 'are': 21, 'the': 0}
l1 = ['I', 'liked', 'the', 'color', 'of', 'this', 'top']
l2 = ['Just', 'grabbed', 'this', 'today', 'great', 'find']
# This is just for data generation for the sake of a complete example.
# Use your already given d here instead.
d = {k: np.random.randint(10) for k in chain(l1, l2)}
print(d)
l1_d = [d.get(k, 0) for k in l1] # <- this is the actual command you need
print(l1_d)
l2_d = [d.get(k, 0) for k in l2]
print(l2_d)
结果:
{'I': 3, 'liked': 3, 'the': 8, 'color': 7, 'of': 3, 'this': 5,
'top': 3, 'Just': 6, 'grabbed': 0, 'today': 0, 'great': 7, 'find': 0}
[3, 3, 8, 7, 3, 5, 3]
[6, 0, 5, 0, 7, 0]