Question

我有一个数据框如下。

ID  Word       Synonyms
------------------------
1   drove      drive
2   office     downtown
3   everyday   daily
4   day        daily
5   work       downtown

我正在阅读一个句子，并希望用上面定义的同义词替换该句子中的单词。这是我的代码：

import nltk
import pandas as pd
import string

sdf = pd.read_excel('C:\synonyms.xlsx')
sd = sdf.apply(lambda x: x.astype(str).str.lower())
words = 'i drove to office everyday in my car'

#######

def tokenize(text):
    text = ''.join([ch for ch in text if ch not in string.punctuation])
    tokens = nltk.word_tokenize(text)
    synonym = synonyms(tokens)
    return synonym

def synonyms(words):
    for word in words:
        if(sd[sd['Word'] == word].index.tolist()):
            idx = sd[sd['Word'] == word].index.tolist()
            word = sd.loc[idx]['Synonyms'].item()
        else:
            word
    return word

print(tokenize(words))

上面的代码标记了输入句子。我想实现以下输出：

：i drove to office everyday in my car
Out ：i drive to downtown daily in my car

但我得到的输出是

Out ：car

如果我跳过synonyms函数，那么我的输出没有问题，并且被分成单个单词。我试图理解我在synonyms函数中做错了什么。另外，请告知是否有更好的解决方案来解决这个问题。

Answer 1

我会利用Pandas / NumPy索引。由于您的同义词映射是多对一的，因此您可以使用Word列重新编制索引。

sd = sd.applymap(str.strip).applymap(str.lower).set_index('Word').Synonyms
print(sd)

Word
drove          drive
office      downtown
everyday       daily
day            daily
Name: Synonyms, dtype: object

然后，您可以轻松地将令牌列表与其各自的同义词对齐。

words = nltk.word_tokenize(u'i drove to office everyday in my car')
sentence = sd[words].reset_index()
print(sentence)

       Word  Synonyms
0         i       NaN
1     drove     drive
2        to       NaN
3    office  downtown
4  everyday     daily
5        in       NaN
6        my       NaN
7       car       NaN

现在，仍然需要使用Synonyms中的令牌，然后再回到Word。这可以通过

来实现

sentence = sentence.Synonyms.fillna(sentence.Word)
print(sentence.values)

[u'i' 'drive' u'to' 'downtown' 'daily' u'in' u'my' u'car']

Answer 2

import re
import pandas as pd
sdf = pd.read_excel('C:\synonyms.xlsx')
rep = dict(zip(sdf.Word, sdf.Synonyms)) #convert into dictionary

words = "i drove to office everyday in my car"
rep = dict((re.escape(k), v) for k, v in rep.iteritems())
pattern = re.compile("|".join(rep.keys()))
rep = pattern.sub(lambda m: rep[re.escape(m.group(0))], words)

print rep

输出

i drive to downtown daily in my car

礼貌：https://stackoverflow.com/a/6117124/6626530

通过检查pandas数据帧替换单词

2 个答案: