Question

我有一个数据行，每行包含一个句子。我需要在这些句子中搜索某些单词的出现。这是我目前的操作方式：

import pandas as pd

p = pd.DataFrame({"sentence" : ["this is a test", "yet another test", "now two tests", "test a", "no test"]})

test_words = ["yet", "test"]
p["word_test"] = ""
p["word_yet"]  = ""

for i in range(len(p)):
    for word in test_words:
        p.loc[i]["word_"+word] = p.loc[i]["sentence"].find(word)

这可以按预期工作，但是可以优化它吗？对于大型数据帧，运行速度相当慢

Answer 1

IIUC，使用简单的列表理解并为每个单词调用str.find：

u = pd.DataFrame({
    # 'word_{}'.format(w)
    f'word_{w}': df.sentence.str.find(w) for w in test_words}, index=df.index)
u
   word_yet  word_test
0        -1         10
1         0         12
2        -1          8
3        -1          0
4        -1          3

pd.concat([df, u], axis=1)

           sentence  word_yet  word_test
0    this is a test        -1         10
1  yet another test         0         12
2     now two tests        -1          8
3            test a        -1          0
4           no test        -1          3

Answer 2

您可以使用str.find

p['word_test'] = p.sentence.str.find('test')
p['word_yet'] = p.sentence.str.find('yet')

    sentence         word_test  word_yet    word_yest
0   this is a test   10         -1          -1
1   yet another test 12          0          0
2   now two tests    8          -1          -1
3   test a           0          -1          -1
4   no test          3          -1          -1

Answer 3

由于您提到使用np.char.find会带来更好的性能

df=pd.DataFrame(data=[np.char.find(p.sentence.values.astype(str),x) for x in test_words],index=test_words,columns=p.index)
pd.concat([p,df.T],axis=1)
Out[32]: 
           sentence  yet  test
0    this is a test   -1    10
1  yet another test    0    12
2     now two tests   -1     8
3            test a   -1     0
4           no test   -1     3

在多个字符串中搜索多个单词

3 个答案: