由于tf-idf矢量化器只会在遇到新标签时崩溃,我正试图从我的新输入中删除新标签。如何更新数据框的列值?我在做:
def clean_unseen(dfcol, vectorizer):
cleanedstring = ""
for entry in dfcol:
for word in entry.split():
if word in vectorizer.vocabulary_:
cleanedstring = cleanedstring + " " + word
print(cleanedstring)
entry = cleanedstring
cleanedstring = ""
return dfcol
示例:
tfifgbdf_vect= TfidfVectorizer()
s2 = pd.Series(['the cat', 'awesome xyz', 'f_g_h lol asd'])
tfifgbdf_vect.fit_transform(s2)
s3 = pd.Series(['the dog the awesome xyz', 'xyz lol asd', 'f_g_h lol aha'])
clean_unseen(s3, tfifgbdf_vect)
然而,这将使原始列保持不变:
Output:
0 the dog the awesome xyz
1 xyz lol asd
2 f_g_h lol aha
dtype: object
答案 0 :(得分:0)
由于系列中的单个条目不是对象,因此它始终是深层副本而不是引用,您需要明确更改 -
def clean_unseen(dfcol, vectorizer):
dfc1 = []
cleanedstring = ""
for entry in dfcol:
for word in entry.split():
if word in vectorizer.vocabulary_:
cleanedstring = cleanedstring + " " + word
#print(cleanedstring)
#entry = cleanedstring
dfc1.append(cleanedstring)
cleanedstring = ""
return pd.Series(dfc1)
tfifgbdf_vect= TfidfVectorizer()
s2 = pd.Series(['the cat', 'awesome xyz', 'f_g_h lol asd'])
tfifgbdf_vect.fit_transform(s2)
s3 = pd.Series(['the dog the awesome xyz', 'xyz lol asd', 'f_g_h lol aha'])
clean_unseen(s3, tfifgbdf_vect)