Question

我正在尝试腌制我的文本分类模型并重新加载到flask应用程序界面中。

我有一个用作分析程序的特定功能，称为 split_into_lemmas

def split_into_lemmas(message):
    message = unicode(message, 'utf8').lower()
    words = TextBlob(message).words
    # for each word, take its "base form" = lemma 
    return [word.lemma for word in words]

from sklearn.pipeline import Pipeline

count_vect = CountVectorizer(analyzer=split_into_lemmas,ngram_range= (1, 3), encoding='utf8',stop_words =None)
tfidf_transformer = TfidfTransformer()
text_clf = Pipeline([('vect', count_vect), ('tdif', tfidf_transformer), ('clf', best_svc)])

%%time
text_clf.fit(X=data['Condition'], y=data['condition_predict'])

我拟合模型并通过腌制将其保存

_ = joblib.dump(text_clf, 'classification_pipeline.pkl')

另一方面
当我尝试重新加载管道

import pandas as pd 
import pickle
from sklearn.feature_extraction.text import CountVectorizer
from textblob import TextBlob
from sklearn.externals import joblib

clf_pipeline = open('C:/Users/Falco/Desktop/directory/WRMD_paper/classification_pipeline.pkl','rb')
clf = joblib.load(clf_pipeline)

我收到以下错误消息

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-bb0859b3946a> in <module>()
      6 
      7 clf_pipeline = open('C:/Users/Falco/Desktop/directory/WRMD_paper/classification_pipeline.pkl','rb')
----> 8 clf = joblib.load(clf_pipeline)

C:\ProgramData\Anaconda2\lib\site-packages\sklearn\externals\joblib\numpy_pickle.pyc in load(filename, mmap_mode)
    586         filename = getattr(fobj, 'name', '')
    587         with _read_fileobject(fobj, filename, mmap_mode) as fobj:
--> 588             obj = _unpickle(fobj)
    589     else:
    590         with open(filename, 'rb') as f:

C:\ProgramData\Anaconda2\lib\site-packages\sklearn\externals\joblib\numpy_pickle.pyc in _unpickle(fobj, filename, mmap_mode)
    524     obj = None
    525     try:
--> 526         obj = unpickler.load()
    527         if unpickler.compat_mode:
    528             warnings.warn("The file '%s' has been generated with a "

C:\ProgramData\Anaconda2\lib\pickle.pyc in load(self)
    862             while 1:
    863                 key = read(1)
--> 864                 dispatch[key](self)
    865         except _Stop, stopinst:
    866             return stopinst.value

C:\ProgramData\Anaconda2\lib\pickle.pyc in load_global(self)
   1094         module = self.readline()[:-1]
   1095         name = self.readline()[:-1]
-> 1096         klass = self.find_class(module, name)
   1097         self.append(klass)
   1098     dispatch[GLOBAL] = load_global

C:\ProgramData\Anaconda2\lib\pickle.pyc in find_class(self, module, name)
   1130         __import__(module)
   1131         mod = sys.modules[module]
-> 1132         klass = getattr(mod, name)
   1133         return klass
   1134 

AttributeError: 'module' object has no attribute 'split_into_lemmas'

当我在笔记本中重新声明该函数时，模型可以很好地加载并运行，但是当我将笔记本另存为.py文件并以flask应用程序运行时，它无法运行，并给我同样的错误。

有人可以帮助我正确保存管道，以便不必声明函数吗？

Answer 1

在重新加载泡菜时，还需要定义split_into_lemmas。

重新加载腌制的sklearn管道时出现问题。未导入Countvector分析器功能

1 个答案: