将数据拆分为训练和测试集

Question

我正在尝试训练分类器按类型标记电影。电影的情节可能属于不止一种类型。 This is what my dataframe looks like 当我试图弄清楚测试集中每个电影类型的准确度分数时，我不断收到此错误消息。错误信息： TypeError：'FramePlotMethods'对象不可迭代

有人可以解释我做错了吗？

我从https://github.com/davidsbatista/text-classification/blob/master/movies_genres_en.csv.bz2

获取了电影数据

以下是从头开始的代码

df = pd.read_csv("movies_genres_en.csv", delimiter='\t')
df.drop('plot_lang', axis=1, inplace=True)
df.info()

# using for loop get a count of movies by genre
df_genres = df.drop(['plot', 'title'], axis=1)
counts = []
categories = list(df_genres.columns.values)
for i in categories:
counts.append((i, df_genres[i].sum()))
df_stats = pd.DataFrame(counts, columns = ['genre','#movies'])
df_stats

# Create a fuction to clean the text
def clean_text(text):
text = text.lower()
text = re.sub(r"what's", "what is ", text)
text = re.sub(r"\'s", " ", text)
text = re.sub(r"\'ve", " have ", text)
text = re.sub(r"can't", "can not ", text)
text = re.sub(r"n't", " not ", text)
text = re.sub(r"i'm", "i am ", text)
text = re.sub(r"\'re", " are ", text)
text = re.sub(r"\'d", " would ", text)
text = re.sub(r"\'ll", " will ", text)
text = re.sub(r"\'scuse", " excuse ", text)
text = re.sub('\W', ' ', text)
text = re.sub('\s+', ' ', text)
text = text.strip(' ')
return text    

# clean up the text in plot
df['plot'] = df['plot'].map(lambda com : clean_text(com))

将数据拆分为训练和测试集

train, test = train_test_split(df, random_state=42, test_size = 0.33, 
shuffle=True)
x_train = train.plot
x_test = test.plot

# Define a pipeline combining a text feature extractor with multi lable 
classifier
NB_pipeline = Pipeline([
            ('tfidf', TfidfVectorizer(stop_words='english')),
            ('clf', OneVsRestClassifier(MultinomialNB(
                fit_prior=True, class_prior=None))),
        ])

NB_pipeline.fit(x_train, train[genre])
prediction = NB_pipeline.predict(x_test)
accuracy_score(test[genre], prediction)

运行最后一个代码块时出现此错误 TypeError：'FramePlotMethods'对象不可迭代

创建df时我做错了什么？

多标签分类器代码错误

将数据拆分为训练和测试集

0 个答案: