使用SciLearn Kit读取Pandas数据框时遇到问题

时间:2018-12-11 02:15:41

标签: python pandas scikit-learn

我是Python的新手,在使用Pandas创建的数据帧上使用SciLearn Kit时遇到麻烦。下面是代码:

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib as plt
import json
%matplotlib inline

data = pd.read_json('C:/Users/Desktop/Machine Learning/yelp_academic_dataset_business.json', lines=True, orient='columns', encoding='utf-8')
dataframe = pd.DataFrame(data)

list(dataframe)
subset_data = dataframe.loc[(dataframe.city == 'Toronto')]
print(subset_data)
documents = subset_data.to_dict('records')

from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer

no_features = 1000

# NMF is able to use tf-idf
tfidf_vectorizer = TfidfVectorizer(max_df=0.95, min_df=2, max_features=no_features, stop_words='english')
tfidf = tfidf_vectorizer.fit_transform(documents)
tfidf_feature_names = tfidf_vectorizer.get_feature_names()

# LDA can only use raw term counts for LDA because it is a probabilistic graphical model
tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2, max_features=no_features, stop_words='english')
tf = tf_vectorizer.fit_transform(documents)
tf_feature_names = tf_vectorizer.get_feature_names()

以下是我得到的错误。

AttributeError: 'dict' object has no attribute 'lower'

可在此处获取数据集:kaggle.com/yelp-dataset/yelp-dataset 数据集:yelp_academic_dataset_business.json

任何帮助将不胜感激。谢谢。

1 个答案:

答案 0 :(得分:1)

如@Jarad所述,您必须将listseries馈入tfidf_vectorizer。因此,解决您的问题的方法是

tfidf = tfidf_vectorizer.fit_transform(subset_data[records])