Question

我想像scikit-learn示例silhouette_analysis一样计算silhouette_score。

from sklearn.feature_extraction.text import TfidfVectorizer

tfidf_vectorizer = TfidfVectorizer(use_idf=True)
sampleText = []
sampleText.append("Some text for document clustering")
tfidf_matrix = tfidf_vectorizer.fit_transform(sampleText)

如何将我的tfidf_matrix转换为这样的事情：

import matplotlib.cm as cm
from sklearn.metrics import silhouette_samples, silhouette_score
import matplotlib.pyplot as plt


for num_clusters in range(2,6):
    # Create a subplot with 1 row and 2 columns
    fig, (ax1, ax2) = plt.subplots(1, 2)
    fig.set_size_inches(18, 7)

    # The 1st subplot is the silhouette plot
    # The silhouette coefficient can range from -1, 1 but in this example all
    # lie within [-0.1, 1]
    ax1.set_xlim([-0.1, 1])
    # The (n_clusters+1)*10 is for inserting blank space between silhouette
    # plots of individual clusters, to demarcate them clearly.
    ax1.set_ylim([0, len(tfidf_matrix) + (num_clusters + 1) * 10])

    km = KMeans(n_clusters=num_clusters,
                n_init=10,                        # number of iterations with different seeds
                random_state=1                    # fixes the seed 
               )

    cluster_labels = km.fit_predict(tfidf_matrix)

    # The silhouette_score gives the average value for all the samples.
    # This gives a perspective into the density and separation of the formed
    # clusters
    silhouette_avg = silhouette_score(tfidf_matrix, cluster_labels)

Answer 1

tf-idf是多维的，必须减少到两个维度。这可以通过将tf-idf减少到具有最高方差的两个特征来完成。我使用PCA来减少tf-idf。完整的例子：

git push --tags

sklearn聚类：计算TF-IDF-weigthed数据的轮廓系数

1 个答案: