Question

我对聚类结果的评估有问题。

我有3个列表：

# 10 objects in my corpus
TOT = [1,2,3,4,5,6,7,8,9,10]

# .... clustering into k=5 clusters

# For each automatic cluster:

    # Objects with ID 2 and 8 are stored into this
    predicted = [2,8]

    # For each cluster in the ground truth:  
        true = [2,4,9]

        # computes TP, FP, TN, FN
        A = set(docs_in_cluster)
        B = set(constraints)

        TP = list(A & B)
        FP = list(A - (A & B))
        TN = list((TOT - A) & (TOT - B))
        FN = list(B - A)

我的问题是：是否可以为每个群集计算TP，FP，TN，FN？有道理吗？

编辑：可复制的代码

短篇小说：

我正在做NLP，我已经用Gensim的Word2Vec处理了9k文档的语料库，提取了向量，并为每个文档计算了一个“文档向量”。之后，我计算出文档向量之间的余弦相似度，得到一个9k x 9k的矩阵。

最后，使用此矩阵，我运行了KMeans和层次聚类。

让我们考虑一下具有14个类的HAC的输出：

id    label
 0        1
 1        8
     ....
9k       12

现在的问题是：如何评估群集的质量？我的教授已阅读了9个文档中的100个，并创建了一些“集群”的说法：“好的文档讨论了label1”或“好的其他讨论了label2和label3

请注意，我的教授提供的标签与聚类过程完全无关，只是该主题的摘要，但是数量相同（在此示例中为14）。

代码

我有两个数据框，上面一个来自HAC群集，另一个来自我的教授的100个文档，看起来像：（以前面的示例为例）

GT

id    label1    label2    label3    ....    label14
 5         1         0         0                 0
34         0         1         1                 0
      ...........................

最后，我的代码执行此操作：

 # since I have labels only for 100 of my 9k documents
 indexes = list(map(int, ground_truth['id'].values.tolist()))
 reduced_df = clusters.loc[clusters['id'].isin(indexes), :]

 # now reduced_df contains only the documents that have been read by my prof
 TOT = set(reduced_df['id'].values.tolist())

 for each cluster from HAC
    doc_in_this_cluster = [ .... ]

    for each cluster from GT
       doc_in_this_label = [ ... ]

        A = set(doc_in_this_cluster )
        B = set(doc_in_this_label )

        TP = list(A & B)
        FP = list(A - (A & B))
        TN = list((TOT - A) & (TOT - B))
        FN = list(B - A)

和代码：

indexes = list(map(int, self.ground_truth['id'].values.tolist()))
    # reduce clusters_file matching only manually analyzed documents:  -------->   TOT
    reduced_df = self.clusters.loc[self.clusters['id'].isin(indexes), :]

    TOT = set(reduced_df['id'].values.tolist())

    clusters_groups = reduced_df.groupby('label')

    for label, df_group in clusters_groups:
        docs_in_cluster = df_group['id'].values.tolist()

        row = []
        for col in self.ground_truth.columns[1:]:
            constraints = list(
                map(int, self.ground_truth.loc[self.ground_truth[col] == 1, 'id'].values.tolist())
            )

            A = set(docs_in_cluster)
            B = set(constraints)

            TP = list(A & B)
            FP = list(A - (A & B))
            TN = list((TOT - A) & (TOT - B))
            FN = list(B - A)

            print(f"HAC Cluster: {label} - GT Label: {col}")
            print(TP, FP, TN, FN)

Answer 1

我假设您正在尝试实现设置操作。您可以尝试以下功能来解决您的情况：

def setSubtract(A,B):
    C=[]
    for i in A:
        if i in B:
            pass
        else:
            C.append(i)
    return C

def setIntersection(A,B):
    C=[]
    for i in A:
        if i in B:
            C.append(i)
    return C

TOT = [1,2,3,4,5,6,7,8,9,10]
A=[1,2,3,4]
B=[2,3]
print("A&B",setIntersection(A,B))
print("TOT-B",setSubtract(TOT,B))

输出：

A&B [2, 3]
TOT-B [1, 4, 5, 6, 7, 8, 9, 10]

Python混淆矩阵

1 个答案: