我有以下data:
State Murder Assault UrbanPop Rape
Alabama 13.200 236 58 21.200
Alaska 10.000 263 48 44.500
Arizona 8.100 294 80 31.000
Arkansas 8.800 190 50 19.500
California 9.000 276 91 40.600
Colorado 7.900 204 78 38.700
Connecticut 3.300 110 77 11.100
Delaware 5.900 238 72 15.800
Florida 15.400 335 80 31.900
Georgia 17.400 211 60 25.800
Hawaii 5.300 46 83 20.200
Idaho 2.600 120 54 14.200
Illinois 10.400 249 83 24.000
Indiana 7.200 113 65 21.000
Iowa 2.200 56 57 11.300
Kansas 6.000 115 66 18.000
Kentucky 9.700 109 52 16.300
Louisiana 15.400 249 66 22.200
Maine 2.100 83 51 7.800
Maryland 11.300 300 67 27.800
Massachusetts 4.400 149 85 16.300
Michigan 12.100 255 74 35.100
Minnesota 2.700 72 66 14.900
Mississippi 16.100 259 44 17.100
Missouri 9.000 178 70 28.200
Montana 6.000 109 53 16.400
Nebraska 4.300 102 62 16.500
Nevada 12.200 252 81 46.000
New Hampshire 2.100 57 56 9.500
New Jersey 7.400 159 89 18.800
New Mexico 11.400 285 70 32.100
New York 11.100 254 86 26.100
North Carolina 13.000 337 45 16.100
North Dakota 0.800 45 44 7.300
Ohio 7.300 120 75 21.400
Oklahoma 6.600 151 68 20.000
Oregon 4.900 159 67 29.300
Pennsylvania 6.300 106 72 14.900
Rhode Island 3.400 174 87 8.300
South Carolina 14.400 279 48 22.500
South Dakota 3.800 86 45 12.800
Tennessee 13.200 188 59 26.900
Texas 12.700 201 80 25.500
Utah 3.200 120 80 22.900
Vermont 2.200 48 32 11.200
Virginia 8.500 156 63 20.700
Washington 4.000 145 73 26.200
West Virginia 5.700 81 39 9.300
Wisconsin 2.600 53 66 10.800
Wyoming 6.800 161 60 15.600
我用它来执行基于状态的层次聚类。 这是完整的工作代码:
import pandas as pd
from sklearn.cluster import AgglomerativeClustering
df = pd.io.parsers.read_table("http://dpaste.com/031VZPM.txt")
samples = df["State"].tolist()
ndf = df[["Murder", "Assault", "UrbanPop","Rape"]]
X = ndf.as_matrix()
cluster = AgglomerativeClustering(n_clusters=3,
linkage='complete',affinity='euclidean').fit(X)
label = cluster.labels_
outclust = list(zip(label, samples))
outclust_df = pd.DataFrame(outclust,columns=["Clusters","Samples"])
for clust in outclust_df.groupby("Clusters"):
print (clust)
请注意,在该方法中,我使用euclidean
距离。我想要做的是使用1-Pearson correlation distance
。在R中它看起来像这样:
dat <- read.table("http://dpaste.com/031VZPM.txt",sep="\t",header=TRUE)
dist2 = function(x) as.dist(1-cor(t(x), method="pearson"))
dat = dat[c("Murder","Assault","UrbanPop","Rape")]
hclust(dist2(dat), method="ward.D")
如何使用Scikit-learn AgglomerativeClustering实现这一目标? 我知道有预先计算好的&#39;亲和力的论据。但不知道如何使用它来解决我的问题。
答案 0 :(得分:7)
您可以将自定义关联矩阵定义为接收数据并返回关联矩阵的函数:
from scipy.stats import pearsonr
import numpy as np
def pearson_affinity(M):
return 1 - np.array([[pearsonr(a,b)[0] for a in M] for b in M])
然后你可以用这个作为亲和力函数调用凝聚聚类(你必须改变联系,因为&#39; ward&#39;只适用于欧几里德距离。
cluster = AgglomerativeClustering(n_clusters=3, linkage='average',
affinity=pearson_affinity)
cluster.fit(X)
请注意,出于某种原因,它似乎不能很好地处理您的数据:
cluster.labels_
Out[107]:
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1,
0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 1, 0])