Bayes PCA无法恢复PyMC3中的主要组件

Question

我想用PyMC3实现一个分层贝叶斯模型。在设计复杂模型之前，我试图通过实现Bayes PCA并将结果与​​sklearn.decomposition.pca进行比较来习惯PyMC3

在1中：

%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

# Generate data
nsamp_cl = 1000 #Number of samples per class and per site
cov = np.matrix([[1, 0.9, 0.05],
                 [0.9, 1, 0.05],
                [0.05, 0.05, 1]])

nfeat = cov.shape[0] #Number of features
X0 = np.random.multivariate_normal(np.zeros(nfeat),cov,nsamp_cl)
X1 = np.random.multivariate_normal(np.zeros(nfeat),cov,nsamp_cl)

# Rotate class 1
theta = np.radians(90)
cos, sin = np.cos(theta), np.sin(theta)
R = np.matrix('{} {}; {} {}'.format(cos, -sin, sin, cos))
X1[:,0:2] = np.dot(X1[:,0:2],R.T)

X = np.concatenate([X0,X1])
Y = np.concatenate([np.zeros(X0.shape[0]),np.ones(X1.shape[0])])
n = X.shape[0]
d = X.shape[1]

在2中：

plt.figure()
cols = ['b','r']            
colors = [cols[y.astype(int)] for y in Y ]
plt.scatter(X[:,0],X[:,1],20,colors)
plt.title('Features 0 and 1')
plt.figure()
cols = ['b','r']            
colors = [cols[y.astype(int)] for y in Y ]
plt.scatter(X[:,1],X[:,2],20,colors)
plt.title('Features 1 and 2')

出2：

在3中：

from pymc3 import Model,Normal,Gamma,math,variational

common_latent_model = Model()
# Builiding a latent model to extract site-robust principal components
with common_latent_model:

    n_latent = 3

    #ARD prior
    alphas = Gamma('alphas', alpha=1e-6, beta=1e-6, shape=n_latent)

    # Weight vector
    w = Normal('w',mu=0,tau=alphas,shape=(d,n_latent))

    # Latent space
    z = Normal('z',mu=0,tau=1,shape=(n,n_latent))

    # Multiply latent variables by W to go from latent to observation space
    t = math.dot(z,w.T)

    # Add bias
    mu = Normal('mu', mu=0, tau=0.01, shape=d)
    u = t + mu

    # Precision of the observation
    sigma = Gamma('sigma',alpha=1e-6, beta=1e-6,shape=1)

    # Likelihood (sampling distribution) of observations
    X_obs = Normal('X_obs', mu=u, tau=sigma, observed=X)

with common_latent_model:
    means, sds, elbos = variational.advi(n=10000,learning_rate=0.1, accurate_elbo=True)#100000)

plt.plot(elbos)
plt.ylabel('ELBO')
plt.xlabel('iteration')    

在[4]中：

for key in means:
   print "key: %s , value: %s" % (key, means[key])
key: mu , value: [ 0.03288066 -0.05347487  0.00260641]
key: alphas_log_ , value: [ 6.94631195  6.85621834  6.84792233]
key: sigma_log_ , value: [-0.009662]
key: z , value: [[-0.01260083 -0.00460729 -0.01360558]
 [-0.02817471  0.04281501  0.01643355]
 [-0.05178572 -0.02470609 -0.05092171]
 ..., 
 [-0.05201711  0.00150599 -0.01167801]
 [-0.01097088 -0.02666511  0.03660954]
 [ 0.0609949   0.01156182  0.01814843]]
key: w , value: [[-0.06004834  0.00599346 -0.03071374]
 [ 0.00668656 -0.01306511  0.00400904]
 [-0.00141243 -0.00778869  0.03257137]]

在[5]中：

PC_bayes = means['z']
plt.figure()
cols = ['b','r']            
colors = [cols[y.astype(int)] for y in Y ]
plt.scatter(PC_bayes[:,0],PC_bayes[:,1],20,colors,alpha=.1)

Out [5]：

在[6]中：     来自sklearn.decomposition导入PCA     pca = PCA（n_components = 2）     pca.fit（X）     PC = pca.transform（X）

在[7]中：

plt.figure()
cols = ['b','r']            
colors = [cols[y.astype(int)] for y in Y ]
plt.scatter(PC[:,0],PC[:,1],20,colors,alpha=.1)

Out [7]：

（你可以在这里找到iPython Notebook： https://github.com/peppeFarAway/pymc3/blob/master/BayesPCA.ipynb）

为什么我的Bayes PCA实现无法恢复主体组件，而sklearn.decomposition.pca可以？我在哪里弄错了？

我用来实现该模型的主要参考是： https://blogs.msdn.microsoft.com/infernet_team_blog/2011/09/30/bayesian-pca/