张量流的基于模型的简单密度估计

Question

在给定高斯分布数据的情况下，我试图估计高斯分布的参数。问题是我无法优化日志丢失目标。这是一个带有玩具数据的代码示例：

import tensorflow as tf
import numpy as np
import math as m
from sklearn.datasets import make_classification as gen_data

def custom_optimizer(mu, sigma, cost, lr):
  grad_mu, grad_sigma = tf.gradients(xs=[mu, sigma], ys=cost)
  new_mu = mu.assign(mu - lr * grad_mu)
  new_sigma = sigma.assign(sigma - lr * grad_sigma)
  return [new_mu, new_sigma, cost]


# Setting hyperparameters
n_observations = 50
lr = 0.00005
n_iter = 800

# Generate input data 
xs, _ = gen_data(n_samples=n_observations, n_features=2, n_redundant=0, 
                  n_informative=2, random_state=0, n_clusters_per_class=1)

X = tf.placeholder(tf.float32, [None, 2])

sigma = tf.Variable(tf.zeros([1]))
mu = tf.Variable(tf.zeros([1, 2]))

p_x = (1 / tf.sqrt(2 * tf.constant(m.pi) * sigma**2)) * \
         tf.exp(-tf.square(tf.norm(X - mu))/(2 * sigma ** 2))

loss = -tf.reduce_mean(tf.log(p_x))
train_step = custom_optimizer(mu, sigma, loss, lr=lr)

sess = tf.Session()
sess.run(tf.global_variables_initializer())

for step in range(n_iter):
    _, Loss = sess.run([train_step, loss], feed_dict={X: xs})                 
    print ("iter: {0}; mean: {1}; sigma: {2}, loss: {3}".format(step, 
                                                      sess.run(mu), 
                                                      sess.run(sigma),
                                                      Loss))

输出为：

iter: 0; mean: [[nan nan]]; sigma: [nan], loss: nan
iter: 1; mean: [[nan nan]]; sigma: [nan], loss: nan
iter: 2; mean: [[nan nan]]; sigma: [nan], loss: nan
iter: 3; mean: [[nan nan]]; sigma: [nan], loss: nan
iter: 4; mean: [[nan nan]]; sigma: [nan], loss: nan
iter: 5; mean: [[nan nan]]; sigma: [nan], loss: nan
iter: 6; mean: [[nan nan]]; sigma: [nan], loss: nan

因此模型不会收敛。