Question

我有一个简单的python代码来计算集合的熵，我试图在Theano中编写相同的东西。

import math

# this computes the probabilities of each element in the set
def prob(values):
    return [float(values.count(v))/len(values) for v in values]

# this computes the entropy
def entropy(values):
    p = prob(values)
    return -sum([v*math.log(v) for v in p])

我正在尝试在Theno中编写等效代码，但我不知道该怎么做：

import theano
import theano.tensor as T

v = T.vector('v') # I create a symbolic vector to represent my initial values
p = T.vector('p') # The same for the probabilities 

# this is my attempt to compute the probabilities which would feed vector p
theano.scan(fn=prob,outputs_info=p,non_sequences=v,n_steps=len(values))

# considering the previous step would work, the entropy is just
e = -T.sum(p*T.log(p))
entropy = theano.function([values],e)

然而，扫描线不正确，我遇到了大量错误。我不确定是否有一种简单的方法（计算向量的熵），或者我是否需要在扫描函数上投入更多精力。有什么想法吗？

Answer 1

除了nouiz提出的观点之外，P不应该被声明为T.vector，因为它将是你的值向量计算的结果。

此外，要计算类似熵的东西，您不需要使用扫描（扫描会引入计算开销，因此只能使用它，因为没有其他方法可以计算您想要的内容或减少内存使用量） ;你可以采取这样的方法：

values = T.vector('values')
nb_values = values.shape[0]

# For every element in 'values', obtain the total number of times
# its value occurs in 'values'.
# NOTE : I've done the broadcasting a bit more explicitly than
# needed, for clarity.
freqs = T.eq(values[:,None], values[None, :]).sum(0).astype("float32")

# Compute a vector containing, for every value in 'values', the
# probability of that value in the vector 'values'.
# NOTE : these probabilities do *not* sum to 1 because they do not
# correspond to the probability of every element in the vector 'values
# but to the probability of every value in 'values'. For instance, if
# 'values' is [1, 1, 0] then 'probs' will be [2/3, 2/3, 1/3] because the
# value 1 has probability 2/3 and the value 0 has probability 1/3 in
# values'.
probs = freqs / nb_values

entropy = -T.sum(T.log2(probs) / nb_values)
fct = theano.function([values], entropy)

# Will output 0.918296...
print fct([0, 1, 1])

Theano中的熵和概率

1 个答案: