Question

我正在研究一种半监督学习方法，我想对损失的群集分配的高方差添加一个惩罚。 Keras中有一个方差函数（我使用tensorflow后端）。我的想法是利用tensorflow的boolean_mask（）函数来一次仅索引一个簇的子矩阵。但是，这将需要一个循环来迭代集群，从而导致问题，即NaN丢失。

有什么方法可以向量化此操作吗？

（这显然是我的代码的非常简化的版本，在这种情况下，损失术语没有多大意义。）

import tensorflow as tf
import keras
from keras.models import Model
from keras.layers import Dropout, Dense, Input
from keras.optimizers import SGD
from keras.activations import relu
from keras.initializers import RandomNormal
import keras.backend as K
import numpy as np

from sklearn.datasets import make_blobs

n_clusters = 100
n_features = 2

# create toy data with 100 clusters
X, y = make_blobs(n_samples=10000, cluster_std = 0.5, centers=n_clusters, random_state=1)

inputs = Input(shape=(n_features,))

outputs = Dropout(0.2)(inputs)
outputs = Dense(500, activation='relu', kernel_initializer=RandomNormal(0.0,0.01))(outputs)
outputs = Dropout(0.2)(outputs)
outputs = Dense(n_clusters, kernel_initializer=RandomNormal(0.0,0.01))(outputs)

# custom loss
def loss_var(y_true, q):
    # input: labels y_true, soft clustering assignments q
    var=0
    for c in range(n_clusters):
        # get indices of cluster c
        mask = K.equal(y_true, c)
        # boolean mask for indexing q, get only cluster c elements
        q_c = tf.boolean_mask(q, mask, axis=0)
        # variance of this cluster
        var += K.mean(K.var(q_c, axis=0))
    var = var / n_clusters
    return var

model = Model(inputs, outputs)
model.compile(optimizer=SGD(lr=0.1, momentum=0.9), loss=loss_var)
model.fit(X,y,batch_size=256, epochs=10)

自定义Keras损失惩罚了群集变量而没有循环？

0 个答案: