Question

我正在尝试为使用Keras的网络引入新的内核正则化。但是，这给了我错误： 'float'对象没有属性'dtype' 我该如何解决？

我在这里找到了代码：KL divergence in keras(tensorflow backend)

这是我的代码：

    from keras import backend as K

    kullback_leibler_divergence = keras.losses.kullback_leibler_divergence

    def kl_divergence_regularizer(inputs):
        means = K.mean((inputs))
        return 0.5 *(0.01 * (kullback_leibler_divergence(0.05, means)
                      + kullback_leibler_divergence(1 - 0.05, 1 - means)))


    model = Sequential([
        Dense(100, input_shape=(x_train_s.shape[1],),kernel_initializer='random_uniform'),
        Activation('elu'),
        Dense(x_train_s.shape[1],kernel_initializer='random_uniform', kernel_regularizer=kl_divergence_regularizer),
        Activation('tanh')
    ])

    model.compile(optimizer='adam',loss='mean_squared_error')

    model.fit(x_train_s, x_train_s, epochs=5,validation_split=0.1, shuffle=True, verbose=1,batch_size=np.uint(x_train_s.shape[0]/100))

完整的错误回溯显示在这里：

AttributeError                            Traceback (most recent call last)
<ipython-input-39-59bc90c687de> in <module>
     39     Activation('elu'),
     40     Dense(x_train_s.shape[1],kernel_initializer='random_uniform', kernel_regularizer=kl_divergence_regularizer),
---> 41     Activation('tanh')
     42 ])
     43 

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\sequential.py in __init__(self, layers, name)
     91         if layers:
     92             for layer in layers:
---> 93                 self.add(layer)
     94 
     95     @property

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\sequential.py in add(self, layer)
    179                 self.inputs = network.get_source_inputs(self.outputs[0])
    180         elif self.outputs:
--> 181             output_tensor = layer(self.outputs[0])
    182             if isinstance(output_tensor, list):
    183                 raise TypeError('All layers in a Sequential model '

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in __call__(self, inputs, **kwargs)
    429                                          'You can build it manually via: '
    430                                          '`layer.build(batch_input_shape)`')
--> 431                 self.build(unpack_singleton(input_shapes))
    432                 self.built = True
    433 

C:\ProgramData\Anaconda3\lib\site-packages\keras\layers\core.py in build(self, input_shape)
    864                                       name='kernel',
    865                                       regularizer=self.kernel_regularizer,
--> 866                                       constraint=self.kernel_constraint)
    867         if self.use_bias:
    868             self.bias = self.add_weight(shape=(self.units,),

C:\ProgramData\Anaconda3\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint)
    253         if regularizer is not None:
    254             with K.name_scope('weight_regularizer'):
--> 255                 self.add_loss(regularizer(weight))
    256         if trainable:
    257             self._trainable_weights.append(weight)

<ipython-input-39-59bc90c687de> in kl_divergence_regularizer(inputs)
     14     means = K.mean((inputs))
     15 #     means=1e-6
---> 16     return 0.5 *(0.01 * (kullback_leibler_divergence(0.05, means)
     17                   + kullback_leibler_divergence(1 - 0.05, 1 - means)))
     18 #     return 1e-10

C:\ProgramData\Anaconda3\lib\site-packages\keras\losses.py in kullback_leibler_divergence(y_true, y_pred)
     79 
     80 def kullback_leibler_divergence(y_true, y_pred):
---> 81     y_true = K.clip(y_true, K.epsilon(), 1)
     82     y_pred = K.clip(y_pred, K.epsilon(), 1)
     83     return K.sum(y_true * K.log(y_true / y_pred), axis=-1)

C:\ProgramData\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in clip(x, min_value, max_value)
   1599     if max_value is None:
   1600         max_value = np.inf
-> 1601     min_value = _to_tensor(min_value, x.dtype.base_dtype)
   1602     max_value = _to_tensor(max_value, x.dtype.base_dtype)
   1603     return tf.clip_by_value(x, min_value, max_value)

AttributeError: 'float' object has no attribute 'dtype'

我需要KL散度在0.05和均值之间，对i计算以下总和：

KL = sum（0.05 * \ log（0.05 / mean [i]））

Answer 1

kullback_leibler_divergence(0.05, means)

这是一个loss函数。期望y_true, y_pred都为张量。您正在传递float（0.05），并且系统正在尝试获取该假定张量的属性，但这不是张量。

在第二个kullback_leibler_divergence调用中，您再次传递浮点数1 - 0.05而不是张量时，也会发生相同的问题。

一个简单的解决方案（但您必须检查这在数学上是否合理-我不知道KL_diff应该做什么）是使用K.ones_like(means) * 0,05并在第二个调用K.ones_like(means)*(1-0.05)中使用。

Answer 2

您应该将KL散度正则化器更改为：

def kl_divergence_regularizer(inputs):
        means = K.mean((inputs))
        down = 0.05 * K.ones_like(means)
        up = (1 - 0.05) * K.ones_like(means)
        return 0.5 *(0.01 * (kullback_leibler_divergence(down, means)
                      + kullback_leibler_divergence(up, 1 - means)))

这是因为KL散度采用两个向量/张量，因此无法计算标量和向量之间的KL（未定义）。

Keras后端均值函数：“'float'对象没有属性'dtype'”？

2 个答案: