Question

我试图通过Tensorflow编写一个带有自定义丢失功能的Keras 2 LSTM：

model.compile(loss=in_top_k_loss, optimizer='rmsprop', metrics=[bin_crossent_true_only, 'binary_crossentropy', 'mean_squared_error', 'accuracy'])

我的训练集包含时间维度大小不同的示例，因此我使用train_on_batch，其中每个批次仅包含具有相同时间维度的实例。批量大小为256。以下代码在第一个时期（首次调用train_on_batch时）引发了一个非常讨厌的异常：

# takes 2 1D arrays of equal length, returns a single value (the negative of my own "precision" measure)
def in_top_k_loss_single(y_true, y_pred):
    y_true_labels = tf.cast(tf.transpose(tf.where(y_true > 0))[0], tf.int32)
    y_pred = tf.reshape(y_pred, [1, tf.shape(y_pred)[0]])
    y_topk_tensor = tf.nn.top_k(y_pred, k=7)
    y_topk_ixs = y_topk_tensor[0][0][:7]
    y_topk = y_topk_tensor[1][0][:7]
    y_topk_len = tf.cast(tf.count_nonzero(y_topk_ixs), tf.int32)
    y_topk = y_topk[:y_topk_len]
    y_topk0 = tf.expand_dims(y_topk, 1)
    y_true_labels0 = tf.expand_dims(y_true_labels, 0)
    re = tf.cast(tf.reduce_any(tf.equal(y_topk0, y_true_labels0), 1), tf.int32) / tf.range(1,y_topk_len+1)
    return (-1) * tf.where(tf.equal(tf.reduce_sum(y_pred), tf.constant(0.0)), tf.constant(0.0), tf.cast(tf.reduce_mean(re),tf.float32))

# takes 2 matrices of equal sizes, 
# applies the upper function for y_true[i] & y_pred[i] for each row i, 
# returns a single value (mean of all row-wise values)
def in_top_k_loss(y_true, y_pred):
    # if I change `in_top_k_loss_single` to `keras.metrics.binary_crossentropy` (for instance) it runs
    return K.mean(tf.map_fn(lambda x: in_top_k_loss_single(x[0], x[1]), (y_true, y_pred), dtype=tf.float32))

其中in_top_k_loss是我在Keras模型中的自定义丢失函数。当我使用不同的输入（甚至是棘手的）单独测试它们时，这些功能似乎有效。似乎只有Keras有问题 - 也许它需要不同的数据类型/形状等等。

来自互联网的一些聪明的想法：尝试改变批量大小，更改优化器和剪切渐变 - 没有成功。还尝试在evaluate之前调用train_on_batch - 没有成功。

其余的代码与Keras的损失以及像这样的损失一起工作：

def bin_crossent_true_only(y_true, y_pred):
    return (1 + keras.backend.sum(y_pred)) * keras.metrics.binary_crossentropy(y_true, y_true * y_pred)

如果在in_top_k_loss数组中使用，函数metrics可以工作并返回有意义的结果。所有输入（y_true，y_pred）都不是NaN。 y_true可能有0和1（每行零个或多个1，即每个训练集的实例）。

例外本身：

Traceback (most recent call last):
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 491, in apply_op
    preferred_dtype=default_dtype)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 702, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\constant_op.py", line 110, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\constant_op.py", line 99, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 360, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 9, in <module>
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\models.py", line 941, in train_on_batch
    class_weight=class_weight)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 1620, in train_on_batch
    self._make_train_function()
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 1002, in _make_train_function
    self.total_loss)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\optimizers.py", line 210, in get_updates
    new_a = self.rho * a + (1. - self.rho) * K.square(g)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\backend\tensorflow_backend.py", line 1225, in square
    return tf.square(x)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\math_ops.py", line 384, in square
    return gen_math_ops.square(x, name=name)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 2733, in square
    result = _op_def_lib.apply_op("Square", x=x, name=name)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 504, in apply_op
    values, as_ref=input_arg.is_ref).dtype.name
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 702, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\constant_op.py", line 110, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\constant_op.py", line 99, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "C:\Users\myname\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 360, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

Answer 1

TensorFlow中的优化器要求损失函数是可微分的，这是由损失结果与具有已定义梯度的TensorFlow图中的变量之间的所有操作决定的。 tf.where()操作没有定义渐变，这意味着整体损失函数不可微分。尝试计算TensorFlow中不可微函数的梯度的结果是None，这会导致您在Keras尝试更新变量时看到的错误。

Tensorflow函数中的异常用作Keras自定义丢失

1 个答案: