Question

有人可以解释张量流eager-mode的工作原理。我试图建立一个简单的回归，如下所示：

编辑：我正在更新我的问题，这是我的完整代码，现在问题来自梯度计算，返回零。我检查了非零的损失值。

import tensorflow as tf
tfe = tf.contrib.eager
tf.enable_eager_execution()
import numpy as np
def make_model():
    net = tf.keras.Sequential()
    net.add(tf.keras.layers.Dense(4, activation='relu'))
    net.add(tf.keras.layers.Dense(1))
    return net

def compute_loss(pred, actual):
    return tf.reduce_mean(tf.square(tf.subtract(pred, actual)))

def compute_gradient(model, pred, actual):
    """compute gradients with given noise and input"""
    with tf.GradientTape() as tape:
        loss = compute_loss(pred, actual)
    grads = tape.gradient(loss, model.variables)
    return grads, loss

def apply_gradients(optimizer, grads, model_vars):
    optimizer.apply_gradients(zip(grads, model_vars))

model = make_model()
optimizer = tf.train.AdamOptimizer(1e-4)

x = np.linspace(0,1,1000)
y = x+np.random.normal(0,0.3,1000)
y = y.astype('float32')
train_dataset = tf.data.Dataset.from_tensor_slices((y.reshape(-1,1)))

epochs = 2# 10
batch_size = 25
itr = y.shape[0] // batch_size
for epoch in range(epochs):
    for data in tf.contrib.eager.Iterator(train_dataset.batch(25)):
        preds = model(data)
        grads, loss = compute_gradient(model, preds, data)
        print(grads)
        apply_gradients(optimizer, grads, model.variables)
#         with tf.GradientTape() as tape:
#             loss = tf.sqrt(tf.reduce_mean(tf.square(tf.subtract(preds, data))))
#         grads = tape.gradient(loss, model.variables)
#         print(grads)
#         optimizer.apply_gradients(zip(grads, model.variables),global_step=None)

Gradient output: [None, None, None, None, None, None] 错误如下：

----------------------------------------------------------------------
ValueError                           Traceback (most recent call last)
<ipython-input-3-a589b9123c80> in <module>
     35         grads, loss = compute_gradient(model, preds, data)
     36         print(grads)
---> 37         apply_gradients(optimizer, grads, model.variables)
     38 #         with tf.GradientTape() as tape:
     39 #             loss = tf.sqrt(tf.reduce_mean(tf.square(tf.subtract(preds, data))))

<ipython-input-3-a589b9123c80> in apply_gradients(optimizer, grads, model_vars)
     17 
     18 def apply_gradients(optimizer, grads, model_vars):
---> 19     optimizer.apply_gradients(zip(grads, model_vars))
     20 
     21 model = make_model()

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py in apply_gradients(self, grads_and_vars, global_step, name)
    589     if not var_list:
    590       raise ValueError("No gradients provided for any variable: %s." %
--> 591                        ([str(v) for _, v, _ in converted_grads_and_vars],))
    592     with ops.init_scope():
    593       self._create_slots(var_list)

ValueError: No gradients provided for any variable:

Answer 1

第1部分：问题确实是您输入的数据类型。默认情况下，您的keras模型期望float32，但是您传递的是float64。您可以更改模型的dtype或将输入更改为float32。

要更改模型，请执行以下操作：

要更改输入，请执行以下操作： def make_model(): net = tf.keras.Sequential() net.add(tf.keras.layers.Dense(4, activation='relu', dtype='float32')) net.add(tf.keras.layers.Dense(4, activation='relu')) net.add(tf.keras.layers.Dense(1)) return net

第2部分：，您需要在tf.GradientTape（）下调用用于计算模型的函数（即y = y.astype('float32')）。例如，您可以将model(data)方法替换为以下内容：

compute_loss

InvalidArgumentError：无法计算MatMul，因为输入＃0（从零开始）应该是浮点张量，但是是双张量[Op：MatMul]

1 个答案: