Question

短版本： 我有一个带有图层的自定义模型，可以通过值sigma进行参数化。

我在Tensorflow Estimator中使用此模型：

    classifier = tf.estimator.Estimator(model_fn=model_fun)

例如，培训声明如下：

    # Train the model
    train_input_fn=tf.estimator.inputs.numpy_input_fn(x={"x": train_data}, 
    batch_size=batch_size, num_epochs=nEpochs, shuffle=True)
    classifier.train(input_fn=train_input_fn,max_steps=nSteps)

在此设置中，如何将sigma传递给我的自定义Estimator，以便可以使用不同的值对其进行训练/评估？

版本更长：

我正在研究这个DNN自动编码器，我有一个层，通过考虑已知标准偏差西格玛分布的random_normal（）值来增加高斯噪声。

这是通信系统的模型，使用model.predict（）函数检索输出logits（来自最后一层），我的度量即Bit Error rate由Tensor Flow 1.5中的自定义函数计算，Python 3.5，Windows 10.

问题如下：

我想训练系统sigma = sigma1，并检索输出logits。（这部分很好，我能够得到所需的输出。）
我还想使用定义的相同Estimator（在相同的程序中）预测sigma = sigma2，sigma = sigma3，sigma = sigma4等的输出。

我的DNN看起来像这样，并在模型函数中定义：

输入图层 - 此处提供一个热编码值。
密+ RELU
密+线性
规范化层
添加噪音：这里我将tf.random_normal（stddev = sigma）添加到前一层的输出中。在这里，我将非常感谢您帮助理解如何为每次运行（火车/测试）使用不同的sigma。我想你可以说sigma应该是一个参数，每次测试都可以有不同的值。
```
 gnoise=tf.random_normal(mean=0,stddev=sigma) 
 Then the layer's output=norm(which is the prev layer's output)+gnoise
```
密+ RELU
Softmax-Output是Logits

我将估算器定义为：

    classifier = tf.estimator.Estimator(model_fn=model_fun)

培训声明如下：

    # Train the model
    train_input_fn=tf.estimator.inputs.numpy_input_fn(x={"x": train_data}, 
    batch_size=batch_size, num_epochs=nEpochs, shuffle=True)
    classifier.train(input_fn=train_input_fn,max_steps=nSteps)

声明预测函数并将其命名为：

    pred_input_fn=tf.estimator.inputs.numpy_input_fn(x={"x": test_data}, 
    batch_size=batch_size, num_epochs=nEpochs, shuffle=False)
    pred_results = classifier.predict(input_fn=pred_input_fn)

Answer 1

您应该将西格玛作为图层的参数，然后在运行时通过features将其值提供给模型（使用列键区分x和sigma）。

如果没有高斯图层代码，很难准确回复，但假设您的模型是以这样的方式定义的：

import tensorflow as tf

def gaussian_noise_layer(x):
    # Currently, your sigma is probably fixed somewhere here, e.g.
    sigma = 1

    dist = tf.distributions.Normal(loc=0., scale=sigma)

    # Build Gaussian kernel from dist:
    # gaussian_kernel = ...

    return tf.nn.depthwise_conv2d(x, gaussian_kernel, [1, 1, 1, 1], padding='SAME')

def model_fun(features, labels, mode, params):

    # Get input x from features:
    x = tf.feature_column.input_layer(features, params.feature_column)

    # ... building your model here, adding at some point the gaussian layer, e.g.
    # ... net = f(x)
    net = gaussian_noise_layer(net)
    # ... predictions = f'(net)
    # ...

    return tf.estimator.EstimatorSpec(
        mode=mode,
        predictions=predictions,
        loss=loss,
        train_op=train_op,
        eval_metric_ops=eval_metric_ops
    )

with tf.Session() as sess:
    # ...

    # Specifying your params and inputs:
    params = tf.contrib.training.HParams(
        # ... other hyperparameters,
        # Define feature column for input x of shape "shape_x" (e.g. (64, 64, 3)):
        feature_column=tf.feature_column.numeric_column(key="x", shape=shape_x)
    )

    classifier = tf.estimator.Estimator(model_fn=model_fun, params=params)

    # For training:
    train_input_fn = tf.estimator.inputs.numpy_input_fn(x={"x": train_data},
                                                        batch_size=batch_size, num_epochs=nEpochs, shuffle=True)
    classifier.train(input_fn=train_input_fn, max_steps=nSteps)

...然后你需要像这样编辑它：

import tensorflow as tf

def gaussian_noise_layer(x, sigma):

    # sigma is now a parameter
    dist = tf.distributions.Normal(loc=0., scale=sigma)

    # Build Gaussian kernel from dist:
    # gaussian_kernel = ...

    return tf.nn.depthwise_conv2d(x, gaussian_kernel, [1, 1, 1, 1], padding='SAME')

def model_fun(features, labels, mode, params):

    # Get input x from features:
    x = tf.feature_column.input_layer(features, params.input_feature_column)
    # Get sigma from features:
    sigma = tf.feature_column.input_layer(features, params.sigma_feature_column)

    # ... building your model here, adding at some point the gaussian layer, e.g.
    # ... net = f(x)
    net = gaussian_noise_layer(net, sigma)
    # ... predictions = f'(net)
    # ...

    return tf.estimator.EstimatorSpec(
        mode=mode,
        predictions=predictions,
        loss=loss,
        train_op=train_op,
        eval_metric_ops=eval_metric_ops
    )

with tf.Session() as sess:
    # ...

    # We now specify which columns contain the actual inputs (x), and which columns contain other parameters (sigma):
    params = tf.contrib.training.HParams(
        # ... other hyperparameters,
        # Define feature column for input x of shape "shape_x" (e.g. (64, 64, 3)):
        input_feature_column=tf.feature_column.numeric_column(key="x", shape=shape_x),
        # Define feature column for input sigma of shape () i.e. scalar (default shape):
        sigma_feature_column=tf.feature_column.numeric_column(key="sigma")
    )

    classifier = tf.estimator.Estimator(model_fn=model_fun, params=params)

    # Train:
    num_train_elements = train_data.shape[0]
    sigma = [1] * num_train_elements  # or e.g. sigma = [1, 1, 2, 1, 3, ...]
    # We can now feed sigma along x:
    train_input_fn = tf.estimator.inputs.numpy_input_fn(
         x={"x": train_data, "sigma": numpy.array(sigma)},
         batch_size=batch_size, num_epochs=nEpochs, shuffle=True)

    classifier.train(input_fn=train_input_fn, max_steps=nSteps)

    # ...

    # Predict:
    sigma = [2] * num_train_elements  # or e.g. sigma = [1, 1, 2, 1, 3, ...]
    pred_input_fn=tf.estimator.inputs.numpy_input_fn(
         x={"x": train_data, "sigma": numpy.array(sigma)},
         batch_size=batch_size, num_epochs=nEpochs, shuffle=True)
    pred_results = classifier.predict(input_fn=pred_input_fn)

如何将输入的其他模型参数输入到Tensorflow Estimator？

1 个答案: