Question

我正在慢慢地将一个玩具神经网络与Tensorflow结合在一起。我一直在改编一个适合自己需要的教程，但对结果感到困惑。

简而言之，我正在输入任意生成的输入以训练网络收敛于更多任意生成的输出值。

这是我的代码：

# Imports

import tensorflow as tf
import numpy as np
import random as rand

# Data parameters

num_frames = 1000

stimuli_value_low = .001
stimuli_value_high = 1.0

pixel_value_low = 1.0
pixel_value_high = 256.0

stimuli_dimension = 1
frame_dimension = 100

stimuli = np.random.uniform(stimuli_value_low, stimuli_value_high, (num_frames, stimuli_dimension))
frames = np.random.uniform(pixel_value_low, pixel_value_high, (num_frames, frame_dimension))

# Parameters
learning_rate = 0.01
training_iterations = 1000
display_iteration = 10

# Network Parameters
n_hidden_1 = 100
n_hidden_2 = 100
num_input_neurons = stimuli_dimension
num_output_neurons = frame_dimension

# Create placeholders
input_placeholder = tf.placeholder("float", [None, num_input_neurons])
output_placeholder = tf.placeholder("float", [None, num_output_neurons])

# Store layers weight & bias
weights = {
    'h1': tf.Variable(tf.random_normal([num_input_neurons, n_hidden_1])),
    'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),
    'out': tf.Variable(tf.random_normal([n_hidden_2, num_output_neurons]))
}
biases = {
    'b1': tf.Variable(tf.random_normal([n_hidden_1])),
    'b2': tf.Variable(tf.random_normal([n_hidden_2])),
    'out': tf.Variable(tf.random_normal([num_output_neurons]))
}

# Create model
def neural_net(input_placeholder):
    # Hidden fully connected layer 
    layer_1 = tf.add(tf.matmul(input_placeholder, weights['h1']), biases['b1'])
    # Hidden fully connected layer
    layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
    # Output fully connected layer with a neuron for each pixel
    out_layer = tf.matmul(layer_2, weights['out']) + biases['out']
    return out_layer

# Construct model
logits = neural_net(input_placeholder)

# Define loss operation and optimizer
loss_operation = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = logits, labels = output_placeholder))
optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate)
train_operation = optimizer.minimize(loss_operation)

# Evaluate mode
correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(output_placeholder, 1))
accuracy_operation = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

# Initialize the variables (i.e. assign their default value)
init = tf.global_variables_initializer()

# Start Training
with tf.Session() as sess:

  # Run the initializer
  sess.run(init)

  for iteration in range(1, training_iterations + 1): 

    sess.run(train_operation, feed_dict = {input_placeholder: stimuli, output_placeholder: frames})

    if iteration % display_iteration == 0 or iteration == 1:

      loss = sess.run(loss_operation, feed_dict = {input_placeholder: stimuli, output_placeholder: frames})
      accuracy = sess.run(accuracy_operation, feed_dict = {input_placeholder: stimuli, output_placeholder: frames})

      print("Step " + str(iteration) + 
            ", Loss = " + "{:.4f}".format(loss) + 
            ", Training Accuracy= " + \
            "{:.3f}".format(accuracy))

  print("Optimization finished!")

这是输出示例：

Step 1, Loss = 3231716.2500, Training Accuracy= 0.012
Step 10, Loss = 5028286.0000, Training Accuracy= 0.013
Step 20, Loss = 8462586.0000, Training Accuracy= 0.013
Step 30, Loss = 13730642.0000, Training Accuracy= 0.013
Step 40, Loss = 21263566.0000, Training Accuracy= 0.013
Step 50, Loss = 31451778.0000, Training Accuracy= 0.013
Step 60, Loss = 44615720.0000, Training Accuracy= 0.013
Step 70, Loss = 61044256.0000, Training Accuracy= 0.013
Step 80, Loss = 80958576.0000, Training Accuracy= 0.013
Step 90, Loss = 104527496.0000, Training Accuracy= 0.013
Step 100, Loss = 131878944.0000, Training Accuracy= 0.013
Step 110, Loss = 163108640.0000, Training Accuracy= 0.013
Step 120, Loss = 198287152.0000, Training Accuracy= 0.013
Step 130, Loss = 237464976.0000, Training Accuracy= 0.013
Step 140, Loss = 280677600.0000, Training Accuracy= 0.013
Step 150, Loss = 327947968.0000, Training Accuracy= 0.013
Step 160, Loss = 379289344.0000, Training Accuracy= 0.013
Step 170, Loss = 434707488.0000, Training Accuracy= 0.013
Step 180, Loss = 494202976.0000, Training Accuracy= 0.013
Step 190, Loss = 557770496.0000, Training Accuracy= 0.013
Step 200, Loss = 625402304.0000, Training Accuracy= 0.013
Step 210, Loss = 697086144.0000, Training Accuracy= 0.013
Step 220, Loss = 772809856.0000, Training Accuracy= 0.013
Step 230, Loss = 852556672.0000, Training Accuracy= 0.013
Step 240, Loss = 936309952.0000, Training Accuracy= 0.013
Step 250, Loss = 1024050176.0000, Training Accuracy= 0.013
Step 260, Loss = 1115761920.0000, Training Accuracy= 0.013
Step 270, Loss = 1211422336.0000, Training Accuracy= 0.013
Step 280, Loss = 1311012096.0000, Training Accuracy= 0.013
Step 290, Loss = 1414512896.0000, Training Accuracy= 0.013
Step 300, Loss = 1521898112.0000, Training Accuracy= 0.013
Step 310, Loss = 1633154560.0000, Training Accuracy= 0.013
Step 320, Loss = 1748253184.0000, Training Accuracy= 0.013
Step 330, Loss = 1867175680.0000, Training Accuracy= 0.013
Step 340, Loss = 1989904384.0000, Training Accuracy= 0.013
Step 350, Loss = 2116416384.0000, Training Accuracy= 0.013
Step 360, Loss = 2246684416.0000, Training Accuracy= 0.013
Step 370, Loss = 2380691712.0000, Training Accuracy= 0.013
Step 380, Loss = 2518416384.0000, Training Accuracy= 0.013
Step 390, Loss = 2659838208.0000, Training Accuracy= 0.013
Step 400, Loss = 2804941312.0000, Training Accuracy= 0.013
Step 410, Loss = 2953692672.0000, Training Accuracy= 0.013
Step 420, Loss = 3106079232.0000, Training Accuracy= 0.013
Step 430, Loss = 3262086400.0000, Training Accuracy= 0.013
Step 440, Loss = 3421681152.0000, Training Accuracy= 0.013
Step 450, Loss = 3584851456.0000, Training Accuracy= 0.013
Step 460, Loss = 3751570688.0000, Training Accuracy= 0.013
Step 470, Loss = 3921825280.0000, Training Accuracy= 0.013
Step 480, Loss = 4095591168.0000, Training Accuracy= 0.013
Step 490, Loss = 4272850432.0000, Training Accuracy= 0.013
Step 500, Loss = 4453586432.0000, Training Accuracy= 0.013
Step 510, Loss = 4637780480.0000, Training Accuracy= 0.013
Step 520, Loss = 4825407488.0000, Training Accuracy= 0.013
Step 530, Loss = 5016456704.0000, Training Accuracy= 0.013
Step 540, Loss = 5210892800.0000, Training Accuracy= 0.013
Step 550, Loss = 5408712192.0000, Training Accuracy= 0.013
Step 560, Loss = 5609891840.0000, Training Accuracy= 0.013
Step 570, Loss = 5814412800.0000, Training Accuracy= 0.013
Step 580, Loss = 6022280704.0000, Training Accuracy= 0.013
Step 590, Loss = 6233428480.0000, Training Accuracy= 0.013
Step 600, Loss = 6447874048.0000, Training Accuracy= 0.013
Step 610, Loss = 6665582592.0000, Training Accuracy= 0.013
Step 620, Loss = 6886544896.0000, Training Accuracy= 0.013
Step 630, Loss = 7110739968.0000, Training Accuracy= 0.013
Step 640, Loss = 7338157056.0000, Training Accuracy= 0.013
Step 650, Loss = 7568768512.0000, Training Accuracy= 0.013
Step 660, Loss = 7802572288.0000, Training Accuracy= 0.013
Step 670, Loss = 8039540224.0000, Training Accuracy= 0.013
Step 680, Loss = 8279664128.0000, Training Accuracy= 0.013
Step 690, Loss = 8522910720.0000, Training Accuracy= 0.013
Step 700, Loss = 8769283072.0000, Training Accuracy= 0.013
Step 710, Loss = 9018755072.0000, Training Accuracy= 0.013
Step 720, Loss = 9271311360.0000, Training Accuracy= 0.013
Step 730, Loss = 9526939648.0000, Training Accuracy= 0.013
Step 740, Loss = 9785614336.0000, Training Accuracy= 0.013
Step 750, Loss = 10047337472.0000, Training Accuracy= 0.013
Step 760, Loss = 10312079360.0000, Training Accuracy= 0.013
Step 770, Loss = 10579824640.0000, Training Accuracy= 0.013
Step 780, Loss = 10850560000.0000, Training Accuracy= 0.013
Step 790, Loss = 11124272128.0000, Training Accuracy= 0.013
Step 800, Loss = 11400942592.0000, Training Accuracy= 0.013
Step 810, Loss = 11680576512.0000, Training Accuracy= 0.013
Step 820, Loss = 11963137024.0000, Training Accuracy= 0.013
Step 830, Loss = 12248598528.0000, Training Accuracy= 0.013
Step 840, Loss = 12536969216.0000, Training Accuracy= 0.013
Step 850, Loss = 12828226560.0000, Training Accuracy= 0.013
Step 860, Loss = 13122355200.0000, Training Accuracy= 0.013
Step 870, Loss = 13419335680.0000, Training Accuracy= 0.013
Step 880, Loss = 13719178240.0000, Training Accuracy= 0.013
Step 890, Loss = 14021866496.0000, Training Accuracy= 0.013
Step 900, Loss = 14327337984.0000, Training Accuracy= 0.013
Step 910, Loss = 14635613184.0000, Training Accuracy= 0.013
Step 920, Loss = 14946696192.0000, Training Accuracy= 0.013
Step 930, Loss = 15260555264.0000, Training Accuracy= 0.013
Step 940, Loss = 15577183232.0000, Training Accuracy= 0.013
Step 950, Loss = 15896529920.0000, Training Accuracy= 0.013
Step 960, Loss = 16218643456.0000, Training Accuracy= 0.013
Step 970, Loss = 16543496192.0000, Training Accuracy= 0.013
Step 980, Loss = 16871051264.0000, Training Accuracy= 0.013
Step 990, Loss = 17201252352.0000, Training Accuracy= 0.013
Step 1000, Loss = 17534134272.0000, Training Accuracy= 0.013
Optimization finished!

我的期望：

我希望损失开始时会很高，然后逐渐下降到一个较低的值，但是到处都是损失，并且怪异地处于较高的值。
我希望起初训练的准确性会很差，然后慢慢变好，但是会保持稳定。

问题：

我尝试生成数据并使网络作为其上的映射功能收敛的方式有意义吗？
我是否使用合理的损失和准确性操作？
有没有办法检查所有这些背后的数学运算？我发现很难不知不觉一直在黑暗中四处张望。

怪异的损失和训练准确性行为

0 个答案: