除了学习速度和激活函数之外，我还在尝试优化深层神经网络的超参数，其结构（每个隐层的数量和每个神经元的数量）。
如您所见，我具有需要优化的主要功能（适应性），我的问题是优化功能没有采用默认的参数集。
它给了我以下输出：
fitness（）缺少3个必需的位置参数：“ num_dense_layers”，“ num_dense_nodes”和“激活”
这意味着它不会从默认参数列表中获取多个参数
我觉得这确实是一个小错误
请注意，Create_model和健身功能正常运行
任何建议？
def create_model(learning_rate,num_dense_layers,
num_dense_nodes, activation):
"""
Hyper-parameters:
learning_rate: Learning-rate for the optimizer.
num_dense_layers: Number of dense layers.
num_dense_nodes: Number of nodes in each dense layer.
activation: Activation function for all layers.
"""
# Start construction of a Keras Sequential model.
model = Sequential()
# Add an input layer which is similar to a feed_dict in TensorFlow.
# Note that the input-shape must be a tuple containing the image-size.
model.add(InputLayer(input_shape=(4,)))
# Add fully-connected / dense layers.
# The number of layers is a hyper-parameter we want to optimize.
for i in range(num_dense_layers):
# Name of the layer. This is not really necessary
# because Keras should give them unique names.
name = 'layer_dense_{0}'.format(i+1)
# Add the dense / fully-connected layer to the model.
# This has two hyper-parameters we want to optimize:
# The number of nodes and the activation function.
model.add(Dense(num_dense_nodes,
activation=activation,
name=name))
# Last fully-connected / dense layer with softmax-activation
# for use in classification.
model.add(Dense(1, activation='linear'))
# Use the Adam method for training the network.
# We want to find the best learning-rate for the Adam method.
optimizer = Adam(lr=learning_rate)
#sgd = SGD(lr=learning_rate, momentum=0.9, decay=1e-6, nesterov=True)
# In Keras we need to compile the model so it can be trained.
model.compile(optimizer= optimizer ,
loss='mse',
metrics=['mse'])
return model
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=1)
x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=0.25, random_state=1) # 0.25 x 0.8 = 0.2
print (x_train.shape, y_train.shape)
print (x_test.shape, y_test.shape)
print (x_val.shape, y_val.shape)
#evaluating the model
path_best_model = '19_best_model.h5'
best_mse = 0.0
def fitness(learning_rate, num_dense_layers,
num_dense_nodes, activation):
"""
Hyper-parameters:
learning_rate: Learning-rate for the optimizer.
num_dense_layers: Number of dense layers.
num_dense_nodes: Number of nodes in each dense layer.
activation: Activation function for all layers.
"""
# Print the hyper-parameters.
print('learning rate: {0:.1e}'.format(learning_rate))
print('num_dense_layers:', num_dense_layers)
print('num_dense_nodes:', num_dense_nodes)
print('activation:', activation)
print()
# Create the neural network with these hyper-parameters.
model = create_model(learning_rate=learning_rate,
num_dense_layers=num_dense_layers,
num_dense_nodes=num_dense_nodes,
activation=activation)
'''
# Dir-name for the TensorBoard log-files.
log_dir = log_dir_name(learning_rate, num_dense_layers,
num_dense_nodes, activation)
# Create a callback-function for Keras which will be
# run after each epoch has ended during training.
# This saves the log-files for TensorBoard.
# Note that there are complications when histogram_freq=1.
# It might give strange errors and it also does not properly
# support Keras data-generators for the validation-set.
callback_log = TensorBoard(
log_dir=log_dir,
histogram_freq=0,
write_graph=True,
write_grads=False,
write_images=False)
we have a callback peoblem
ProfilerNotRunningError: Cannot stop profiling. No profiler is running.
the solution is somewho discussed on
tensorflow website
'''
# Use Keras to train the model.
history = model.fit(x_train,
y_train,
epochs=200,
batch_size=1,
validation_data= (x_val,y_val))
# Get the classification accuracy on the validation-set
# after the last training-epoch.
mse = history.history['val_mse'][-1]
# Print the classification accuracy.
print()
print("MSE: {0:.2%}".format(mse))
print()
K.clear_session()
tensorflow.reset_default_graph()
return mse
# Define the Hyperparameters ranges:
dim_learning_rate = Real(low=1e-6, high=1e-2, prior='log-uniform', name='learning_rate')
dim_num_dense_layers = Integer(low=1, high=15, name='num_dense_layers')
dim_num_dense_nodes = Integer(low=5, high=512, name='num_dense_nodes')
dim_activation = Categorical(categories=['relu', 'sigmoid','tanh','softmax'],name='activation')
HPs = [dim_learning_rate,dim_num_dense_layers,dim_num_dense_nodes,dim_activation]
default_parameters = [1e-5, 1, 16, 'relu']
search_result = gp_minimize(func=fitness,
dimensions=HPs,
acq_func='EI', # Expected Improvement.
n_calls=40,
x0=default_parameters)