Question

亲爱的，有没有人有一个想法为什么'dropout_rate'和'learning_rate'只返回0并且不会搜索我在超参数上进行RandomizedSearchCV时给出的范围？

以下是使用keras / tensoflow进行人工神经网络的代码：

# Create the model

    def create_model(neurons = 1, init_mode = 'uniform', activation='relu', inputDim = 8792, dropout_rate=0.7, learn_rate=0.01, momentum=0, weight_constraint=0): #, learn_rate=0.01, momentum=0):
        model = Sequential()
    model.add(Dense(neurons, input_dim=inputDim, kernel_initializer=init_mode, activation=activation, kernel_constraint=maxnorm(weight_constraint), kernel_regularizer=regularizers.l2(0.001))) # one inner layer
    #model.add(Dense(neurons, input_dim=inputDim, activation=activation)) # second inner layer
    model.add(Dropout(dropout_rate))
    model.add(Dense(1, activation='sigmoid'))
    optimizer = RMSprop(lr=learn_rate)
# compile model
model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
return model

# model

model = KerasClassifier(build_fn=create_model, verbose=0)

# Define K-fold cross validation test harness

kfold = StratifiedKFold(n_splits=3, shuffle=True, random_state=seed)
for train, test in kfold.split(X_train, Y_train):
    print("TRAIN:", train, "VALIDATION:", test)

# Define Hyperparameters

# specify parameters and distributions to sample from
from scipy.stats import randint as sp_randint

param_dist = {'neurons': sp_randint(300, 360), #, 175, 180, 185, 190, 195, 200],
                  'learn_rate': sp_randint (0.001, 0.01), 
                  'batch_size': sp_randint(50, 60),
                  'epochs': sp_randint(20, 30),
                  'dropout_rate': sp_randint(0.2, 0.8),
                  'weight_constraint': sp_randint(3, 8)
                 }

# run randomized search
n_iter_search = 100

print("[INFO] Starting training digits")
print("[INFO] Tuning hyper-parameters for accuracy")
grid = RandomizedSearchCV(estimator=model, param_distributions=param_dist,
                                   n_iter=n_iter_search, n_jobs=10, cv=kfold)
start = time.time()
grid_result = grid.fit(X_train, Y_train)
print("[INFO] GridSearch took {:.2f} seconds".format(time.time() - start))

我的回答：

[INFO] GridSearch took 1164.39 seconds
[INFO] GridSearch best score 1.000000 using parameters: {'batch_size': 54, 'dropout_rate': 0, 'epochs': 20, 'learn_rate': 0, 'neurons': 331, 'weight_constraint': 7}
[INFO] Grid scores on development set:
0.614679 (0.034327) with: {'batch_size': 54, 'dropout_rate': 0, 'epochs': 29, 'learn_rate': 0, 'neurons': 354, 'weight_constraint': 6}
0.883792 (0.008650) with: {'batch_size': 53, 'dropout_rate': 0, 'epochs': 27, 'learn_rate': 0, 'neurons': 339, 'weight_constraint': 7}
0.256881 (0.012974) with: {'batch_size': 59, 'dropout_rate': 0, 'epochs': 27, 'learn_rate': 0, 'neurons': 308, 'weight_constraint': 4}
...

感谢您的帮助。

Answer 1

0.2和0.8不是整数，因此当使用sp_randint（0.2,0.8）时，这些将转换为整数，因此它与sp_randint（0,0）相同。您必须使用生成浮点数的等效函数，而不是整数。

例如，您可以使用统一分布（来自scipy.stats的统一）来生成实数。

dropout_rate和learning_rate在RandomSearchCV中不会改变

1 个答案: