如何创建一个模型,每次我们改变学习率时优化都从头开始?

时间:2018-04-10 06:19:58

标签: python machine-learning deep-learning keras

也许有人可以帮助我 - 我正在使用顺序模型中的学习率。我希望每当我改变学习率时,优化就从头开始,以便公平地比较每个学习率在结果中的表现。那么如何在python中创建一个函数来生成一个新模型来优化循环中的学习速率?

""" optimizing learning rate""" 

# Create list of learning rates: lr_to_test
lr_to_test = [0.000001, 0.01, 1]

# Loop over learning rates
for lr in lr_to_test:
    print('\n\nTesting model with learning rate: %f\n'%lr )

    # Build new model to test, unaffected by previous models
    model = Sequential()

    # Add the layers
    model.add(Dense(50, activation='relu', input_shape=(n_cols,)))
    model.add(Dense(32, activation='relu'))
    model.add(Dense(1))

    # Create SGD optimizer with specified learning rate: my_optimizer
    my_optimizer = SGD(lr=lr)

    # Compile the model
    model.compile(optimizer=my_optimizer, loss='mean_squared_error')

    # Fit the model
    model.fit(predictors, target, epochs=10)

结果我得到了:

Testing model with learning rate: 0.000001

Epoch 1/10
534/534 [==============================] - 0s 661us/step - loss: 120.5427
Epoch 2/10
534/534 [==============================] - 0s 29us/step - loss: 111.6158
.....
Epoch 10/10
534/534 [==============================] - 0s 59us/step - loss: 65.8593


Testing model with learning rate: 0.010000

Epoch 1/10
534/534 [==============================] - 0s 693us/step - loss: nan
Epoch 2/10
534/534 [==============================] - 0s 59us/step - loss: nan
Epoch 3/10
534/534 [==============================] - 0s 29us/step - loss: nan 
....<>

1 个答案:

答案 0 :(得分:1)

您可以循环查看学习率列表并在最后评估结果,以便了解哪种费率最适合您。

learning_rates = [0.00001, 0.0001, 0.001, 0.01, 0.1]
best_lr = 0
best_rmse = 999999
for lr in learning_rates:
    """Build sequential model"""
    my_optimizer = SGD(lr=lr)

    """Compile, fit and evaluate"""
    rmse = "Calculate your evaluation metric"

    if rmse < best_rmse:
        best_rmse = rmse
        best_lr = lr