Question

我正在python 3.6.8中使用tensorflow 1.14.0训练去噪自动编码器。每个训练步骤都包括模型的拟合，并且还应该根据先前的预测来优化另一个变量theta（用于损失的二项式分布），这会对下一个预测产生影响。

这是我的问题：在每个列的每个训练步骤之后，如何最小化该θ？那么基于当前预测的替代训练作为每列最小化问题的输入？

我已经尝试过的方法：

使用了带有最小化功能的tf.map_fn，该功能为每个列引入了新的会话和优化器以将其最小化->效果很好，但由于创建了多个会话而太慢了（请参阅Minimize a function of one variable in Tensorflow中的tf_minimize）
为每一列创建优化器哈希表以最小化theta->单独优化一万列优化器是不可行的（比较：How to alternate train op's in tensorflow?）

### no functioning python code, just to give you an idea
### input in tensorflow model: ae_input, theta


### training steps
for epoch in range(200):
    model.fit( x = ae_input+theta,
        y = ae_input,
        epochs = epoch+1)
    current_pred = model.predict(ae_input+theta)

    theta = update_theta(ae_input, current_pred)   # PROBLEMATIC STEP


### current way because map_fn is not working and
### would also be too slow due to multiple session creation
def update_theta(y_true, y_pred):
    theta_list=[]
    for col in range(y_true.get_shape()[1]):
        t = theta_optimize(y_true[:,x], y_pred[:,x])
        theta_list.append(t)
    return theta_list

### minimize theta for single column
def theta_optimize(y_t, y_p):
    var_theta = tf.Variable(5.)
    func_to_minimize = loss_per_col(y_t, y_p, var_theta) 
    optimial_theta = tf_minimize(func_to_optimize, output=var_theta) 
    ### tf_minimize from link mentioned above
    return optimal_theta

Answer 1

对于这些仍然感兴趣的人，我通过在每个步骤之间切换到scipy来解决了这个问题。当然不是理想的方法，但是速度更快并且对我来说效果很好

### CODE FOR TENSORFLOW 2.0.0 beta

### function for scipy to minimize theta between 0-1000
def theta_loss_per_col(y_t, y_p, theta):
    # ... 
    return loss


### function for scipy to optimize
def func_minimize_theta(y_t, y_p):
    return lambda x: theta_loss_per_col(y_t, y_p, x).numpy()


### return updated list of thetas
def get_updated_theta(y_true, y_pred):
    y_true_cols = tf.range(y_true.shape[1])
    theta_list =  tf.map_fn(lambda i:
scipy.optimize.fminbound(func_minimize_theta(y_true[:,i], y_pred[:,i]),
1e-5, 1000) , y_true_cols, dtype=tf.float32) # maybe add:parallel_iterations=10
    return theta_list

Tensorflow-替代训练：最小化每列功能

1 个答案: