Question

玩具回归示例。使用dropout=0.0这可以正常工作并降低成本。使用dropout=0.5我收到错误：

ValueError: Got num_leading_axes=1 for a 1-dimensional input, leaving no trailing axes for the dot product.

有什么想法？

import theano
import theano.tensor as T
import lasagne
import numpy as np

num_features=10
N=1000

# Set up the network                                                                                                                                                                                                                       
x=T.fmatrix('x')
y=T.fvector('y')

dropout=0.5
network = lasagne.layers.InputLayer(shape=(None, num_features), input_var=x)
if dropout > 0.0:
    network = lasagne.layers.dropout(network, p=dropout),
network = lasagne.layers.DenseLayer( network, num_units=1, nonlinearity=None )

pred = lasagne.layers.get_output(network)

cost = lasagne.objectives.squared_error(pred, y).mean()

# Compile training function                                                                                                                                                                                                                
params = lasagne.layers.get_all_params(network, trainable=True)
train_fn = theano.function([x, y], cost, updates=lasagne.updates.adamax(cost, params) )

# Generate some synthetic data                                                                                                                                                                                                             
X=np.random.randn( N,num_features ).astype( theano.config.floatX )
b=np.random.randn( num_features ).astype( theano.config.floatX )
Y=np.dot( X, b ) + np.random.randn( N ).astype(theano.config.floatX ) * 0.1

# Train for 100 iterations                                                                                                                                                                                                                 
for i in range(100):
    print train_fn(X,Y)

Answer 1

删除dropout图层后的逗号。该代码将在InputLayer或DenseLayer之后立即使用dropout。逗号使用网络变量（网络，）创建一个元组，导致错误。

network = lasagne.layers.InputLayer(shape=(None, num_features), input_var=x)
if dropout > 0.0:
    network = lasagne.layers.dropout(network, p=dropout),
network = lasagne.layers.DenseLayer( network, num_units=1, nonlinearity=None )

为什么我不能在千层面回归模型的最后一层使用dropout？

1 个答案: