我正在寻找一种在每个时期(或每n个时期)重新开始随机分配一部分权重的方法,我发现this解释了如何重新初始化层。我可以使用
weights = layer.get_weights()
,然后使用numpy操作以重新初始化一部分权重,或创建一个虚拟层,从其中提取新的初始化权重并将其与set_weights一起使用。我正在寻找一种更优雅的方法来将我的权重的某个(或随机)部分初始化为一个层。
谢谢
答案 0 :(得分:1)
Keras具有set_weights方法来设置图层的权重。要在每个纪元处重置图层的权重,请使用回调。
class My_Callback(keras.callbacks.Callback):
def on_epoch_begin(self, logs={}):
return
def on_epoch_end(self, epoch, logs={}):
layer_index = 0 ## index of the layer you want to change
# random weights to reset the layer
new_weights = numpy.random.randn(*self.model.layers[layer_index].get_weights().shape)
self.model.layers[layer_index].set_weights(new_weights)
要重置层的随机n个权重,可以使用numpy来获取要重置的随机索引。现在的代码是
def on_epoch_end(self, epoch, logs={}):
layer_index = np.random.randint(len(self.model.layers)) # Random layer index to reset
weights_shape = self.model.layers.get_weights().shape
num = 10 # number of weights to reset
indexes = np.random.choice(weights_shape[0], num, replace=False) # indexes of the layer to reset
reset_weights = numpy.random.randn(*weights_shape[1:]) # random weights to reset the layer
layer_weights = self.model.layers[layer_index].get_weights()
layer_weights[indexes] = reset_weights
self.model.layers[layer_index].set_weights(layer_weights)
类似于重置层的随机权重p %
,第一个numpy可用于选择层权重的p %
索引。
def on_epoch_end(self, epoch, logs={}):
layer_index = np.random.randint(len(self.model.layers)) # Random layer index to reset
weights_shape = self.model.layers.get_weights().shape
percent = 10 # Percentage of weights to reset
indexes = np.random.choice(weights_shape[0], int(percent/100.) * weights_shape[0], replace=False) # indexes of the layer to reset
reset_weights = numpy.random.randn(*weights_shape[1:]) # random weights to reset the layer
layer_weights = self.model.layers[layer_index].get_weights()
layer_weights[indexes] = reset_weights
self.model.layers[layer_index].set_weights(layer_weights)