Question

如果我想在每批结束后更新特定层的输出激活，我该如何在Keras中做到这一点？

例如，特定层的输出形状= [9,1500]，其中9是批处理大小，而1500是输出要素。每个批次结束后如何更新此输出？

这是我的模型架构：

from keras.models import Sequential,Model  
from keras.layers import Input,Embedding,Conv1D,Multiply,Activation,MaxPooling1D,Dense,Flatten  

vocab_size=467   
outPut_dimention=8  
sequence_length=429278   
main_input = Input(shape=(sequence_length,), name='main_input')  
embedding_layer=Embedding(vocab_size+1,outPut_dimention,input_length=sequence_length)(main_input)  
one_D_conv_layer1=Conv1D(128,32,strides=32 , activation='sigmoid' )(embedding_layer)  
one_D_conv_layer2=Conv1D(128,32,strides=32, name="conv1d")(embedding_layer)  
merge=Multiply()([one_D_conv_layer1,one_D_conv_layer2])  
max_pooling=MaxPooling1D(pool_size=400)(merge)  
flat_layer=Flatten()(max_pooling)  
fully_connected=Dense(128)(flat_layer)  
main_output=Dense(9, activation='softmax')(fully_connected)  
model=Model(inputs=[main_input], outputs=[main_output])  

model.compile(loss='categorical_crossentropy',
              optimizer="Nadam",
              metrics=['accuracy'])  

history1=model.fit_generator(generator=training_generator,
                    validation_data=validation_generator,
                    use_multiprocessing=True,
                    workers=6,
                    epochs=1,callbacks=[myCallBack])

我想做的是，在培训过程中，我想在每批末尾更新full_connected层的输出。例如，如果fully_connected层的输出是形状为（？，128）的numpy数组，则其中？是批处理大小，那么我想在每个批处理末尾更新该numpy数组。

Answer 1

我在Keras官方网站link上找到了一种在“开发新的正则化器”部分下定义服装正则化的方法。由于我需要访问图层输出（batch_size，n_nodes），因此我需要定义一个自定义的activity_regularizer。

from keras import backend as K

class Regularizer(object):
    """Regularizer base class.
    """

    def __call__(self, x):
        return 0.

    @classmethod
    def from_config(cls, config):
        return cls(**config)



class DeCovRegularizer(Regularizer):

#     def set_layer(self, layer):
#         self.layer = layer

    def __call__(self, x):
#         print(x.shape)
        # x will be the output of the layer: (batch_size,n_nodes)
        # means=[u1,u2,u3,u4,....] : where u1 is the (sum of node1 over the batch size)/batch_size
        means=(K.sum(x,axis=0)/batch_size)  #axis=0 to sum over the column not the rows 
        sub_mean=x-means # we subtract the mean from the nodes; for Example, node1-u1, node2-u2 ...... 
        node_numbers=x.shape[1] # in our example it will be 128
        # Constructing the C matrix; however it will be 1Dim=[c1_2,c1_3....c1_n,c2_1,c2_3......,c2_d.......]
        C=K.variable(np.array([0])) # initialize C with 0 --> C=[0]

        for i in range (0,node_numbers):
            for j in range(0,node_numbers):
                if(i !=j): 
                    C_i_j=K.sum(sub_mean[:,i]*sub_mean[:,j],keepdims=True)
                    C=K.concatenate([C, C_i_j], axis=-1)

        DecovLoss=K.sum(C*C)/0.5
        return DecovLoss

#     def get_config(self):
#         return {"name": self.__class__.__name__,
#             "p": self.l1}

在我的模型中，我添加了Decov损失，如下所示：

fully_connected=Dense(128,activity_regularizer=DeCovRegularizer())(flat_layer)

只需确保在训练过程中通过 call 函数返回的值将被添加到主要损失（交叉熵）中，对吗？这样Decov损失将自动添加到交叉熵损失中，对吗？我可以正确实现DeCov损失吗？

如何使用Keras在每个批次结束后更新激活功能的输出？

1 个答案: