Question

我有一个用于文本识别的CRNN模型，该模型已在Github上发布，并接受了英语培训，

现在我正在使用此算法执行相同的操作，但是是阿拉伯语。

我的ctc函数是：

def ctc_lambda_func(args):
        y_pred, labels, input_length, label_length = args
        # the 2 is critical here since the first couple outputs of the RNN
        # tend to be garbage:
        y_pred = y_pred[:, 2:, :]
        return K.ctc_batch_cost(labels, y_pred, input_length, label_length)

我的模特是：

def get_Model(training):

        img_w = 128
        img_h = 64


        # Network parameters
        conv_filters = 16
        kernel_size = (3, 3)
        pool_size = 2
        time_dense_size = 32
        rnn_size = 128


        if K.image_data_format() == 'channels_first':
            input_shape = (1, img_w, img_h)
        else:
            input_shape = (img_w, img_h, 1)
        # Initialising the CNN
        act = 'relu'
        input_data = Input(name='the_input', shape=input_shape, dtype='float32')
        inner = Conv2D(conv_filters, kernel_size, padding='same',
                           activation=act, kernel_initializer='he_normal',
                           name='conv1')(input_data)
        inner = MaxPooling2D(pool_size=(pool_size, pool_size), name='max1')(inner)
        inner = Conv2D(conv_filters, kernel_size, padding='same',
                           activation=act, kernel_initializer='he_normal',
                           name='conv2')(inner)
        inner = MaxPooling2D(pool_size=(pool_size, pool_size), name='max2')(inner)

        conv_to_rnn_dims = (img_w // (pool_size ** 2), (img_h // (pool_size ** 2)) * conv_filters)
        inner = Reshape(target_shape=conv_to_rnn_dims, name='reshape')(inner)

            # cuts down input size going into RNN:
        inner = Dense(time_dense_size, activation=act, name='dense1')(inner)

            # Two layers of bidirectional GRUs
            # GRU seems to work as well, if not better than LSTM:
        gru_1 = GRU(rnn_size, return_sequences=True, kernel_initializer='he_normal', name='gru1')(inner)
        gru_1b = GRU(rnn_size, return_sequences=True, go_backwards=True, kernel_initializer='he_normal', name='gru1_b')(inner)
        gru1_merged = add([gru_1, gru_1b])
        gru_2 = GRU(rnn_size, return_sequences=True, kernel_initializer='he_normal', name='gru2')(gru1_merged)
        gru_2b = GRU(rnn_size, return_sequences=True, go_backwards=True, kernel_initializer='he_normal', name='gru2_b')(gru1_merged)

            # transforms RNN output to character activations:
        inner = Dense(num_classes+1, kernel_initializer='he_normal',
                          name='dense2')(concatenate([gru_2, gru_2b]))
        y_pred = Activation('softmax', name='softmax')(inner)
        Model(inputs=input_data, outputs=y_pred).summary()

        labels = Input(name='the_labels', shape=[30], dtype='float32')
        input_length = Input(name='input_length', shape=[1], dtype='int64')
        label_length = Input(name='label_length', shape=[1], dtype='int64')
            # Keras doesn't currently support loss funcs with extra parameters
            # so CTC loss is implemented in a lambda layer
        loss_out = Lambda(ctc_lambda_func, output_shape=(1,), name='ctc')([y_pred, labels, input_length, label_length])

            # clipnorm seems to speeds up convergence



            # the loss calc occurs elsewhere, so use a dummy lambda func for the loss

        if training:
            return Model(inputs=[input_data, labels, input_length, label_length], outputs=loss_out)

            return Model(inputs=[input_data], outputs=y_pred)

然后我使用SGD优化器（尝试过SGD，adam）对其进行编译

sgd = SGD(lr=0.0000002, decay=1e-6, momentum=0.9, nesterov=True, clipnorm=5)
model.compile(loss={'ctc': lambda y_true, y_pred: y_pred}, optimizer=sgd)

然后，我将模型与我的训练集（最多30个字符的单词图像）拟合到（标签顺序为30

model.fit_generator(generator=tiger_train.next_batch(),
                steps_per_epoch=int(tiger_train.n / batch_size),
                epochs=30,
                callbacks=[checkpoint],
                validation_data=tiger_val.next_batch(),
                validation_steps=int(tiger_val.n / val_batch_size))

一旦启动，它给我的损失= inf，经过多次搜索，我没有发现任何类似的问题。

所以我的问题是，我该如何解决这个问题，怎样才能使ctc_loss计算出无限的成本？

预先感谢

Answer 1

当图像文本具有相同顺序的两个相等字符时会发生此错误，例如发生-> pp。，以便您可以删除具有此特征的数据。

Answer 2

我发现了问题，这是尺寸问题，

对于使用CTC layer的{{1}}，如果您检测到length n的序列，则图像的宽度至少应为(2*n-1)。越好，直到您达到最佳图像/时间步长比例，以使CTC layer能够正确识别字母。如果的图片小于(2*n-1)，则会损失nan。

深度学习Keras模型CTC_Loss给出损失=无穷大

2 个答案: