Question

我正在尝试使用GAN重新创建深度照明（https://arxiv.org/pdf/1710.09834.pdf），但我没有学习网络来渲染全局照明，而是只关注环境光遮挡。

在我的情况下，输入的是法线贴图和深度图，组合后得到的图像具有四个通道-（240、320、4）

def build_generator(image_shape):

    inputs = Input(shape = image_shape)

    x1 = Conv2D(filters=4, kernel_size=(3,3), strides = 1, padding='same')(inputs)
    x1 = LeakyReLU(0.2)(x1)
    x1 = BatchNormalization()(x1)

    x2 = Conv2D(filters=8, kernel_size=(3,3), strides = 2, padding='same')(x1)
    x2 = LeakyReLU(0.2)(x2)
    x2 = BatchNormalization()(x2)

(...)

    x1s = Conv2DTranspose(filters=4, kernel_size=(3,3), strides = 2, padding='same')(x2s)
    x1s = LeakyReLU(0.2)(x1s)
    x1s = BatchNormalization()(x1s)

    x1s = Add()([x1s, x1]) # SKIP Connection

    x_out = Conv2DTranspose(filters=1, kernel_size=(3,3), strides = 1, padding='same')(x1s)
    output = Activation("tanh")(x_out)

    model = Model(inputs=inputs, outputs=output, name='Generator')
    return model

生成器有更多的卷积，但是我切断了不占太多的位置。内核大小保持不变，步幅也一样，除了中间一层包含32个过滤器的步幅为1。

def build_discriminator(input_shape):    

    inputs = Input(shape = input_shape)

    x = Conv2D(filters = 4, kernel_size = 3, strides = 2, padding = "same", input_shape=input_shape)(inputs)
    x = LeakyReLU(0.2)(x)

    x = Conv2D(filters = 8, kernel_size = 3, strides = 2, padding = "same", input_shape=input_shape)(x)
    x = LeakyReLU(0.2)(x)
    x = BatchNormalization()(x)

    x = Conv2D(filters = 16, kernel_size = 3, strides = 1, padding = "same", input_shape=input_shape)(x)
    x = LeakyReLU(0.2)(x)
    x = BatchNormalization()(x)

    x = Conv2D(filters = 32, kernel_size = 3, strides = 1, padding = "same", input_shape=input_shape)(x)
    x = LeakyReLU(0.2)(x)
    x = BatchNormalization()(x)

    x = Flatten()(x)
    x = Dense(1, activation='sigmoid')(x)

    model = Model(inputs=inputs, outputs=x, name='Discriminator')
    return model

def get_gan_network(discriminator, shape, generator, optimizer, d_loss):
    discriminator.trainable = False
    gan_input = Input(shape=shape)
    x = generator(gan_input)
    gan_output = discriminator(x)
    gan = Model(inputs=gan_input, outputs=[x,gan_output])
    gan.compile(loss=[d_loss, "binary_crossentropy"], loss_weights=[1., 1e-3], optimizer=optimizer)

    return gan

def train(X, Y, epochs, steps, batch_size):#, save_interval=50):
    valid = np.ones((batch_size, 1))
    fake = np.zeros((batch_size, 1))

    for epoch in range(epochs):
        for step in range(steps):
            rand_num = np.random.permutation(num_of_data)
            image_batch_x = X[rand_num[:batch_size]]
            image_batch_y = Y[rand_num[:batch_size]]
            gen_imgs = g.predict(image_batch_x)

            d.trainable = True
            d_loss_real = d.train_on_batch(image_batch_y, valid)
            d_loss_fake = d.train_on_batch(gen_imgs, fake)
            d_loss = 0.5 * (d_loss_fake + d_loss_real)
            d.trainable = False

            g_loss = gan.train_on_batch(image_batch_x, [image_batch_y, valid])

我测试了各种损失函数，批量大小，损失重量和优化器-在所有情况下，它都会导致损失函数降至零，而GAN的损失则停留在某个值ex上。 16,75344。话虽这么说-我应该如何解释损失函数值，我应该期望什么值？

由于GAN非常复杂且难以训练，因此我不希望得到完美的解决方案作为答案。但是也许我缺少明显的东西，或者犯了一些我无法捕捉到的愚蠢错误，导致培训失败。

对于任何建议或想法，我将非常感谢。

GAN的环境光遮挡-适当的架构和损失函数？

0 个答案: