Question

我是pytorch的新手，他想把头缠住。我已经阅读了自定义损失函数，据我所知，它们不能与内部计算图分离。这意味着损失函数会消耗张量，对其进行操作（在pytorch中实现），然后输出张量。有什么方法可以使损耗计算解耦并以某种方式重新插入？

用例

我正在尝试训练编码器，在该编码器中，潜在空间将被优化以达到某种统计质量。这意味着我不分批训练，而是为整个时期和整个数据集计算单个损失值。甚至有可能以这种方式教网络吗？

class Encoder(nn.Module):
    def __init__(self, genome_size: int):
        super(Encoder, self).__init__()

        self.fc1 = nn.Linear(genome_size, genome_size)
        self.fc2 = nn.Linear(genome_size, genome_size)
        self.fc3 = nn.Linear(genome_size, genome_size)

        self.genome_size = genome_size

    def forward(self, x):
        x = self.fc1(x)
        x = self.fc2(x)
        x = self.fc3(x)

        return x

def train_encoder(
    net: nn.Module,
    optimizer: Optimizer,
    epochs: int,
    population: Tensor,
    fitness: Tensor,
):
    running_loss = 0.0

    for epoch in range(epochs):

        optimizer.zero_grad()
        outputs = net(population)

        # encoder_loss is computationally heavy and cannot be done only on tensors
        # I need to unwrap those tensors to numpy arrays and use them as an input to another model
        loss = encoder_loss(outputs, fitness)
        running_loss += loss

        running_loss.backward()
        optimizer.step()

        print('Encoder loss:', loss)

我看到了一些带有累积running_loss的示例，但是我的编码器无法学习任何东西。收敛图只是到处跳跃。

感谢您的时间<3

PyTorch-自定义损失函数-图形外部计算

0 个答案: