Question

我正在尝试按像素进行变换，以使一个图像（+背景）适合结果。 Background Image + Input image应转换为desired result

要实现此目的，我正在使用PyTorch gridsampler和autograd来优化网格。转换后的输入将添加到未更改的背景中。


ToTensor = torchvision.transforms.ToTensor()
FromTensor = torchvision.transforms.ToPILImage()

backround= ToTensor(Image.open("backround.png"))
pic = ToTensor(Image.open("pic.png"))
goal = ToTensor(Image.open("goal.png"))

empty = empty.expand(1, 3, empty.size()[1], empty.size()[2])
pic = pic.expand(1, 3, pic.size()[1], pic.size()[2])
goal = goal.expand(1, 3, goal.size()[1], goal.size()[2])

def createIdentityGrid(w, h):
    grid = torch.zeros(1, w, h, 2);
    for x in range(w):
        for y in range(h):
            grid[0][x][y][1] = 2 / w * (0.5 + x) - 1
            grid[0][x][y][0] = 2 / h * (0.5 + y) - 1
    return grid

w = 256; h=256 #hardcoded imagesize

grid = createIdentityGrid(w, h)
grid.requires_grad = True

for i in range(300):
    goal_pred = torch.nn.functional.grid_sample(pic, grid)[0]
    goal_pred = (empty + 0.75 * goal_pred).clamp(min=0, max=1)
    out = goal_pred

    loss = (goal_pred - goal).pow(2).sum()
    loss.backward()

    with torch.no_grad():
        grid -= grid.grad * (1e-2)
        grid.grad.zero_()

FromTensor(out[0]).show()

这些是结果：

实际上这个简单的例子可以解决，但是我观察到一些奇怪的行为。网格刚开始在一侧发生变化。为什么会这样，为什么整个网格不会立即更改？我缺少一些明显的部分吗？

Answer 1

from PIL import Image
import torch

ToTensor = torchvision.transforms.ToTensor()
FromTensor = torchvision.transforms.ToPILImage()

lr = 1
backround= ToTensor(Image.open(r"C:\Users\dj\Pictures\Saved Pictures\background.png"))
pic = ToTensor(Image.open(r"C:\Users\dj\Pictures\Saved Pictures\input.png"))
goal = ToTensor(Image.open(r"C:\Users\dj\Pictures\Saved Pictures\result.png"))

empty = backround.expand(1, 3, backround.size()[1], backround.size()[2])
pic = pic.expand(1, 3, pic.size()[1], pic.size()[2])
goal = goal.expand(1, 3, goal.size()[1], goal.size()[2])

def createIdentityGrid(w, h):
    grid = torch.zeros(1, w, h, 2);
    for x in range(w):
        for y in range(h):
            grid[0][x][y][1] = 2 / w * (0.5 + y) - 1
            grid[0][x][y][0] = 2 / h * (0.5 + x) - 1
    return grid

w = 256; h=256 #hardcoded imagesize

grid = createIdentityGrid(w, h)
grid.requires_grad = True

for i in range(9):
    goal_pred = torch.nn.functional.grid_sample(pic, grid, mode="bilinear")[0]
    goal_pred = F.relu(empty + 0.75 * goal_pred)
    out = goal_pred

    loss = (goal_pred - goal).pow(2).sum()
    loss.backward()

    with torch.no_grad():
        grid -= grid.grad * lr
        lr = lr/1.1 #learning rate a0-ing
        grid.grad.zero_()

FromTensor(out[0]).show()

实际上这个简单的例子可以解决，但是我观察到一些奇怪的行为。网格刚开始在一侧发生变化。为什么会这样，为什么整个网格不会立即更改？

我不知道。我只是骗了你的榜样，对我来说是自下而上的。

使用gridsampler进行PyTorch图像转换，在优化网格时出现怪异的行为

1 个答案: