Question

我刚刚完成了pytorch代码中一个疯狂错误的调试，因此我想分享自己的经验。

作为一个最小的示例，下面的一些代码会崩溃：

import torch
ts = torch.tensor
t = ts([i+.5 for i in range(5)], requires_grad=True) #some tensor
f = torch.sum(t**3) #some function of t
[lg] = torch.autograd.grad(f,t,create_graph=True,retain_graph=True) #the gradient of f wrt t
[lgg] = torch.autograd.grad(lg[1],t,create_graph=True,retain_graph=True) #one row of the Hessian of f wrt t
lgg2 = lgg.contiguous().view(-1)
lgg3 = torch.zeros(6,6,requires_grad=True) #if we were calculating the full Hessian, it would be a matrix...
lgg3[1,1:].add_(lgg2.type_as(lgg3)) #...and we'd fill it in row by row 
l2 = torch.sum(lgg3) #a loss that's a function of the Hessian
l2.backward() #we want the gradient of that loss
print(t.grad)
print(lgg3.requires_grad)

出现错误：

RuntimeError: leaf variable has been moved into the graph interior

如何解决？我会给你答案...

Answer 1

在创建lgg3时不要设置requires_grad。在此代码末尾，lgg3仍将其设置为True，但这将停止错误。这是伏都教，但有效。

import torch
ts = torch.tensor
t = ts([i+.5 for i in range(5)], requires_grad=True) #some tensor
f = torch.sum(t**3) #some function of t
[lg] = torch.autograd.grad(f,t,create_graph=True,retain_graph=True) #the gradient of f wrt t
[lgg] = torch.autograd.grad(lg[1],t,create_graph=True,retain_graph=True) #one row of the Hessian of f wrt t
lgg2 = lgg.contiguous().view(-1)
lgg3 = torch.zeros(6,6) #if we were calculating the full Hessian, it would be a matrix...
lgg3[1,1:].add_(lgg2.type_as(lgg3)) #...and we'd fill it in row by row 
l2 = torch.sum(lgg3) #a loss that's a function of the Hessian
l2.backward() #we want the gradient of that loss
print(t.grad)
print(lgg3.requires_grad) #look, it's True.

Pytorch：根据Hessian的功能向后区分：

1 个答案: