Question

我正在尝试在PyTorch中创建卷积模型，

固定一层（初始化为指定值）
学习了另一层（但最初的猜测来自规定值）。

这是模型定义的示例代码：

import torch.nn as nn

class Net(nn.Module):
    def __init__(self, weights_fixed, weights_guess):
        super(Net, self).__init__()
        self.convL1 = nn.Conv1d(1, 3, 3, bias=False)
        self.convL1.weight = weights_fixed # I want to keep these weights fixed

        self.convL2 = nn.Conv1d(3, 1, 1, bias=False)
        self.convL1.weight = weights_guess # I want to learn these weights

    def forward(self, inp_batch):
        out1 = self.convL1(inp_batch)
        out2 = self.convL2(out1)

        return out2

和示例用法：

weights_fixed = ...
weights_guess = ...

model = Net(weights_fixed, weights_guess)

loss_fn = nn.CrossEntropyLoss()
optim = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)

train_dataset = ... #define training set here

for (X, y) in train_dataset:
    optim.zero_grad()
    out = model(X)
    loss = loss_fn(out, y)
    loss.backward()
    optim.step()

我如何进行举重 weights_fixed-固定和 weights_guess-可以学习吗？

我的猜测是 weights_fixed = nn.Parameter（W1，requires_grad = False） weights_guess = nn.Parameter（W2，requires_grad = True）为了完整起见将numpy导入为np 进口火炬

krnl = np.zeros((5,order+1))
krnl[:,0] = [ 0. , 1., 0. ]
krnl[:,1] = [-0.5, 0., 0.5]
krnl[:,2] = [ 1. ,-2., 1. ]
W1 = torch.tensor(krnl)

a = np.array((1.,2.,3.))
W2 = torch.tensor(a)

但是我完全感到困惑。任何建议或参考将不胜感激。当然，我浏览了PyTorch文档，但是并没有增加我的理解范围。

Answer 1

您可以这样做：

# this will be inside your class mostly
self.conv1.weight.requires_grad = False

这就是您定义优化器的地方：

optimizer = optim.SGD(filter(lambda p: p.requires_grad, net.parameters()), lr=0.1)

因此，优化器将仅使用启用了渐变的参数。

Answer 2

只需使用nn.Parameter包装可学习的参数（默认为requires_grad=True，无需指定此参数），并且将权重固定为没有nn.Parameter包装器的张量。

所有nn.Parameter权重都会自动添加到net.parameters()中，因此当您像optimizer = optim.SGD(net.parameters(), lr=0.01)这样进行训练时，固定权重将不会更改。

基本上就是这样：

weights_fixed = W1
weights_guess = nn.Parameter(W2)

Answer 3

您可以仅将要学习的参数传递给优化器：

optim = torch.optim.SGD(model.convL2.parameters(), lr=0.1, momentum=0.9)
# Now optimizer bypass parameters from convL1

如果您建模的图层更多，则必须将参数转换为列表：

params_to_update = list(model.convL2.parameters()) + list(model.convL3.parameters())
optim = torch.optim.SGD(params_to_update, lr=0.1, momentum=0.9)

如此处所述：https://discuss.pytorch.org/t/giving-multiple-parameters-in-optimizer/869

Answer 4

将您的模型定义修改为：

import torch.nn as nn

class Net(nn.Module):
    def __init__(self, weights_fixed, weights_guess):
        super(Net, self).__init__()
        self.convL1 = nn.Conv1d(1, 3, 3, bias=False)
        self.convL1.weight = weights_fixed # I want to keep these weights fixed
        self.convL1.requires_grad = False

        self.convL2 = nn.Conv1d(3, 1, 1, bias=False)
        self.convL1.weight = weights_guess # I want to learn these weights

    def forward(self, inp_batch):
        out1 = self.convL1(inp_batch)
        out2 = self.convL2(out1)

        return out2

PyTorch指定模型参数

4 个答案: