PyTorch：使用numpy数组初始化权重+创建一个常数张量

Question

我有以下代码：

self.wi = nn.Embedding(num_embeddings, embedding_dim)
self.wj = nn.Embedding(num_embeddings1, embedding_dim)
self.bi = nn.Embedding(num_embeddings, 1)
self.bj = nn.Embedding(num_embeddings1, 1)

self.wi.weight.data.uniform_(-1, 1)
self.wj.weight.data.uniform_(-1, 1)
self.bi.weight.data.zero_()
self.bj.weight.data.zero_()

我想用numpy数组初始化权重，我想创建一个常数张量，它也是一个numpy数组。 我是PyTorch的新手，感谢您的帮助。

Answer 1

您可能希望将numpy数组转换为火炬张量How to convert Pytorch autograd.Variable to Numpy?

Answer 2

您可以使用函数nn.Embedding.from_pretrained()初始化嵌入层。

在您的特定情况下，您仍然必须首先将numpy.array转换为torch.Tensor，但是否则非常简单：

import torch as t
import torch.nn as nn
import numpy as np

# This can be whatever initialization you want to have
init_array = np.zeros([num_embeddings, embedding_dims])

# As @Daniel Marchand mentioned in his answer, 
# you do have to cast it explicitly as a tensor, otherwise it won't work.
wi = nn.Embedding.from_pretrained(t.tensor(init_array), freeze=False)

如果以后仍要训练网络，则参数freeze=False很重要，否则您将使嵌入保持相同的恒定值。 通常，.from_pretrained用于“转移”学习到的嵌入，但它也适用于您的情况。