Question

我尝试加载经过预先训练的XLNet，但是这种情况发生了。我以前曾尝试过此方法，但它一直有效，但是现在没有了。关于如何解决此问题有什么建议吗？

model = XLNetForSequenceClassification.from_pretrained("xlnet-large-cased", num_labels = 2)
model.to(device)

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-55-d6f698a3714b> in <module>()
----> 1 model = XLNetForSequenceClassification.from_pretrained("xlnet-large-cased", num_labels = 2)
      2 model.to(device)

3 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/sparse.py in __init__(self, num_embeddings, embedding_dim, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse, _weight)
     95         self.scale_grad_by_freq = scale_grad_by_freq
     96         if _weight is None:
---> 97             self.weight = Parameter(torch.Tensor(num_embeddings, embedding_dim))
     98             self.reset_parameters()
     99         else:

RuntimeError: Trying to create tensor with negative dimension -1: [-1, 1024]

Answer 1

您应该从 XLNetForSequenceClassification 而不是从 pytorch-transformers 导入 transformers。首先，确保安装了变压器：

> pip install transformers

然后，在您的代码中：

from transformers import XLNetForSequenceClassification
model = XLNetForSequenceClassification.from_pretrained("xlnet-large-cased", num_labels = 2)

这应该有效。

Answer 2

如果您未在内部进行任何更改，则很可能是版本不匹配。您是否升级了任何相关模块？如果有，请返回以前的版本。

Pytorch Quantization RuntimeError: Trying to create tensor with negative dimension

XLNetForSequenceClassification预训练模型无法加载

2 个答案: