Question

如何使用pytorch从所有隐藏状态中提取第一步？

我正在做BERT，下面是一些代码：

    def predict(self, x1, x2=None):
        assert x2 is None
        #TODO: x1 is a batch of sequence pairs. 
        #TODO: BERT (and DistillBERT) have been trained such that
        #TODO: the ouput at the first time-step can be used for sentence similarity classification.
        #TODO: Pass the x1 tensor to the `self.bert_model`
        #TODO: Note the Bert model will return a tuple, you only need the first item (which are the hidden states from the last layer of Bert) in the tuple for this task.
        #TODO: documentation for the Bert model can be found here: https://huggingface.co/transformers/model_doc/bert.html
        #TODO: the result should have shape (batch_size, seq_size, 768)
        this_bert = self.bert_model(x1)

        #TODO: Extract the first time step hidden state from all the hidden states.
        #TODO: Pass the first time step hidden state through the `self.output` linear layer
        #TODO: Pass the output of the linear layer through a sigmoid function and name the result `out`.
        #TODO: `out` should have the shape (batch_size, 1)
        first_step = this_bert[0]
        so = self.output(first_step) 
        out = torch.nn.Sigmoid(so).view(batch_size, 1)

错误消息：TypeError： init （）接受1个位置参数，但给出了2个

如何从PyTorch中的所有隐藏状态中提取第一步

0 个答案: