Question

我正在按照BERT指令进行微调，如here

所述

这是我的代码：

from sentence_transformers import SentenceTransformer, SentencesDataset, InputExample, losses, evaluation
from torch.utils.data import DataLoader

# load model
embedder = SentenceTransformer('bert-large-nli-mean-tokens')
print("embedder loaded...")

# define your train dataset, the dataloader, and the train loss
train_dataset = SentencesDataset(x_sample["input"].tolist(), embedder)
train_dataloader = DataLoader(train_dataset, shuffle=False, batch_size=16)
train_loss = losses.CosineSimilarityLoss(embedder)

sentences1 = ['This list contains the first column', 'With your sentences', 'You want your model to evaluate on']
sentences2 = ['Sentences contains the other column', 'The evaluator matches sentences1[i] with sentences2[i]', 'Compute the cosine similarity and compares it to scores[i]']
scores = [0.3, 0.6, 0.2]
evaluator = evaluation.EmbeddingSimilarityEvaluator(sentences1, sentences2, scores)

# tune the model
embedder.fit(train_objectives=[(train_dataloader, train_loss)], 
    epochs=1, 
    warmup_steps=100, 
    evaluator=evaluator, 
    evaluation_steps=1)

在4％的情况下，培训停止并且程序存在，没有警告或错误。没有输出。

我不知道如何进行故障排除-任何帮助都会很棒。

编辑：将标题从失败更改为停止/退出，因为我不知道它是否失败

这是我在终端上看到的内容：时期：0％| 被杀：0％|

“杀死”一词与“迭代”一词重叠...也许是内存问题？仅供参考：我是在Windows的ubuntu vm上使用wsl在vscode的终端上运行它的

在github上发现了问题： https://github.com/ElderResearch/gpu_docker/issues/38

Answer 1

我的解决方案是将批处理和工作程序设置为一个，而且速度很慢

HttpWebRequest.AllowWriteStreamBuffering = false

Bert句子变形器在微调期间停止/退出

1 个答案: