拥抱面部变压器中用于文本生成(XLNet,GPT-2)的白名单令牌

时间:2020-09-16 13:15:38

标签: python-3.x nlp pytorch huggingface-transformers

在有关文本生成(https://huggingface.co/transformers/main_classes/model.html#generative-models)的文档中,可以选择放置

bad_words_ids (List[int], optional) – List of token ids that are not allowed to be generated. In order to get the tokens of the words that should not appear in the generated text, use tokenizer.encode(bad_word, add_prefix_space=True).

是否还可以选择在“ allowed_words_ids”中添加一些内容?这个想法是限制所生成文本的语言。

0 个答案:

没有答案