tensorflow tokenizer没有此类文件或目录

Question

我已将“数据框”列文本转换为

ShuffleDataset shapes: ((), ()), types: (tf.string, tf.int64)

当我将其标记化时，出现以下错误

import tensorflow_datasets as tfds
tokenizer = tfds.features.text.Tokenizer()

vocabulary_set = set()
for text_tensor ,_ in all_labeled_data:
  some_tokens = tokenizer.tokenize(all_labeled_data.numpy())
  vocabulary_set.update(some_tokens)

vocab_size = len(vocabulary_set)
vocab_size

错误正在显示要被标记化的文本，被标记化工具标记为文件目录

     66    Waits for ops dispatched in this executor to > finish.
---> 67     > pywrap_tensorflow.TFE_ExecutorWaitForAllPendingNodes(self._handle)
     68 
     69   def clear_error(self):

NotFoundError: RE: VyprVPN for Business; No such file or directory