Tensorflow数据集-令牌化-'SparseTensor'对象没有属性'name'

时间:2019-05-22 21:44:58

标签: tensorflow python-3.7 tensorflow-transform

我正在使用Tensorflow 1.13,并且收到以下错误:

Traceback (most recent call last):
  File "train.py", line 188, in <module>
    history = model.fit(train_dataset, epochs=epochs, steps_per_epoch=steps_per_epoch)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 880, in fit
  validation_steps=validation_steps)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 266, in model_iteration
batch_outs = f(actual_inputs)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 3073, in __call__
self._make_callable(feed_arrays, feed_symbols, symbol_vals, session)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 3008, in _make_callable
connection.from_tensor = from_tensor.name  # Data tensor
AttributeError: 'SparseTensor' object has no attribute 'name'

使用DataSet API预处理.map时。这是我第一次尝试使用tf.data.Dataset.map API,并且尝试将我的过程从使用tf.keras.preprocessing.text.Tokenizer API转换为使用tensorflow_transform。我看到的唯一示例是针对sentiment analysis的示例,以下是我的逻辑(为简洁起见被截断): 将tensorflow作为tf导入 将tensorflow_transform导入为tft

class Tokenizer(object):

    def __init__(self, label_table):
        self.label_table = label_table

    def __call__(self, data, labels):
        split_data = tf.string_split([data], ' ')
        tokenized_data = tft.compute_and_apply_vocabulary(split_data, name='compute_vocabulary')
        labels = self.label_table.lookup(labels, name='assign_labels')
        print(tokenized_data) # <SparseTensor ...>
        return tokenized_data, labels

if __name__ == '__main__':
    training_text = tf.constant(raw_train_data, name='raw_text')
    training_label_tensor = tf.constant(raw_train_labels, name='labels')
    train_dataset = tf.data.Dataset.from_tensor_slices((training_text, training_label_tensor))

    label_tensor = tf.constant(['Label 1', 'Label 2', 'Label n'])
    label_table = tf.contrib.lookup.index_table_from_tensor(mapping=label_tensor, num_oov_buckets=1, default_value=-1)
    data_preprocessor = Tokenizer(label_table)
    train_dataset = train_dataset.map(data_preprocessor)

我已经将我的tokenized_data变量标识为SparseTensor,但是我不知道为什么它没有name属性。我认为我的概念方法可能是错误的,但是我不知道从哪里开始修复它。有人对我的方法有指导吗?

0 个答案:

没有答案