我有一堆图像,每个图像都有一个句子。
我想创建一个具有图像句对的数据集。
我做了什么:
def npy_to_tfrecords(numpy_array, text_file, output_file):
f = open(text_file)
# write records to a tfrecords file
writer = tf.python_io.TFRecordWriter(output_file)
# Loop through all the features you want to write
for X, line in zip(numpy_array, f) :
#let say X is of np.array([[...][...]])
#let say y is of np.array[[0/1]]
txt = "{}".format(line[:-1])
txt = txt.encode()
# Feature contains a map of string to feature proto objects
feature = {}
feature['x'] = tf.train.Feature(float_list=tf.train.FloatList(value=X.flatten()))
feature['y'] = tf.train.Feature(bytes_list=tf.train.BytesList(value=[txt]))
# Construct the Example proto object
example = tf.train.Example(features=tf.train.Features(feature=feature))
# Serialize the example to a string
serialized = example.SerializeToString()
# write the serialized objec to the disk
writer.write(serialized)
writer.close()
现在,我想将其视为数据集并使用它:
但是,
def read_tfr_file(filename):
dataset = tf.data.TFRecordDataset(filename)
# for version 1.5 and above use tf.data.TFRecordDataset
# example proto decode
def _parse_function(example_proto):
keys_to_features = {'x':tf.FixedLenFeature([], tf.float32),
'y': tf.FixedLenSequenceFeature([], tf.string, allow_missing=True)}
parsed_features = tf.parse_single_example(example_proto, keys_to_features)
return parsed_features['x'], parsed_features['y']
# Parse the record into tensors.
dataset = dataset.map(_parse_function)
return dataset
我总是会收到maperror或通过这种方式读取记录无法访问对象,如何解决此问题?
如何正确创建固定长度图像特征和可变长度句子的数据集?