I have tfrecords in s3 and I want to directly read those files and feed them into tensorflow training.
filename = "s3location"
filename_queue = tf.train.string_input_producer([filename])
with tf.Session() as sess:
_, value = tf.TFRecordReader().read(filename_queue)
feature = {'train/image': tf.FixedLenFeature([], tf.string),
'train/label': tf.FixedLenFeature([], tf.int64)}
features = tf.parse_single_example(value, features=feature)
image = tf.decode_raw(features['train/image'], tf.float32)
label = tf.cast(features['train/label'], tf.int32)
image = tf.reshape(image, [32, 32, 1])
images, labels = tf.train.shuffle_batch([image, label], batch_size=50, capacity=100, num_threads=1, min_after_dequeue=10)
But when I am trying to feed this sess.run(train, feed_dict={x:images, y: labels, hold_prob: 0.5})
I get the following error -
TypeError: The value of a feed cannot be a tf.Tensor object. Acceptable feed values include Python scalars, strings, lists, numpy ndarrays, or TensorHandles.
I have 2 questions -
Is there a way to read data directly from s3 in tensorflow. looks like the support was added,但我找不到任何示例。
如果支持,我的方法是否正确?