RandomShuffleQueue' _2_shuffle_batch / random_shuffle_queue'已关闭且元素不足(请求2,当前大小为0)

时间:2018-06-03 12:42:06

标签: tensorflow tensorflow-datasets

我有一个tfrecord格式的数据集,我试图回读并显示图像以调试数据集问题。但是我在阅读时遇到了麻烦,因为事情看起来与tf.train.shuffle_batch()有关。

The error returned is:
     [[Node: DecodeRaw = DecodeRaw[little_endian=true, out_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](ParseSingleExample/Squeeze_data/raw_image)]]
Traceback (most recent call last):
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1323, in _do_call
    return fn(*args)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1302, in _run_fn
    status, run_metadata)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 2, current size 0)
     [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Workspace\eclipse\satnav_turn_classifier\datasetutils\play_tfrecord.py", line 53, in <module>
    img, lbl = sess.run([images, labels])
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 889, in run
    run_metadata_ptr)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1317, in _do_run
    options, run_metadata)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 2, current size 0)
     [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Caused by op 'shuffle_batch', defined at:
  File "D:\Workspace\eclipse\satnav_turn_classifier\datasetutils\play_tfrecord.py", line 43, in <module>
    images, labels = tf.train.shuffle_batch([image, label], batch_size=2, capacity=20, num_threads=1, min_after_dequeue=10)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\training\input.py", line 1225, in shuffle_batch
    name=name)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\training\input.py", line 796, in _shuffle_batch
    dequeued = queue.dequeue_many(batch_size, name=name)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\ops\data_flow_ops.py", line 464, in dequeue_many
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 2417, in _queue_dequeue_many_v2
    component_types=component_types, timeout_ms=timeout_ms, name=name)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\framework\ops.py", line 2956, in create_op
    op_def=op_def)
  File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\framework\ops.py", line 1470, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 2, current size 0)
     [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

创建文件的代码包括:

writer = tf.python_io.TFRecordWriter(tfrecord_file)
for i in range(len(addresses)):    

    image_file = addresses[i]
    fp = open(image_file, 'rb')
    raw_image = fp.read()
    fp.close()         

    image = coder.decode(raw_image)
    height, width, _ = image.shape


    label = labels[i]
    if 2.0 > float(label) > -2.0:

        # Create a feature
        feature = {'data/label': _float_feature(label),
                   'data/raw_image': _bytes_feature(raw_image),
                   'data/height': _int64_feature(height),
                   'data/width': _int64_feature(width)}

        # Create an example protocol buffer
        example = tf.train.Example(features=tf.train.Features(feature=feature))

        # Serialize to string and write on the file
        try:
            writer.write(example.SerializeToString())
        except:
            print('Error with csv line %d, Steering Angle float value %f' % (i, label))


    # print how many images are saved every 128 images
    if not i % 128:
        print('Converted data: {}/{}'.format(i, len(addresses)))
        sys.stdout.flush()

writer.close()
sys.stdout.flush()  

读者代码是:

tfrecords_path = 'd:/datasets/gtav/_tfrecord_files/cs003.trn.tfrecords'
with tf.Session() as sess:
    feature = {'data/label': tf.FixedLenFeature([], tf.float32),
               'data/raw_image': tf.FixedLenFeature([], tf.string),
               'data/height': tf.FixedLenFeature([], tf.int64),
               'data/width': tf.FixedLenFeature([], tf.int64)}

    # Create a list of filenames and pass it to a queue
    filename_queue = tf.train.string_input_producer([tfrecords_path], num_epochs=1)
    # Define a reader and read the next record
    reader = tf.TFRecordReader()
    _, serialized_example = reader.read(filename_queue)
    # Decode the record read by the reader
    features = tf.parse_single_example(serialized_example, features=feature)
    # Convert the image data from string back to the numbers
    image = tf.decode_raw(features['data/raw_image'], tf.float32)


    # Cast label data into float32
    label = tf.cast(features['data/label'], tf.float32)
    # Reshape image data into the original shape
    image = tf.reshape(image, [160, 320, 3])
    #image_shape = tf.shape(image);

    # Any preprocessing here ...

    # Creates batches by randomly shuffling tensors
    images, labels = tf.train.shuffle_batch([image, label], batch_size=2, capacity=20, num_threads=1, min_after_dequeue=10)

    # Initialize all global and local variables
    init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
    #sess.run(image_shape)
    sess.run(init_op)
    # Create a coordinator and run all QueueRunner objects
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)
    for batch_index in range(5):
        img, lbl = sess.run([images, labels])
        img = img.astype(np.uint8)
        for j in range(6):
            plt.subplot(2, 3, j+1)
            plt.imshow(img[j, ...])
            plt.title('cat' if lbl[j]==0 else 'dog')
        plt.show()
    # Stop the threads
    coord.request_stop()

    # Wait for threads to stop
    coord.join(threads)
    sess.close()

关于随机shuffle队列错误的任何想法? 我不确定我做错了什么。

0 个答案:

没有答案