GTA5包含24966张图像。将它们转换为一个tfrecord文件后，大约需要50G。
当我读取这个大文件时，似乎无法完成一个纪元。我的意思是它确实输出了部分数据。但是在阅读过程中，它将引发以下错误。 它从一开始就读取一些数据。但是在中间阶段，如经过948步之后，它将引发错误。我怀疑是否是因为我的tfrecord文件太大？
有人可以给我一些建议吗？ 预先谢谢你。
Step 946
(8, 512, 1024, 3) (8, 512, 1024, 1)
Step 947
(8, 512, 1024, 3) (8, 512, 1024, 1)
Step 948
(8, 512, 1024, 3) (8, 512, 1024, 1)
Traceback (most recent call last):
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1323, in _do_call
return fn(*args)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1302, in _run_fn
status, run_metadata)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 8, current size 3)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_UINT8], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:/PycharmProjects/adversarial_gradient/datasets/GTA5/convert_gta5.py", line 187, in <module>
l = sess.run(label_batch)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\client\session.py", line 889, in run
run_metadata_ptr)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1317, in _do_run
options, run_metadata)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 8, current size 3)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_UINT8], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
Caused by op 'shuffle_batch', defined at:
File "D:/PycharmProjects/adversarial_gradient/datasets/GTA5/convert_gta5.py", line 163, in <module>
img_batch, label_batch = inputs(params, shuffle=True)
File "D:/PycharmProjects/adversarial_gradient/datasets/GTA5/convert_gta5.py", line 148, in inputs
return generate_batch([img, label], params, shuffle, num_examples)
File "D:/PycharmProjects/adversarial_gradient/datasets/GTA5/convert_gta5.py", line 121, in generate_batch
min_after_dequeue=min_queue_examples)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\training\input.py", line 1225, in shuffle_batch
name=name)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\training\input.py", line 796, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\ops\data_flow_ops.py", line 464, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 2417, in _queue_dequeue_many_v2
component_types=component_types, timeout_ms=timeout_ms, name=name)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 2956, in create_op
op_def=op_def)
File "D:\Program Files\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 1470, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access