Question

我有一些文件包含图像文件路径和功能，有些图像可能丢失或损坏。我想知道如何通过跳过这些图像并将其从队列中删除来稳健地处理错误。

我注意到只是捕获错误并继续将导致队列输出相同的图像，因此它将在同一图像上反复出错。有没有办法在出错时将图像出列？

另外，我有一个＆＃39; tf.Print（）＆＃39;用于记录文件名的语句，但结果是：＆＃39;我的日志中的行显示处理的有效图像没有相应的打印输出。为什么＆＃39; tf.Print（）＆＃39;只打印不存在的文件的名称，而不是正确处理的文件？

下面是一个小例子，与我的大型程序具有相同的错误处理代码：

代码：

#!/usr/bin/python3

import tensorflow as tf

example_filename = 'example.csv'
max_iterations = 20

### Create the graph ###
filename_container_queue = tf.train.string_input_producer([ example_filename ])
filename_container_reader = tf.TextLineReader()

_, filename_container_contents = filename_container_reader.read(filename_container_queue)
image_filenames = tf.decode_csv(filename_container_contents, [ tf.constant('', shape=[1], dtype=tf.string) ])

# decode_jpeg only works on a single image at a time
image_filename_batch = tf.train.shuffle_batch([ image_filenames ], batch_size=1, capacity=100, min_after_dequeue=0)
image_filename = tf.reshape(image_filename_batch, [1])

image_filenames_queue = tf.train.string_input_producer(image_filename)
image_reader = tf.WholeFileReader()
_, image_contents = image_reader.read(image_filenames_queue)
image = tf.image.decode_jpeg(tf.Print(image_contents, [ image_filename ]), channels=3)

counter = tf.count_up_to(tf.Variable(tf.constant(0)), max_iterations)

result_op = tf.reduce_mean(tf.image.convert_image_dtype(image, tf.float32), [0,1]) # Output average Red, Green, Blue values.

init_op = tf.initialize_all_variables()

### Run the graph ###
print("Running graph")
with tf.Session() as sess:
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)
    sess.run([ init_op ])
    n = 0
    try:
        while not coord.should_stop():
            try:
                result, n = sess.run([ result_op, counter ])
                print("Result:", result)
            except tf.errors.NotFoundError as e:
                print("Skipping file due to image not existing")
                # coord.request_stop(e) <--- We only want to skip, not stop the entire process.
    except tf.errors.OutOfRangeError as e:
        print('Done training -- epoch limit reached after %d iterations' % n)
        coord.request_stop(e)
    finally:
        coord.request_stop()
        coord.join(threads)

数据：

example.csv包含：

/home/mburge/Pictures/junk/109798.jpg
nonexistent.jpg

节目输出：

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
Running graph
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:925] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning N
UMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties: 
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.8475
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 6.83GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
I tensorflow/core/kernels/logging_ops.cc:79] [nonexistent.jpg]
Result: [ 0.33875707  0.39879724  0.28882763]
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
W tensorflow/core/framework/op_kernel.cc:968] Not found: nonexistent.jpg
         [[Node: ReaderRead_1 = ReaderRead[_class=["loc:@WholeFileReader", "loc:@input_producer_1"], _device="/job:localhost/replica:0/task:0/cpu:0"](WholeFileReader, input_produ
cer_1)]]
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Skipping file due to image not existing
Done training -- epoch limit reached after 0 iterations

Answer 1

您可以手动定义出队操作：

filename_deq = image_filenames_queue.dequeue()

以后，如果您发现读取文件有问题，请从文件名队列中取出该文件：

except tf.errors.NotFoundError as e:
    print("Skipping file due to image not existing")
    sess.run(filename_deq)

在Tensorflow中删除不存在或损坏的文件

1 个答案: