Speeding up TensorFlow Cifar10 Example for Experimentation

Question

The TensorFlow tutorial for using CNN for the cifar10 data set has the following advice:

EXERCISE: When experimenting, it is sometimes annoying that the first training step can take so long. Try decreasing the number of images that initially fill up the queue. Search for NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN in cifar10.py.

In order to play around with it, I tried decreasing this number by a lot but it doesn't seem to change the training time. Is there anything I can do? I tried even changing it to something as low as 5 and the training session still continued very slowly. Any help would be appreciated!

Answer 1

Note that this exercise only speeds up the first step time by skipping the prefetching of a larger from of the data. This exercise does not speed up the overall training

That said, the tutorial text needs to be updated. It should read

Search for min_fraction_of_examples_in_queue in cifar10_input.py.

If you lower this number, the first step should be much quicker because the model will not attempt to prefetch the input.