Image Comparison - Automotive Fusebox

Question

I have a task in which I need to check an automotive fusebox (see image) - The goal is to ensure that the correct amperage fuses are in their assigned slots and not swapped or missing.

Subtraction Approach: A very simplistic approach would be to have a fixture in which the fusebox is mounted, picture taken and then subtracted from a reference (training image). The drawback is that mounting the fusebox in the fixture will be time-consuming and the difference method may be too sensitive light changes.

Here is what I am thinking instead: Have yellow guiding markers overlaid on a live video feed, as shown below. Have a color matching routine running in a loop and as soon as the software detects the 'correct' colors in the boxes, it informs the user that the fusebox configuration is correct.

By 'color matching' I mean histogram comparison of the regions inside the boxes with a corresponding reference image.

My Reservations:

1. I am concerned that this approach may also be too sensitive to motion. The fuse may hover in and outside the box and the software will therefore 'dither' somewhat between "Good" and "No Good". I hope I was able to explain that clearly.
2. How sensitive would histogram matching would be to changes in light? For instance, notice the fuse in the bottom left corner - it is somewhat washed out. Is there a way to improve histogram matching that such a washout a does not affect accuracy?

What other approaches can I try to analyse the fuse box?