Can bounding boxes further improve the performance of a CNN classifier?

Question

Suppose I have a standard image classification problem (i.e. CNN is shown a single image and predicts a single classification for it). If I were to use bounding boxes to surround the target image (i.e. convert this into an object detection problem), would this increase classification accuracy purely through the use of the bounding box?

I'm curious if the neural network can be "assisted" by us when we show it bounding boxes as opposed to just showing it the entire image and letting it figure it all out by itself.

Answer 1

Information outside of the bounding box could still be useful as context. So not to lose it you can but bounding box as pixel mask into additional "pseudo-color" layer. That way you can also have many bounding boxes without changing input architecture. You will give network additional info without losing anything, so result shouldn't be worse at least.

Answer 2

Another way to ask the question is: Does sound get clearer when you remove the background noise?

The obvious answer is yes and in the case of image classification, the answer is also generally yes.

In most cases reducing the noise (irrelevant pixels) will strengthen the signal (activations) the neural network is trying to find.

Answer 3

For sure, this may be helpful. You remove excessive information from image and make the classification task a bit more simple. But you need to remember that bounding boxes may work not perfectly and accuracy of the classification algorithm may suffer from corrupted inputs (when bounding boxes are corrupted).

Answer 4

One could imagine using a segmentation network as a first step of processing. Then feeding an area corresponding to a bounding box of each segmented object to the classifier.

Potentially that could yield an increase in performance in classifying objects in an image, but not without a cost of training time, sine suddenly there are two networks to train instead of just one.