7

I am training a convolutional neural network for object detection. Apart from the learning rate, what are the other hyperparameters that I should tune? And in what order of importance? Besides, I read that doing a grid search for hyperparameters is not the best way to go about training and that random search is better in this case. Is random search really that good?

hanugm
  • 4,102
  • 3
  • 29
  • 63
S.E.K.
  • 41
  • 1
  • 5

3 Answers3

1

It depends a lot on what type of architecture you are using. However, most of the standard architectures are quite stable and there is no need for much hypreparameter tuning.

  1. Choose whether you want to use a standard CNN architecture or a more advanced one like a ResNet.
  2. Size? How many conv layers will you be using.
  3. Number of filters in each of the layers. This is where you might need a bit of tuning.
  4. You can try tuning the other parameters that are not specific to the problem, e.g. learning rate, regularization, dropout. If you already have a lot of feel for what works good and what not, then grid search according to your intuition would probably work pretty well. No need for random search in this case.
pi-tau
  • 995
  • 6
  • 12
0

Firstly when you say an object detection CNN, there are a huge number of model architectures available. Considering that you have narrowed down on your model architecture a CNN will have a few common layers like the ones below with hyperparameters you can tweak:

  1. Convolution Layer:- number of kernels, kernel size, stride length, padding
  2. MaxPooling Layer:- kernel size, stride length, padding
  3. Dense Layer:- size
  4. Dropout:- Percentage to keep/drop
keshav
  • 56
  • 3
0

You can play around with batch Size, epochs, learning rate.

Start with a batch size of 8 and gradually increase two folds => 8,16,32

Start with a higher learning rate to find the direction towards global minima, then reduce learning rate to avoid bouncing around and reach local minima

Start with a low number of epochs to get quick feedback on the rest of hyperparameters. Once you have the optimal direction for hyperparameters move to higher number of epochs

Marib Sultan
  • 76
  • 3