I am trying to train a model, based on the results of a weak classifier.
Suppose I have some training samples of unlabelled data, and a weak classifier, my procedure is what i call self-training, which means in the first round I train a model with all training samples, labelled by the weak classifier. Then I use the trained model to label the training samples, and train on this label (instead of the weak classifier label), then I use the newly trained model to label the training samples again, then train on this label, etc, such is the iterative self training process.
My question is theoretically, will this be helpful? What is the difference between doing this and just training for a long time for once under the label of weak classifier? Is there any theoretical analysis for or against this training scheme?
