so I got this question for my Lab: Q: Show how linear regression for classification can improve pocket algorithm with PLA.
I thought linear regression was bad for classification? So how can linear regression for classification be used to improve pocket algorithm with PLA? or simply just PLA I don't know why they specified pocket in there, maybe there's a reason I don't know.
The pocket algorithm tracks the best weights seen during Rosenblatt’s single perceptron learning algorithm (PLA) iterations in a "pocket" which is an extension of PLA and able to handle non-linearly-separable data.
The pocket algorithm with ratchet (Gallant, 1990) solves the stability problem of perceptron learning by keeping the best solution seen so far "in its pocket". The pocket algorithm then returns the solution in the pocket, rather than the last solution. It can be used also for non-separable data sets, where the aim is to find a perceptron with a small number of misclassifications
Linear regression (LR) isn’t for classification, use logistic regression instead which has no closed-form solution, but LR's closed-form solution is a computationally cheap way to bootstrap PLA or pocket algorithm variant to speed up training. LR's solution approximates the direction of the optimal separating hyperplane by minimizing the MSE between the predicted and actual class labels. Instead of starting PLA or pocket algorithm with random weights, use LR's weights as the initial heuristic weights to help PLA/pocket supervised learning algorithm's iterative delta training rule converge faster with fewer updates based upon misclassification.
