The point of the enrollment penalty is basically the same as the ACA penalty. Any sort of health insurance - or really, any insurance - is funded by creating a risk pool of high and low risk people and pricing it so that the overall payments cover the total risk.
That means, however, that on average the low risk people end up paying more than their share - more than it would have cost them, without the insurance, excepting any provider agreements to charge less (which is significant in the health insurance business). (Of course some of them do end up using more than they pay - but not on average, assuming the risk was calculated accurately.)
While there isn't really a completely low risk pool in Medicare, there is a significant difference in utilization (=cost) between younger (65-70) and older enrollees. As such, for many health 65 year olds, it would be beneficial to not enroll in Medicare right away - delay a few years, if they're fully healthy, and wait until they are less healthy. Since Medicare won't turn you away for pre-existing conditions, that's a risk some would take.
In order to accommodate for that, Medicare effectively says, "If you didn't help subsidize the costs of the high users when you were younger, you need to pay more to make up for that fact" - hence the enrollment penalty.
The New York Times explains this in part in a 2006 article discussing Part D (which was new that year, and has a similar penalty):
The purpose of the late enrollment penalty is to encourage people to
sign up as soon as possible, before they have significant drug costs.
