I read the book, and I'm willing to believe you'd have a good chance of beating the market with this strategy - it is a reasonable, rational, and mechanical investment discipline.
I doubt it's overplayed and overused to the point that it won't ever work again.
But only IF you stick to it, and doing so would be very hard (behaviorally). Which is probably why it isn't overplayed and overused already. This strategy makes you place trades in companies you often won't have heard of, with volatile prices.
The best way to use the strategy would be to try to get it automated somehow and avoid looking at the individual stocks, I bet, to take your behavior out of it.
There may well be some risk factors in this strategy that you don't have in an S&P 500 fund, and those could explain some of the higher returns; for example, a basket of sketchier companies could be more vulnerable to economic events.
The strategy won't beat the market every year, either, so that can test your behavior.
Strategies tend to work and then stop working (as the book even mentions). This is related to whether other investors are piling in to the strategy and pushing up prices, in part.
But also, outside events can just happen to line up poorly for a given strategy; for example a bunch of the "fundamental index" ETFs that looked at dividend yield launched right before all the high-dividend financials cratered. Investing in high-dividend stocks probably is and was a reasonable strategy in general, but it wasn't a great strategy for a couple years there. Anytime you don't buy the whole market, you risk both positive and negative deviations from it.
Here's maybe a bigger-picture point, though. I happen to think "beating the market" is a big old distraction for individual investors; what you really want is predictable, adequate returns, who cares if the market returns 20% as long as your returns are adequate, and who cares if you beat the market by 5% if the market cratered 40%.
So I'm not a huge fan of investment books that are structured around the topic of beating the market. Whether it's index fund advocates saying "you can't beat the market so buy the index" or Greenblatt saying "here's how to beat the market with this strategy," it's still all about beating the market. And to me, beating the market is just irrelevant. Nobody ever bought their food in retirement because they did or did not beat the market.
To me, beating the market is a game for the kind of actively-managed mutual fund that has a 90%-plus R-squared correlation with the index; often called an "index hugger," these funds are just trying to eke out a little bit better result than the market, and often get a little bit worse result, and overall are a lot of effort with no purpose. Just get the index fund rather than these.
If you're getting active management involved, I'd rather see a big deviation from the index, and I'd like that deviation to be related to risk control: hedging, or pulling back to cash when valuations get rich, or avoiding companies without a "moat" and margin of safety, or whatever kind of risk control, but something. In a fund like this, you aren't trying to beat the market, you're trying to increase the chances of adequate returns - you're optimizing for predictability.
I'm not sure the magic formula is the best way to do that, focused as it is on beating the market rather than on risk control.
Sorry for the extra digression but I hope I answered the question a bit, too. ;-)
