Here are some of the factors to consider before switching your cat to a live food diet.
Legality
In some jurisdictions it is illegal to feed live mice (or illegal to feed live food when the animal will eat dead food). The laws are generally written with reptiles in mind, but may apply to cats depending on how they are written.
Willingness of your cat to kill live mice
With live prey you have to consider how you will offer the mouse to the cat in a way that will prevent it from escaping into your house if your cat is not interested or does not kill the mouse right away.
Is it animal cruelty?
If you buy frozen mice, the mice have been killed quickly. If you offer your cat a live mouse you are giving it mental stimulation/enrichment at the cost of the mental wellbeing of the mouse.
Your willingness to clean the leftovers
Unlike reptiles, cats do not swallow mice whole and may leave bits for you to clean up. I've read some accounts (when researching nutitrion) that cats often leave the stomach/intestines uneaten. Are you willing to clean this up?
Health of the feeder mice
There have been some accounts of pet-store mice having parasites and other transmissable diseases because they are kept in such close contact with each other. Are you able/willing to fully vet the health of your food source?
Monitoring your cat's health
You're counting on the mice to have all of the essential nutrients that cats need (such as taurine) that you're supplimenting right now. If you plan to feed exclusively live food, you should know the signs of deficiency and make sure your cat has regular vet check ups in case the mice are deficient in some way.
My experience
My cats won't eat anything that does not look like cat food. I've experimented with food items like raw chicken necks and my cats don't seem to recognize that it's food (they think it's a toy). Before making the jump to live food I would suggest trying similar items (already dead mice, for example) to see if they recognize it as food.
