There must be a expansion tank connected to a radiator. If it is full of cooling liquid, it will force the water through the cap.
The reason for expansion tank is to have a space where expanded water runs and compresses an air in the tank. If the pressure is too high, the cap will release a pressure by letting the liquid out. Why the pressure can be too high - the liquid is non-compressable, so if the level in the expansion tank is too high, it means there is not enough space for the air which to compress. If your expansion tank is empty, the cooling system can grab some air from it when cools down, which will stop a circulation of a cooling liquid on a cold start. Nothing wrong until you rev up and blow a cold water in hot engine. That will decrease a lifetime of an engine. So the liquid level in a expansion tank should be about half full.
If it is not the issue, make sure your cooling temperature doesn't exceed 100°C. Normally it should be around 90°C. If the engine warms up quickly, your issue might be water pump....