No, you cannot just call the police and throw him out.
Here are two concepts which sound a bit abstract: "who does the property belong to" and "who can use this property".
Let's say I rent an apartment. One night, the landlord knocked on the door and asked to stay for the night. I have every right to kick him out. Why? This is my home. This area is for my private residential use. I certainly has ultimate decision power on what activities can happen inside this area.
But the landlord owns the place. He can sell, transfer, or otherwise stop renting the place to me, provided that a reasonable time is given to notify the occupant. He cannot, out of no where, show up next morning and tell me I have 30 minutes to get all my stuff out of the apartment. That is just utterly unfair.
Does he need to physically hand over the key?
Well, not really. He needs to return the place to its original condition. He needs to remove all stuff brought by him. If he drilled a hole in a wall, he has to patch it. If he removed a cabinet, he need to put it back. Finally he needs to give you access. Physically handing over the key is one of many ways.
Ideally I would add a clause to the contract so a tenant waives this right when this kind of breach occurs.
Nope. You cannot. Even if you add this clause, the court would likely consider this clause unenforceable (but does not void the rest of the contract!). Such clause is to bias towards the landlord. You must give the occupant a reasonable time to respond to your request of asking him to move out.
Let's suppose:
I have already sent several notices, demanding him to move out, but
after 3 months, he is still occupying the property
If that is the case, you can take him to the court. He has no legal ground to occupy and continue to use the property which he does not belong.
A reasonable time of a move out notice is usually 30 days, here in HK.