Cops in the US aren't powerless if they didn't catch you in the act of a violent crime. They may detain you on reasonable suspicion that you committed, are committing, or are about to commit a crime; powers of arrest vary a bit by state, but in general they may arrest pursuant to an arrest warrant, or for any offense committed in their presence, or when they have probable cause to believe that a felony was committed and the person they're arresting committed it.
If a cop doesn't have reasonable suspicion based on articulable facts (not a hunch, not "he's in this neighborhood while black," but actual facts that give rise to the reasonable suspicion), he cannot legally seize you and prevent you from leaving. However, the question of reasonable suspicion is not one he needs to justify to the person he's detaining; it's something which is meant to be challenged in the courts, and attempting to judge it for yourself on the street will result in you becoming familiar with some of the tools cops have on their belts. A cop doesn't strictly need to answer "am I being detained," but if they say "yes" then there's really nothing you can do about it.
Because cops can't stop you without reasonable suspicion, they can't force you to identify yourself without it. With it, some states let them force you to identify yourself (and all allow them to do so while you're driving). Cops can never force you to answer questions that might incriminate you; if a cop asks you if you've been drinking, you have the right to refuse to say "yes" or "no." You can't legally be punished for exercising your rights; that's not to say you won't be (if a cop gets angry at you and tases you, that's not legal, but you still got tased), but it's not legal to be.
If the person refuses to answer questions, and the cop doesn't have grounds to do anything more, the cop has two excellent legal options: first, the cop can continue to ask questions for a bit (although detentions on reasonable suspicion can't go on too long), and second the cop can let you go.
