The way I see it:
application of Hamilton's stationary action is a two stage process:
- Use stationary action to obtain an equation of motion
- Use the equation of motion and initial conditions to obtain a trajectory
For Hamilton's stationary action the case of harmonic oscillation is especially interesting.
As we know: in the case of idealized harmonic oscillation: the period of the oscillation is independent of the amplitude.
We'll take the midpoint of the oscillation as the center of the coordinate system, and we will express the potential energy with respect to that zero point.
Evaluate Hamilton's action from one crossing of the midpoint to the next crossing of the midpoint. That is: Hamilton's action is evaluated for half a period of the oscillation. (Any integer multiple of the half-period gives the same result.)
$$ E_k = \tfrac{1}{2}mv^2 $$
With $F=-kx$ as the force that sustains the oscillation the potential increases with the square of the displacement.
$$ E_p = \tfrac{1}{2}kx^2 $$
The kinetic energy:
When the amplitude of the oscillation is doubled then the integral of the kinetic energy quadruples.
The potential energy:
When the amplitude of the oscillation is doubled then the integral of the potential energy quadruples.
The case of harmonic oscillation is unique in the following way: it is the one case where the potential energy is described with the same type of function as the kinetic energy: quadratic. It is the squaring operation that counts.
Kinetic and potential function of a different variable, $\tfrac{dx}{dt}$ and $x$ respectively, but differentiation is a linear operation, so any multiplication factor is just propagated. Hence: in the case of harmonic oscillation kinetic and potential have identical response to variation.
Counterphase
If you graph the kinetic energy and potential energy (as a function of time) then those two graphs are in counterphase, they are a mirror of each other. In the integration the mirroring makes the counterphase cancel out.
That is why it is interesting to evaluate Hamilton's action from one crossing of the midpoint to the next crossing: that is the one setup such that the counterphase cancels out.
Variation of amplitude
We define the amplitude of the oscillation as the variation space.
For the setup described above:
For every amplitude of the oscillation the value of Hamilton's action is the same: zero.
That is, if you graph the value of Hamilton's action as a function of the variation (variation of oscillation amplitude) then there is neither a minimum nor a maximum, because the graph is a flat line; zero for every value of the applied variation.
We have: for the setup described above: Hamilton's stationary action doesn't narrow down the possibilities to a single solution.
In order to narrow down to a single solution an initial condition must be provided. For example, you can treat it as a case where the oscillation is started by releasing from a particular displacement.
My point is:
What you get from Hamilton's stationary action is the equation of motion.
You then use the equation of motion and initial conditions to obtain a trajectory.
In your question you appear to describe attempts to obtain a trajectory directly from Hamilton's stationary action, and you describe having to weed out unphysical results.
I'm reminded of what happens when in the course of solving an equation you square both sides. That operation introduces a spurious solution, so at the end you must check both solutions against the original equation.
My thinking is that the necessity to weed out unphysical trajectories is related to that.