The speed of sound does indeed depend on the amplitude/loudness/excitation force. However, for sufficiently weak waves, and if we ignore a variety of additional effects (such as drag and internal friction), we can approximate the speed of sound as constant. The intuitive reason is that any smooth minimum in a curve looks like a parabola up close. I'll explain what I mean as follows.
Some background: A propagating compressive wave induces harmonic motion in the molecules of the constituent material. The key characteristics are (1) a restoring force and (2) inertia; without these, we don't have a wave. Now, if we just imagine as a simple model a weight with mass $m$ hanging from an ideal spring with spring constant $k$, then the natural frequency $\omega=\sqrt{k/m}$. Regardless of how hard we strike the weight, it'll oscillate with that same natural frequency because a larger strike increases the speed but also increases the opposing force as the mass displaces farther from its equilibrium position. The two increases scale up exactly equally in the ideal case.
A sound wave moving through a material operates in a similar way, although the nature of the restoring force varies across material classes (entropic for gases, enthalpic for condensed matter). There's a slight change from an ideal frequency of $\omega=\sqrt{k/m}$ to an ideal speed of $v=\sqrt{E/\rho}$, where $E$ is an elastic modulus and $\rho$ is the density, but the general form is the same: a stiffness term divided by an inertial term.
At this point, we have to examine the constraints of this ideal case: $k$ and $m$ (or $E$ and $\rho$) remaining constant. We don't expect the mass to vary for a uniform material moving at nonrelativistic speeds, so that takes care of inertial variation. How about stiffness variation that would tend to alter $v$?
Broadly, a material's stiffness arises from its free energy increasing as we deform it. In condensed matter, we can model this deflection–energy relationship using the pair potential, where the minimum energy corresponds to the equilibrium spacing between molecules.
The minimum energy region will generally have some type of asymmetric shape. But by Taylor series expansion, we can show that all small minima look like parabolas up close. A parabolic energy profile is the characteristic of an ideal spring with constant spring constant $k$: with increasing deformation, the restoring force scales up linearly, and the energy scales up quadratically. For this reason, most stiff solids can be characterized by a single Young's modulus for uniaxial loading, regardless of magnitude and sign of the load.
Therefore, assumptions of a constant speed of sound rely on small disturbances and an idealized simple model—much like the assumptions of symmetric linear elasticity. Alternatively, we'd say that acoustic wave dispersion is zero for small perturbations. This is usually a pretty good model, but it is a simplified one.
