Considering the time-independent Schrödinger equation, I can see for a finite potential, why the wavefunction has to be continuous, I can also see why the first derivative of the wavefunction is discontinuous when there is a Dirac delta potential, but I can't see what is forcing the wavefunction to be continuous in the case of an infinite potential (mathematically as well as physically).
Can someone explain this to me?
Without any attempt for rigorous arguments: If the wave function is not continuous across the delta potential, then evaluating the second derivative of it (because of the kinetic term in the Hamiltonian) would give you the derivative of a delta function, which is not a delta function, and the Schroedinger equation is not satisfied at the position of the potential.
Similarly, if the potential is a step function, not only the wave function is continuous at the step, but also so is its derivative, otherwise the Schroedinger equation is not satisfied at the step.
In short: it is important to see that the continuous conditions for the wave function and/or its derivative are required by the Schroedinger equation itself, and not something you take in from elsewhere.
In any quantum mechanical setting, regardless of the type of the potential, one imposes the requirement that the wave-function $\Psi(x)$ (i.e., the solution to the Schrödinger equation) be $C^1$-smooth (continuous with well-defined 1st derivative) or at least, continuous, $C^0$ [see here for a definition], in position. Notice that a discontinuous wave-function will lead to a physically absurd situation, since the probability $P(x)$ of finding the particle in a position interval $[x, x + dx]$, is defined in terms of the absolute value of the wave-function $\Psi(x)$: $$P(x) = |\Psi(x)|^2 ~ ;$$ it is obvious that if the wave-function is not continuous at some point, the probability of finding the particle in the interval containing $x$ will not be defined! Therefore, to have a well-defined physical picture, the wave-function should be at least continuous in its domain.
Having a well-defined first derivative leads to a proper behaviour of other observable physical quantities like the particle current: $$ j = \frac{\hbar}{2m \mathrm{i}}\left(\Psi^* \frac{\partial \Psi }{\partial x}- \Psi \frac{\partial \Psi^* }{\partial x} \right) ~.$$ [Note that this is the simplest expression for the current, but that suffices for the current issue.]
For a detailed discussion, consult, e.g., Ballentine, L. E. “Quantum mechanics: a modern development”. World scientific (1998), sections 4.4 and 4.5.
Correction: The interpretation of $P(x)$ was corrected due to a comment by @ACuriousMind.
