Merzbacher in his Quantum Mechanics says that for the "particle in a box" potential ($V(x) = 0$ for $|x|\le L$ and $+\infty$ otherwise),
Since the expectation value of the potential energy must be finite, the wavefunction must vanish within and on the walls of the box.
However, I don't quite get this reasoning. Why must the potential energy's expectation value be finite?
Let's first write the Time-independent Schrödinger equation
$$\hat{H}|\Psi\rangle=E|\Psi\rangle$$
or
$$\left(\frac{\hat{P}^2}{2m}+\hat{V}(x)\right)|\Psi\rangle=E|\Psi\rangle$$ or $$\langle\Psi|\left(\frac{\hat{P}^2}{2m}+\hat{V}(x)\right)|\Psi\rangle=E\langle\Psi|\Psi\rangle=E$$ or $$\langle\Psi|\frac{\hat{P}^2}{2m}|\Psi\rangle+\langle\Psi|\hat{V}(x)|\Psi\rangle=E$$ or $$\langle\Psi|\hat{V}(x)|\Psi\rangle=E-\frac{1}{2m}\langle\Psi|\hat{P}^2|\Psi\rangle$$
On a right hand side, You have a finite quantity and so you will expect $$\langle\Psi|\hat{V}(x)|\Psi\rangle < \infty$$ This can further be written as ( in position basis) $$\langle \hat{V}(x) \rangle=\int dx\psi^*(x)V(x)\psi(x)<\infty$$
Now for interval $x\in [-L,L]$, $\langle V(x) \rangle$ is zero as $V(x)=0.$ For other value of $x$, $V(x)=\infty$ and so $\psi(x)$ must be zero otherwise $\langle V(x) \rangle=\infty$.
Thus the wavefuntion (position ) must vanish outside the well. Cheers :)
Young Kindaichi already provided a response to your question. What I add here is an alternative way of justifying the vanishing of the wave function in a region of infinite potential.
Let's start with some general discussion. Consider the Hamiltonian eigenvalue equation in a region of constant potential $V$ in the position representation:
$$ -\frac{\hbar^2}{2m}\frac{d^2\psi(x)}{dx^2}+V\psi(x)=E\psi(x). $$
For a particle with energy $E>V$, the general solution is:
$$ \psi(x)=Ae^{ikx}+Be^{-ikx}, $$ with $k=\sqrt{2m(E-V)}/\hbar$. For a particle with energy $E<V$, the solution is:
$$ \psi(x)=Ae^{qx}+Be^{-qx}, $$ with $q=\sqrt{2m(V-E)}/\hbar$. For a piece-wise potential made of constant regions, then you solve the constant potential problem in each region and match boundary conditions at the points between regions.
Going back to your question, a good way to understand the vanishing of the wave function where the potential becomes infinite is to consider this situation as a limit of the finite step potential:
$$ V(x)= \begin{cases} V_0, &x>0 \\ 0, &x<0. \end{cases} $$
This is an example of a piece-wise potential discussed above, and let us call $x<0$ "region I" and $x>0$ "region II". Let the energy of our particle be $0<E<V_0$. In region I, the wave function reads:
$$ \psi_I(x)=Ae^{ikx}+Be^{-ikx}, $$
with $k=\sqrt{2mE}/\hbar$. In region II the solution is
$$ \psi_{II}(x)=Ce^{qx}+De^{-qx}, $$
with $q=\sqrt{2m(V_0-E)}/\hbar$.
Now let's turn to boundary conditions. We must set $C=0$, as otherwise $\psi_{II}(x)$ is not bounded as $x\to\infty$. This means that the solution becomes: $\psi_{II}(x)=De^{-qx}$. Applying boundary conditions at $x=0$ gives:
$$ \begin{cases} \frac{B}{A}=\frac{k-iq}{k+iq},\\ \frac{D}{C}=\frac{2k}{k+iq}, \end{cases} $$
This is the general solution for a step potential.
We can now recover your situation of an infinite barrier by taking the limit $V_0\to\infty$. This gives $q\to\infty$ such that:
$$ \begin{cases} \frac{B}{A}=\frac{k-iq}{k+iq}\Longrightarrow B=-A,\\ \frac{D}{A}=\frac{2k}{k+iq}\Longrightarrow D=0. \end{cases} $$
As $D=0$, the wave function vanishes in the region $x>0$ for an infinite potential.
