We all know that gravity decreases as the distance between the two increases. Hence
$$ F = G \frac{Mm}{r^2}. $$
Hence the acceleration due to gravity $$ g =\frac{F}{m}= G \frac{M}{r^2} $$
increases as $r$ decreases. Then why does it decrease as we go deep into the earth?
It's actually not entirely true that the strength of the Earth's gravitational field decreases as a function of depth. It is true for certain regions in the Earth, but it's untrue for others because of the non-trivial dependence of the Earth's density on depth.
To see what's going on, assume that the Earth is a sphere whose density is spherically symmetric.
Now consider a mass $m$ at some radius $r$ from the center of the Earth. Using Newton's Law of Gravitation, one can show that that given the spherical symmetry, the gravitational attraction on $m$ of all mass with radii greater than $r$ exert no net force on it. It follows that only the mass with radii less than or equal to $r$ contribute to the gravitational force on $m$, which, by the Law of Gravitation is \begin{align} F(r) = G\frac{M(r)m}{r^2} \end{align} where $M(r)$ is the mass of stuff at radii less than or equal to $r$. Notice, then, that $F(r)$ will be an increasing function of $r$ (and will decrease as $r\to 0$), provided $M(r)/r^2$ is an increasing function of $r$.
Now, If the Earth were uniformly dense with density $\rho_0$, then the mass within a radius $r$ would be \begin{align} M(r) = \frac{4}{3}\pi r^3 \rho_0 \end{align} namely just the density times the volume of a sphere of radius $r$, and in this case the strength of the gravitational field as a function of radius would be \begin{align} g(r) = \frac{F(r)}{m} = G\frac{1}{r^2}\frac{4}{3}\pi r^3\rho_0 = \left(\frac{4}{3}\pi g\rho_0\right) r \end{align} So in this case, it would be true that the strength of the gravitational field would decrease with increasing depth.
However, the Earth's density is not constant, and instead has some non-trivial dependence on $r$. See, for example, this PREM table. You can clearly see that there are regions in the Earth where the strength of the gravitational field increases because of the varying density.
That equation applies for point sources, which the Earth technically is not. We can, however, treat the Earth as a point source as long as its internal structure is irrelevant (i.e. as long as we are outside of it). Once we enter the surface of the Earth, we can no longer simplify it by pretending it's a point and we have to perform a full analysis of the system.
It's important to note, as joshphysics pointed out, that because the density of material in the Earth isn't a constant, moving deeper underground will actually put us closer to a region that is denser (and therefore pulls on us more strongly) while putting a region that is less dense farther away, which would increase the force of gravity. For a first approximation, however, we can assume that the density of the Earth is constant.
So, given this assumption, we can show that only the amount of mass that is still underneath us actually exerts a net pull on us in a variety of ways.
For instance, it's a known result that at any point inside a spherical shell of mass, there is no net gravitational force due to symmetry. We therefore know that once we are inside the Earth, any mass that isn't as deep as we are has no net effect on us. One can also use Gauss's Law to demonstrate the same thing.
In short, the only mass that exerts a net force on us is the mass that is below us, and the deeper we travel underground, the less mass is beneath us. Therefore, there is less gravitational pull as we travel deeper beneath the surface.
