The Newtonian limit is defined as
$${g_{\mu \nu }} = {\eta _{\mu \nu }} + {h_{\mu \nu }},\,{\rm{where}}\,\,\,\left| {{h_{\mu \nu }}} \right| < < 1.$$
From the definition of the inverse metric, i.e. ${g_{\mu \lambda }}{g^{\lambda \nu }} = \delta _\mu ^\nu $, one can check that ${g^{\mu \nu }} = {\eta ^{\mu \nu }} - {h^{\mu \nu }}$ is the inverse up to first order in $h$, i.e.
$$\begin{array}{l}
{g_{\mu \lambda }}{g^{\lambda \nu }} = ({\eta _{\mu \lambda }} + {h_{\mu \lambda }})({\eta ^{\lambda \nu }} - {h^{\lambda \nu }})\\
\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, = {\eta _{\mu \lambda }}{\eta ^{\lambda \nu }}\underbrace { - {\eta _{\mu \lambda }}{h^{\lambda \nu }} + {h_{\mu \lambda }}{\eta ^{\lambda \nu }}}_{ = 0} - {h_{\mu \lambda }}{h^{\lambda \nu }}\\
\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, = \delta _\mu ^\nu \, - O({h^2}).
\end{array}$$
An edit motivated by the Frobenius's comment:
When I first encountered the problem of inverting a metric tensor, I solved it for myself in this way:
$$\begin{array}{l}
{g_{\mu \lambda }}{g^{\lambda \nu }} = \delta _\mu ^\nu \, \Rightarrow \,{g^{\lambda \nu }} = \frac{{\delta _\mu ^\nu }}{{{g_{\mu \lambda }}}}\\
\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, = \frac{{\delta _\mu ^\nu }}{{{\eta _{\mu \lambda }} + {h_{\mu \lambda }}}}\\
\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, = \delta _\mu ^\nu ({\eta ^{\mu \lambda }} - {h^{\mu \lambda }})\\
\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, = {\eta ^{\lambda \nu }} - {h^{\lambda \nu }},
\end{array}$$
where ${(1 + x)^{ - 1}} \approx 1 - x$ has been used in the third line. My essential rule (which, I admit, is a slightly awkward thing to do) was this: In expansions, every (tensor) quantity that goes from the denominator to the nominator, its indices also go up and down accordingly.