Ok, let me summarize what I learned so far:
Theorem. The following statements are equivalent:
You are standing on (locally) inertial frame.
Your accelerometer reading is zero.
You feel zero proper acceleration.
There are no fictitious or pseudo forces (including gravitation) in your frame.
Christoffel symbols vanish (locally).
All laws of physics are satisfied.
Regarding (5), in general relativity, 4-acceleration $A=(A^0, A^1, A^2, A^3)$, in general (pun intended), can be written as
$$A^\alpha=\frac{dU^\alpha}{d\tau}+\Gamma_{\mu\nu}^\alpha U^\mu U^\nu$$
where $\Gamma_{\mu\nu}^\alpha U^\mu U^\nu$ represents acceleration contribution from spacetime geometry (this is source of all the fictitious forces like coriolis force for example).
Suppose you are at rest in some inertial frame and are grasping an apple in your hand. When you release the apple, it either will remain stationary or move in a straight line (yes, the apple feels zero proper acceleration and the frame of the apple is inertial). Why? The equation of motion for the apple of mass $m$ can be written as
$$f^\alpha=mA^\alpha=m\frac{dU^\alpha}{d\tau}+m\Gamma_{\mu\nu}^\alpha U^\mu U^\nu=0$$
where $U$, $A$, and $f$ are apple's 4-velocity, 4-acceleration and 4-force respectively. Since we are in an inertial frame, Christoffel symbols vanish, i.e. $\Gamma_{\mu\nu}^\alpha=0$ for all $\mu=0,1,2,3$ and $\nu=0,1,2,3$. So the fictitious force contribution is zero: $m\Gamma_{\mu\nu}^\alpha U^\mu U^\nu=0$. The equation of motion is now $m\frac{dU^\alpha}{d\tau}=-m\Gamma_{\mu\nu}^\alpha U^\mu U^\nu=0$. So the apple will remain in rest or move in a straight line.
If we are standing on the surface of Earth (assume the Earth is an infinite plane with uniform gravitational field $g$), which is non-inertial frame, then the apple's equation of motion (in z-direction) will be $m\frac{dU^3}{d\tau}=-m\Gamma_{\mu\nu}^3 U^\mu U^\nu\approx-mg$ (for other components, $m\frac{dU^0}{d\tau}=m\frac{dU^1}{d\tau}=m\frac{dU^2}{d\tau}=0$) instead. The apple falls downwards with acceleration of $g$, in agreement with Newtonian mechanics.
Let me know if I made any misleading statements!