First of all, your definition of height is unclear to me. I'm assuming by height you mean the separation from the optical axis of the image of a point object through the objective.
One way of looking at this is that for an object infinitely far away and the light from which arrives at the objective at a non-zero angle with respect to the optical axis, the "height" of the object is also "infinite", so the image rendered has a finite height rather than zero.
Now suppose the object with height $h_o$ is a distance $x_o$ away from the objective, and its image forms at $x_i$ with height $h_i$. If the rays from the object reach the objective at an angle $\theta_o$, we have
$$h_o = \tan(\theta_o)x_o. $$
For an object very far away, we have
$$x_i \approx f$$
where $f$ is the focal length of the objective. The linear magnification is the negative ratio of distances of the image and the object to the objective:
$$ M = -\frac{x_i}{x_o} $$
so the height of the image is
$$ h_i = M h_o = -\frac{x_i}{x_o} h_o = -\frac{x_i}{x_o}\tan(\theta_o)x_o = -\tan(\theta_o) x_i \approx -\tan(\theta_0)f $$
where the negative sign indicates that the image is inverted. This relation can also be seen by simply drawing rays from the object, without doing math.