Note: We should be clear about what is meant by "steepest path". In the diagram below, there are two possible paths. They both drop by the same amount over the same overall distance, so they technically have the same average steepness. However, we can make a distinction by defining the "steepest path" as being the path with the greatest initial gradient immediately downstream of the location under consideration.

In the diagram, blue and red paths represent two open channel viaducts with identical surface roughness, water cross sections and initial head heights. Interestingly, the flow in the blue path is approximately $3$ times as much as the flow in the red path, even though the red path has the straightest path and the lowest total path length. The final section of the blue path has a shallow slope, but this can be compensated for by making that section wider. That section could be a natural river with gently rising banks that can accommodate the extra flow. Note that the flow in the blue path is greater than the flow in the red path, even though the head difference in the steep part of the blue path is only $3$ units, and the total head difference of the blue path is $4$ units.
I think this question can initially be analysed by using an electrical circuit analogy and then return to the geological case with water flow in a landscape, which obviously has some differences. Consider a simple circuit with a $12 \ \text{V}$ battery and two resistors ($\text{A}$ & $\text{B}$) in parallel, where $\text{A}$ has a resistance of $3 \ \Omega$ and $\text{B}$ has resistance of $6 \ \Omega$. The total resistance is given by the equation: $$R_{\text{AB}} = \frac{R_{\text{A}} R_{\text{B}}}{R_{\text{A}}+R_{\text{B}}} = \frac{18}{9} = 2 \ \Omega.$$ From this the current through both resistors is $$I_{\text{AB}} = \frac{V_{\text{AB}}}{R_{\text{AB}}} = \frac{12}{2} = 6 \ \text{A}.$$
The current through resistor $\text{A}$ is $$\frac{V_{\text{AB}}}{R_{\text{A}}} = \frac{12}{3} = 4 \ \text{A}$$ and the current through $\text{B}$ is $$\frac{V_{\text{AB}}}{R_{\text{B}}} = \frac{12}{6} = 2 \ \text{A}.$$
It should be obvious that in this analogy, the electrical current represents the water current, and the voltage (or electrical potential) represents the height of the landscape at a given point. The $6 \ \Omega$ resistor can be thought of as two $3 \ \Omega$ resistors in series, and the drop in voltage across each resistor in the $\text{B}$ path is half the drop in voltage across the $\text{A}$ resistor, so the $\text{B}$ path can be thought of as less steep than the $\text{A}$ path. (The drop in potential per unit length of the resistor is lower.)
The important thing to note here is that while most of the current goes through the resistor with the lowest resistance (and steepest path by analogy), not all the current goes through resistor $\text{A}$. So the expression "the current takes the path of least resistance" is not strictly true if by "the current" we mean all the current. If all the current only flowed through resistor $\text{A}$ (the path of least resistance), the total current around the circuit would be only $4 \ \text{A}$. It appears that rather than taking the path of least resistance, the current takes all available paths in order to maximise the total current flow per unit time.
Now, back to the case of a river in a real landscape. Imagine we have a river flowing directly South. It flows into a lake where there is a natural dam preventing the river from continuing South. (The South directed momentum of the river is no longer a factor due to this barrier) Let's say that two rivers flow out of the lake to the East and the West, and the river to the East goes down a steeper slope, so the flow through the East River is greater than the flow in the West River. One factor that has not been taken into account so far is erosion. The faster flow in the East River causes a greater degree of erosion in the East River, making it get wider and deeper faster than the West River. This increases the flow in the East River even more, and it tends to drop the level of the lake. The drop level in the lake might drop below the depth of the West River, cutting the flow off completely in the West River so that the final route of the total river is via the steeper East route. Erosion is the key factor in the fact that real rivers in a real landscape tend to end up taking the steepest route. If a river happens to split into two paths, and one path is over hard rock, and the other path is over soft rock, the soft rock will erode faster, making that path deeper and wider until all the water ends up flowing down the steeper path created in the soft rock.
==============================================
I get the impression the OP is more interested in a more abstract mathematical interpretation than a practical, real-world scenario with real-world complications like erosion. Consider a drop of water on an inclined hydrophobic plane. We could consider the extreme where the plane is vertical. The question becomes: why does the water droplet go straight down and not horizontally or upwards? Intuitively, the answer seems obvious, and, informally, if there is a gravitational potential field, we can assign a gradient vector to every point in the field, and the particle will always be accelerated in the direction of the gradient that goes from a higher potential to a lower potential, but its actual future trajectory will also depend on its current velocity. If the droplet has a significant initial horizontal velocity of $\dfrac{\mathrm{d}x}{\mathrm{d}t}$ and the force of gravity is accelerating the droplet in the $y$ direction, its velocity vector rotates more towards the downhill's $y$ direction over time. If there is resistance and $\dfrac{\mathrm{d}x}{\mathrm{d}t}$ gets smaller over time, the velocity vector rotates even faster to the downhill's $y$ direction. Since the OP has asked us to ignore inertia, $\dfrac{\mathrm{d}x}{\mathrm{d}t}$ goes to zero, and the only possible velocity vector is straight downhill.
More formally, the answer is that the drop takes the path of least action. Briefly, if we consider different paths the droplet could take to get from $\text{A}$ to $\text{B}$, the action ($S$) of any given path is equal to the integral of the Kinetic Energy minus the integral of the Potential Energy with respect to time. To find the KE and PE as functions of time, we first need the equations of motion for distance (which I will use for height) and for velocity. They are:
$$h = v_0 t - \frac{1}{2} g t^2 \quad \text{and} \quad v=v_0 + g t.$$
The equations for $\text{PE}$ and $\text{KE}$ are:
$$\text{PE} = mgh \quad \text{and} \quad \text{KE} = \frac{1}{2} m v^2.$$
Substituting the equations for $h$ and $v$ as functions of time into the equations for $\text{PE}$ and $\text{KE}$ gives:
$$\text{PE}(t) = mg \left(-\frac{1}{2} g t^2 \right) = -\frac{1}{2} m g^2 t^2 \quad \text{and} \quad \text{KE}(t) = \frac{1}{2} m (g t)^2 = \frac{1}{2} m g^2 t^2.$$
I have left out $v_0$ as I will use an initial velocity of zero for the water droplet. This simplifies things a lot.
Now we find the action:
\begin{align}
S &= \int\limits_0^t \text{KE}(t) \ \mathrm{d}t - \int\limits_0^t \text{PE}(t) \ \mathrm{d}t \\
&= \int\limits_0^t \left(\frac{1}{2} m g^2 t^2 \right) \ \mathrm{d}t - \int_0^t \left(-\frac{1}{2} m g^2 t^2 \right) \ \mathrm{d}t \\
&= \frac{1}{3} g^2 m t^3.
\end{align}
Any other path would yield a higher value of $S$.
Calculating $S$ for diagonal paths is more complex. For the vertical, inclined plate and a non-vertical path, the equations for a projectile trajectory and an initial non-zero velocity would be required. For an inclined plane, the cosine of the gravitational acceleration factor would have to be used.