Want to prove a differentiation based formula for propagated uncertainty

Question

I recently learned of the concept of propagated uncertainty, and I was introduced to the rule that if $$ X = AB$$ then if $A$ has an uncertainty of $\Delta A$ and $B$ has an uncertainty of $\Delta B$, the uncertainty of $X$ can be found using the formula $$\frac{\Delta X}{X} = \frac{\Delta A}{A} + \frac{\Delta B}{B}$$

I've been told that this is somewhat based off differentiation; I want to know the logic behind this. Could anyone please help me understand how they came to this formula? It's not very intuitive.

Answer 1

For X = AB

$\frac{dX}{dB}=A$ ... this is the rate of change of X when A varies infintesimally

$\frac{dX}{dA}=B$ ... this is the rate of change of X when B varies infintesimally

thus, given the two errors $\Delta A$ and $\Delta B$ and , we can determine the approximate variation in X. (Smaller errors will make this more accurate.)

$\Delta X = \frac{dX}{dA} \Delta A + \frac{dX}{dB} \Delta B$

(aside: the term in $\Delta A \Delta B$ is small and has been omitted.)

$$\boxed{\Delta X = B\Delta A + A\Delta B }$$

The above is the equation I use. Division by AB (noting AB=X) gives a slightly more memorable version,

$\frac{\Delta X}{X} = \frac{\Delta A}{A} + \frac{\Delta B}{B} $

This can be visualised using the example of calculation of area for a rectangle.

Answer 2

@JMLCarter's answer is what would be expected of you at your academic level, I guess, but I have to warn you that your formula is actually incorrect if you are serious about probability and statistics.

Why probability and statistics, you may ask? Because in a lot of practical applications, $A$ does actually not represent a single value but a random distribution of values, what you would get by repeated measurements. You can then define a mean value $\newcommand{EE}[1]{\langle #1 \rangle} \EE{A}$ and we also need to define how the values are spread about that mean value, which will give us a way to measure the uncertainty. This is done by looking at the mean value of the square of the differences to the mean: $V(A)=\EE{(A-\EE{A})^2}$, a quantity called the variance of $A$. Then the uncertainty you seek is just the square root of the variance: $\sigma_A = \sqrt{V(A)}$, or conversely the variance is the square of that uncertainty. So your $\Delta A$ is my $\sigma_A$, etc. $\sigma_A$ is called the standard uncertainty of $A$ by the way. We define similar quantities for $B$ and $X$.

I will assume a very important condition: that $A$ and $B$ are independent, i.e. that whatever is the source of error for one does not influence the other. This has an important consequence: the mean of $AB$ is just the product of the means (i.e. there is no "interference" between $A$ and $B$), or mathematically.

$$\EE{AB}=\EE{A}\EE{B}.$$

Thus the variance reads

$$V(AB) = \EE{(AB-\EE{AB})^2}=\EE{(AB-\EE{A}\EE{B})^2}.$$

Then, we have the awesome trick:

$$AB - \EE{A}\EE{B} = \EE{A}\EE{B}\left[\left(\underbrace{\frac{A-\EE{A}}{\EE{A}}}_{\displaystyle\delta_A}+1\right)\left(\underbrace{\frac{B-\EE{B}}{\EE{B}}}_{\displaystyle\delta_B}+1\right)-1\right] =\EE{A}\EE{B}\left[\delta_A + \delta_B + \delta_A\delta_B\right]. \tag{I} $$

We will then assume that $\delta_A$ and $\delta_B$ are small. Such a statement makes sense since their mean values is 0. We can therefore allow ourselves to neglect the term $\delta_A\delta_B$. We square:

$$(AB - \EE{A}\EE{B})^2=\EE{A}^2\EE{B}^2(\delta_A^2+\delta_B^2 + 2\delta_A\delta_B)$$

and take the mean,

$$V(AB)=\EE{A}^2\EE{B}^2\left(\frac{V(A)}{\EE{A}^2}+\frac{V(B)}{\EE{B}^2}\right),$$

exploiting the fact that $\delta_A$ and $\delta_B$ are independent since $A$ and $B$ are so, and therefore that $\EE{\delta_A\delta_B}=\EE{\delta_A}\EE{\delta_B}=0$.

Finally, we get

$$\frac{\sigma_X}{\EE{X}} = \sqrt{\left(\frac{\sigma_A}{\EE{A}}\right)^2+\left(\frac{\sigma_B}{\EE{B}}\right)^2}\tag{II}$$

This is actually the correct formula as per the laws of statistics. So how much does it differ from your naive formula? You can see it on the following diagram.

You are basically replacing Pythgoras theorem with: the hypothenuses is the sum of the two other sides. Well, we have the triangular identity

$$\frac{\sigma_X}{\EE{X}} \le \frac{\sigma_A}{\EE{A}} + \frac{\sigma_B}{\EE{B}}$$

So at least the naive formula gives an upper bound: you find an uncertainty for $X$ which is guaranteed to be bigger than the correct one, which is fortunate!

Addendum: since the question was marked as a duplicate, I should at least try to add to the previous answer. We can compute $\sigma_X$ exactly without neglecting anything. Let's restart from eqn (I): we square without neglecting anything and get

$$(AB - \EE{A}\EE{B})^2=\EE{A}^2\EE{B}^2\big(\delta_A^2+\delta_B^2 + 2\delta_A\delta_B + 2\delta_A^2\delta_B + 2\delta_A\delta_B^2+\delta_A^2\delta_B^2\big)$$

and then take the mean value, using the fact that $\delta_A$ and $\delta_B$ are independent,

$$\sigma_X^2 = \EE{A}^2\EE{B}^2\big(\EE{\delta_A^2}+\EE{\delta_B^2} + 2\underbrace{\EE{\delta_A}}_0\EE{\delta_B} + 2\EE{\delta_A^2}\underbrace{\EE{\delta_B}}_0 + 2\underbrace{\EE{\delta_A}}_0\EE{\delta_B^2}+\EE{\delta_A^2}\EE{\delta_B^2}\big).$$

Thus we finally get

$$\frac{\sigma_X}{\EE{X}} = \sqrt{\left(\frac{\sigma_A}{\EE{A}}\right)^2+\left(\frac{\sigma_B}{\EE{B}}\right)^2+\left(\frac{\sigma_A\sigma_B}{\EE{A}\EE{B}}\right)^2}$$

thus adding a corrective term to the previous eqn. (II). And that is the fully correct formula without any approximation whatsoever.

Answer 3

This is more of a comment on the answers given by @JMLCarter and @LucJ.Bourhis rather than an answer.

$A$ has an uncertainty of $\Delta A$ and $B$ has an uncertainty of $\Delta B$

in its simplest interpretation means that the actual value of $A$ is somewhere between $A+\Delta A$ and $A-\Delta A$ and that the actual value of $B$ is somewhere between $B+\Delta B$ and $B-\Delta B$

So the maximum value of $AB$ is $(A+\Delta A)(B+\Delta B) = AB + A\Delta B + B\Delta A+ \Delta A \Delta B$

So the maximum fractional error in $AB$ is given by $\dfrac {\Delta (AB)}{AB} = \dfrac{\Delta A}{A} + \dfrac {\Delta B}{B}+ \dfrac{\Delta A \Delta B}{AB}$

If $\Delta A$ and $\Delta B$ are small then $\Delta A \Delta B$ (a second order term) will be very small and so can be neglected.
This is the sort of procedure used in calculus to find a differential from first principles.

---

The @JMLCarter error is a sort of maximum error in that the error in $AB$ is calculated from the spread of possible values of $AB$ from when $A$ and $B$ are both a maximum ($A+\Delta A$ and $B+\Delta B$) at the same time to when $A$ and $B$ are both a minimum ($A+\Delta A$ and $B+\Delta B$) at the same time.

---

The @LucJ.Bourhis error, which is smaller than the @JMLCarter error, is worked out on the assumption that having the errors in both $A$ and $B$ at a maximum (or a minimum) at the same time are not very likely events.
For example the errors could be in opposite directions ($A+\Delta A$ and $B-\Delta B$ or $A-\Delta A$ and $B+\Delta B$) and it is much more likely that the errors are smaller than $\Delta A$ and $\Delta B$.
So the chances of the fractional error in $AB$ being the sum of the fractional errors in $A$ and $B$ is small.

A better estimator of the error in $AB$, and hence the likelihood of the actual value of $AB$ being between $AB +\,\rm error$ and $AB - \,\rm error$, is found by evaluating the square root of the squares of the fraction errors in $A$ and $B$ which will give the fractional error in $AB$.