13

Analysis of data is integral in bridging the gap between theory and experiment. How much do the results of the analysis depend upon the choice between Bayesian and frequentist methods?

For instance, consider experiments in particle physics. This paper by Louis Lyons states that particle physicists use a hybrid approach to analysis (frequentist hypothesis testing and Bayesian parameter estimation). If a purely Bayesian/Frequentist approach were applied, would there be a difference in the results of the analysis? What implication would any difference have on the interpretation of data?

DanielSank
  • 25,766
Comp_Warrior
  • 1,241

1 Answers1

9

If there are enough data and the prior is not completely unreasonable, the frequentist and the Bayesian approach give essentially the same answer. This is related to the central limit theorem.

If data are fairly scarce, the two approaches may differ a lot. In this case the Bayesian approach is far preferable but only if the prior reflects true prior knowledge and not just prejudice. (With a drastically wrong prior and limited data, the Bayesian approach tends to reaffirm the prejudice, and hence will be far worse than the frequentist result.)

To see this, take a prior and assume that just one observation. It is clear that the Bayesian outcome is just a small change of the prior. So if the prior was appropriate (reflected true knowledge), the result is an improvement, while if the prior was bogus (just prejudice), the outcome is as bad. If the number of observations is large, their contribution dominates the outcome, and the result is essentially prior-independent, and is easily seen to agree with the frequentist (maximum likelihood) result.

All this is completely independent of physics. But in statistical mechanics we have a good enough theory that enables us to choose appropriate priors. This is the (only) reason why the maximum entropy principle works there.

Arnold Neumaier
  • 46,304