by Giles Strong

So, I’m back from my statistics school in Autrans, and Lisbon has managed to get even hotter (currently 31 °C!). Luckily I’m escaping off to Sweden soon, where Google informs me that the weather is much more acceptable – 18 °C. Anyway, I’ve had a bit of time to digest the topics covered during the school, one of which was Bayesian statistics.

I’d had an introduction to Bayesian statistics before at a two-day workshop during my Glasgow masters, and had kind of got the gist of it, but forgotten most of the details. I had remembered though, that Bayesian versus Frequentist approaches were something hotly debated by proponents of each.

Bayesian statistics revolves around Bayes’ Theorem: the probability of one’s model for some experiment being correct, given the experimental results one observes, is equal to the probability of one’s model producing the results observed, divided by the probability of any model producing the result one observes, times one’s prior understanding of the experiment.

With no prior understanding of the experiment one can simply assume that all outcomes are equally likely. If one does already have some information on the experiment, this can be included, and one’s knowledge refined.

As I was being taught about it, I gradually warmed to the approach; problems could easily be read and interpreted in terms of Bayes’ Theorem, and then calculated to provide a solution which considered the data and one’s prior knowledge. Then we got to the Bayes factor.

The Bayes factor, the ratio of the probability of the data given one model to the probability of it given another model, is similar to the Frequentist p-value; a way of comparing two models and, if large enough, justifying the rejection of one model in preference of a new one, e.g. particle-physics’ five-sigma significance.

Ok, that’s useful, but wait… what’s this… the Bayes factor can also be formulated using the ratio of the probability of one model given the data to the probability of the other model given the data, times the ratio of our beliefs in each model; something we are free to decide ourselves. Subjectivity on our surgically sterilised table of science?! Gross! With this, a reader is free to decide how convincing our results are; and if they have no belief in our model, then no amount of evidence will ever convince them.

My attitude towards Bayesian analysis became somewhat colder, but the lecturer moved onto some examples. The first was the six-sigma evidence for super-luminal neutrinos reported in 2011. Far exceeding the five-sigma requirement for a discovery, this model should have been blindly accepted under a Frequentist analysis.

However in order to be accepted in a Bayesian framework, it would have required us to believe that the probability of there being super-luminal neutrinos divided by the probability that there weren’t, was greater than 1%. Given that experiments show that neutrinos have mass, and Einstein’s theory of relativity states that no massive objects can move at the speed of light, this belief is hard to justify. Conversely, a belief that there was a problem with the timing system is a much easier model to justify belief in.

The second example concerned the discovery of the Higgs boson. The significance was greater than five-sigma and so the Higgs was determined to be discovered, but in realty there were an infinite number of other possible explanations for the data; the lecturer gave the example of magical elves pushing against particles and providing them with mass. So why did we discover the Higgs and not elves? Because unconsciously we held no belief in the elf hypothesis. This subjectivity was never acknowledged in the Frequentist calculation, but was present nonetheless. The Bayesian framework simply takes subjectivity into account explicitly.

Subjectivity, it seems, is a superbug that has always been present on our ‘sterile’ tables, but perhaps it is here to help, and should be given the explicit quantitative assessment it deserves; lest Legolas languish in our labs.

For those interested in learning more, the slides for the lecture are available here.

Cover image taken from XKCD.