News + Publications

Bayesian methods for data analysis have become more popular as computing power has increased and become more easily accessible. We’ve talked about some of them in previous posts on Bayesian parameter estimation and Kalman filters. When thinking about Bayesian methods, it’s worth reminding ourselves what exactly makes them different from methods based on frequentist statistics, and why they need more computing power.

Bayesian methods are based on Bayes’ theorem, which gives the conditional probability of some set of parameters, θ, describing a process having particular values, given a set of data. This probability is proportional to the likelihood of the data given the parameter values and to probability of the parameters having those values (known as the prior). By calculating this conditional probability for various parameter values, you can find the most likely values and their distribution given the data available.

One of the advantages of Bayesian methods is that they allow for existing knowledge or reasonable guesses to be incorporated through the specification of the priors and their probability distributions. This means that if you have a rough idea of what to expect from your process, you can use that to improve your model. It also means that if your guesses are not quite right, the data incorporated into the model will push you toward better values. Another advantage is that new data or new insight into the prior values can be incorporated easily.

The computational requirements for Bayesian methods stem from the fact that you need to search for the most likely prior values by repeatedly evaluating the product of the likelihood and prior distributions over the parameter space. For complex models with large amounts of data and multiple parameters that each have their own distributions, this can be computationally expensive. Fortunately, sufficient computing power is now often available, and current methods, e.g. Markov chain Monte Carlo, make this practical.