Bayesian statistics provide a conceptually simple process for updating uncertainty in the light of evidence. Initial beliefs about some unknown quantity are represented by a prior distribution. Information in the data is expressed by the likelihood function. The prior distribution and the likelihood function are then combined to obtain the posterior distribution for the quantity of interest. The posterior distribution expresses our revised uncertainty in light of the data,

in other words an organized appraisal in the consideration of previous experience.

To propose a distribution of the unknown parameters of a statistical model may be characterized as a probabilization of uncertainty, i.e. as an axiomatic reduction from the notion of unknown to the notion of random, (as opposed to the conventional dichotomic investigation over whether or not parameters are equal to zero). In this manner, the Bayesian approach allows us to make direct probability statements about unique and singular systems, whereas classical statistics, concerned with the long run performance of inferential procedures (tests or confidence intervals) in some hypothetically infinite sequence of applications, can hardly be applied to any unique case.

Thus, the Bayesian paradigm induces a dramatic shift in the interpretation of probabilities and their associated random variables: whereas to a frequentist "probability" can only refer to the result of an infinite series of trials under identical conditions, a Bayesian interprets probabilities to refer to the observer's degree of belief.

back