Bill Bolstad’s response to Xi’an’s review of his book Understanding Computational Bayesian Statistics included the following comment, which I found interesting:

Frequentist p-values are constructed in the parameter dimension using a probability distribution defined only in the observation dimension. Bayesian credible intervals are constructed in the parameter dimension using a probability distribution in the parameter dimension. I think that is more straightforward.

Classical statistics is concerned with the distribution of statistics that estimate a fixed population parameter. And, statistics are clearly constructed in the observation dimension. But, consider that a statistic evaluated at the population level (i.e., computed using all members of a finite population, or the limit in an infinite sequence of observations) is also a population parameter. In this sense, there is no distinction between the observation and parameter dimensions.

It seems natural to ask: “What is the value of a statistic if computed at the population level?” Classical statistics offer an answer. The Bayesian analog is not immediately clear.

Perhaps ‘classical statistics’ should be just ‘statistics’, and ‘Bayesian statistics’, just ‘parameters’.