Arrogance sampling

Posted on January 7, 2011 by xi'an in R bloggers, Uncategorized | 0 Comments

[This article was first published on Xi'an's Og » R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

A new posting on arXiv by Benedict Escoto on a simulation method for approximating normalising constants (i.e. evidence) with an eye-catching name! Here is the abstract

This paper describes a method for estimating the marginal likelihood or Bayes factors of Bayesian models using non-parametric importance sampling (“arrogance sampling”). This method can also be used to compute the normalizing constant of probability distributions. Because the required inputs are samples from the distribution to be normalized and the scaled density at those samples, this method may be a convenient replacement for the harmonic mean estimator. The method has been implemented in the open source R package margLikArrogance.

The crux of the arrogant sampling method is in using a non-parametric estimation of the target function, based on a preliminary simulation from the posterior distribution. The nonparametric estimate is entered in an harmonic mean representation we previously exploited in our HPD proposal for evidence approximation

$1bigg/frac{1}{T}sum_t {hat{pi}(theta_t)}big/{p(theta_t,x)}$

This estimate $hatpi$ is an histogram estimate that is smoothed by the knowledge of the joint density at points in the bins. Given that the support of the histogram is not restricted to be smaller than the support of $pi(theta|x)$ , as in our proposal, there could be support problems since the bins of the histogram are data-dependent. The associated R package margLikArrogance seems to be able to construct the bins by itself, but I have not tested it. Overall, the method seems to be bound to suffer from the curse of dimensionality, unless the bin construction is aggressively reacting to empty bins. Since the paper contains no illustration whatsoever, it is difficult to tell… I would also like to see more details on the convergence results, including the CLT, on the arrogant approximation because $hatpi$ depends on the whole sample, thus does not benefit from standard importance convergence results.