Possible error in Bayesian bootstrap
[This article was first published on R snippets, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
After my last post on Bayesian bootstrap I got a question why the sample from Dirichlet distribution is taken as weights for calculating mean in the procedure and not as weights used for sampling from the original data set. Actually this mistake is subtle and occurs even in textbooks, see example Chernick (2008), page 122. In this post I want to clarify the issue.Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
In the example I give correct bootstrap and Bayesian bootstrap procedures and wrong ones. The wrong Bayesian bootstrap follows description from Chernick (2008), page 122 (that is equivalent to the comment to my last post).
Here is the code that I used:
library(gtools)
ok.mean.bb <- function(x, n) {
apply(rdirichlet(n, rep(1,length(x))), 1, weighted.mean, x = x)
}
ok.mean.fb <- function(x, n) {
replicate(n, mean(sample(x, length(x), TRUE)))
}
wrong.mean.bb <- function(x, n) {
replicate(n, mean(sample(x, length(x), TRUE,
diff(c(0, sort(runif(length(x) – 1)), 1)))))
}
wrong.mean.fb <- function(x, n) {
replicate(n, mean(sample(sample(x, length(x), TRUE),
length(x), TRUE)))
}
set.seed(1)
reps <- 10000
x <- cars$dist
par(mar=c(5,4,1,2))
plot(density(ok.mean.fb(x, reps)), main = “”, xlab = “Bootstrap mean”)
lines(density(ok.mean.bb(x, reps)), col = “red”)
lines(density(wrong.mean.fb(x, reps)), col = “blue”)
lines(density(wrong.mean.bb(x, reps)), col = “green”)The figure it produces is:
Black curve is standard bootstrap density, red is Bayesian bootstrap and blue and green are generated by wrong bootstrapping procedures (respectively frequentist and Bayesian).
We can see that wrong Bayesian bootstrap has an equivalent in standard bootstrap approach that is generated by repeating the sampling twice (sampling from a sample) and it clearly increases dispersion of the results.
To leave a comment for the author, please follow the link and comment on their blog: R snippets.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.