Confusing slice sampler

Posted on May 18, 2010 by xi'an in R bloggers | 1 Comment

[This article was first published on Xi'an's Og » R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Most embarrassingly, Liaosa Xu from Virginia Tech sent the following email almost a month ago and I forgot to reply:

I have a question regarding your example 7.11 in your book Introducing Monte Carlo Methods with R. To further decompose the uniform simulation by sampling a and b step by step, how you determine the upper bound for sampling of a? I don’t know why, for all y(i)=0, we need a+bx(i)>- log(u(i)/(1-u(i))). It seems that for y(i)=0, we get 0>log(u(i)/(1-u(i))). Thanks a lot for your clarification.

There is nothing wrong with our resolution of the logit simulation problem but I acknowledge the way we wrote it is most confusing! Especially when switching from $(alpha,beta)$ to $(a,b)$ in the middle of the example….

Starting with the likelihood/posterior

$L(alpha, beta | mathbf{y}) propto prod_{i=1}^n left(dfrac{e^{ alpha +beta x_i }}{1 + e^{ alpha +beta x_i }}right)^{y_i}left(dfrac{1}{1 + e^{ alpha +beta x_i }}right)^{1-y_i}$

we use slice sampling to replace each logistic expression with an indicator involving a uniform auxiliary variable

$U_i sim mathcal{U}left( 0,dfrac{e^{ y_i(alpha +beta x_i) }}{1 + e^{ alpha +beta x_i }} right)$

[which is the first formula at the top of page 220.] Now, when considering the joint distribution of

$(alpha,beta,u_1,...,u_n)$ ,

we only get a product of indicators. Either indicators that

$u_i<text{logit}(alpha+beta x_i)$ or of $u_i<1-text{logit}(alpha+beta x_i)$ ,

depending on whether y_i=1 or y_i=0. The first case produces the equivalent condition

$\alpha+\beta x_i > \log(u_i/(1-u_i))$ log(u_i/(1-u_i))’ title=’alpha+beta x_i > log(u_i/(1-u_i))’ class=’latex’ />

and the second case the equivalent condition

$alpha+beta x_i < - log(u_i/(1-u_i))$

This is how we derive both uniform distributions in $alpha$ and $beta$.

What is both a typo and potentially confusing is the second formula in page 220, where we mention the uniform over the set.

$\left\{ (a,b)\,:\ y_i(a+bx_i) > \log\dfrac{u_i}{1-u_i} \right\}$ logdfrac{u_i}{1-u_i} right}’ title=’left{ (a,b),: y_i(a+bx_i) > logdfrac{u_i}{1-u_i} right}’ class=’latex’ />

This set is missing (a) an intersection sign before the curly bracket and (b) a $(1-)^y_i$ instead of the $y_i$ . It should be

$\displaystyle{\bigcap_{i=1}^n} \left\{ (a,b)\,:\ (-1)^{y_i}(a+bx_i) > \log\dfrac{u_i}{1-u_i} \right\}$ logdfrac{u_i}{1-u_i} right}’ title=’displaystyle{bigcap_{i=1}^n} left{ (a,b),: (-1)^{y_i}(a+bx_i) > logdfrac{u_i}{1-u_i} right}’ class=’latex’ />