P-values: the Continuing Saga

I highly recommend the blog post by Yoav Benjamini and Tal Galili in defense of (carefully used) p-values. I disagree with much of it, but the exposition is very clear, and there is a nice guide to relevant R tools, including for simultaneous inference, a field in which Yoav is one of the most prominent, indeed pre-eminent, researchers. I do have a few points to make.

First, regarding exactly what the ASA said, I would refer readers to my second post on the matter, which argues that the ASA statement was considerably stronger than Yoav and Tal took it to be.

Second, Yoav and Tal make the point that one can’t beat p-values for simplicity of assumptions. I’d add to that point the example of permutation tests. Of course, my objections remain, but putting that aside, I would note that I too tend to be a minimalist in assumptions — I’ve never liked the likelihood idea, for instance — and I would cite my example in my second post of much-generalized Scheffe’ intervals as an example. Those who read my 50% draft book on regression and classification will see this as a recurring theme.

I of course agree strongly with Yoav and Tal’s point about problems with just checking whether a confidence interval contains 0, a point I had made too.

What I would like to see from them, though, is what I mentioned several times in the last couple of days — a good, convincing example in which p-values are useful. That really has to be the bottom line.