Evidence for a limit to effective peer review

I missed it first time around but apparently, back in October, Nature published a somewhat-controversial article: Evidence for a limit to human lifespan. It came to my attention in a recent tweet:

The source: a fact-check article from Dutch news organisation NRC titled “Nature article is wrong about 115 year limit on human lifespan“. NRC seem rather interested in this research article. They have published another more recent critique of the work, titled “Statistical problems, but not enough to warrant a rejection” and a discussion of that critique, Peer review post-mortem: how a flawed aging study was published in Nature.

Unfortunately, the first NRC article does itself no favours by using non-comparable x-axis scales for its charts and not really explaining very well how the different datasets (IDL and GRG) were used. Data nerds everywhere then, are wondering whether to repeat the analysis themselves and perhaps fire off a letter to Nature.

My advice: don’t waste your time since Philipp Berens and Tom Wallis already did a great job, which is described here and documented (in a manner that you wish applied to Nature articles) at Github.

That said: I have wasted a little time and the results are in this Github repository. My code illustrates one way to obtain the data, and apply some regression models (linear, segmented and LOESS). One contentious aspect of the original paper was the choice of breakpoint (1995) for segmented regression, which seemed rather arbitrary. In fact it is possible to fit something very similar (albeit with a breakpoint in 1999) to the “maximum age at death by year” data using the R segmented package. Thanks to Nathan Lemoine for his tutorial on that topic.

As to whether segmented linear regression should have been applied, the arguments continue. According to Berens and Wallis, it is supported by the GRG dataset but not by the data from the IDL. All of the articles linked to in this post are worth a read, in particular the third of the NRC articles which contains some telling quotes regarding how articles come to be published. This one is my favourite:

Until then, I had thought focusing on statistics was a major task for a reviewer. But at Nature, that is apparently not a problem at all. In the eleven primary questions drawn up by the journal for an “ideal review”, statistics and methodology are not explicitly addressed. They feature in the second list of questions, for “if time is available”.

Filed under: publications, R, statistics Tagged: human, longevity, peer review