R, drug development and the FDA

by Joseph Rickert

When you not directly working in an industry it is often extremely difficult to get any real insight into common practices that may be blindly transparent to people who are. With some persistence though, every once in awhile you can stumble into an opportunity to see why things are the way they are. Last week, at the JSM in Montreal, I was lucky enough  to hear some candid discussion of the relative roles of SAS and R in the pharmaceutical industry during the invited session Opening the Doors to Open Source Programming in Drug Development.

Now, one doesn’t have to be much of an insider to know that SAS has been dominant in pharma for quite some time. This is still true even though there is ample evidence that the situation is changing and R is seeing quite a bit of use in the drug development process. At useR! 2007, Mat Sokup of the FDA gave a talk entitled R: Regulatory Compliance and Validation Issues A Guidance Document for the Use of R in Regulated Clinical Trial Environments in which he concluded that there was no impediment to using R in an FDA submission. Last year at useR 2012 FDA statistician Jea Brodsky presented a poster described how FDA scientists “use R on a daily basis” and have themselves written R packages for use at various stages in the drug submission process. Nevertheless, certain myths persist.

During his talk: Open Source Software in the Biopharma Industry: Challenges and Opportunities, José Pinheiro of Jansen R & D (and coauthor of an influential text on mixed-effects models) addressed several of these myths including two that I encounter frequently: “the FDA will only accept SAS submissions” and “Unlike SAS, R code can not be validated”. I was surprised that Dr Pinheiro  felt compelled to address these untruths, but I suppose I should not have been. Six years isn’t much time for ideas floated at useR conferences to have much of an impact on statisticians who have been using SAS all of their working careers. One can easily imagine that in an industry as conservative as pharma even informed people might ignore information that would perturb what is perceived to be a successful, established pattern. The twin pressures of not wanting to disturb a process that has worked successfully in the past and the fear that even small changes could contribute to additional FDA scrutiny are powerful forces to working to preserve the status quo.

However, according to Dr Pinheiro not only has R made its way into drug development, R is probably used more than SAS. (Apparently, R based simulations have become a good part of the daily work of designing drugs.) Nevertheless, R is very rarely used in submissions. The insight I got from Dr Pinheiro's presentation and the conversation at the JSM session is that the daily simulations are a relatively new task in the drug development process. Therefore, one would expect new tools could be applied if they were useful. But in the ultra conservative pharma environment it is unreasonable to assume that a new tool would replace the customary tool in an established process that seems to be working well. Something big has to break before the current generation of SAS trained statisticians give R a chance in the FDA submission process. The irony that R is used by FDA statisticians for all kinds of task associated with the submission process. Jyoti Rayamahi of Eli Lilly and Company, the discussant for the JSM session had the best line.  “At Lilly we do our submissions in SAS and the FDA uses R to check them”.

Note last week the R Foundation refreshed the date on the document: R: Regulatory Compliance and Validation Issues A Guidance Document for the Use of R in Regulated Clinical Trial Environments which it first published a few years age. As it states in the Introduction, the document “ …is intended to provide a reasonable consensus position on the part of the R Foundation for Statistical Computing … relative to the use of R within these regulated environments (environments in which the FDA regulates the use of statistical software) and to provide a common foundation for end users to meet their own internal standard operating procedures, documentation requirements and regulatory obligations.” It lists the pertinent regulatory documents and provides information that should be helpful to anyone whoe would like to deploy R in a regulated activity.