Adverse Events

An adverse event (AE) is any untoward medical occurrence in a patient or participating in a clincial trial. These events are not necessarily drug related. It could anything from a headache to a sporting accident to a life-threatening pneunomia. It is important though for the safety of the patients that these events are being reported in a timely manner from the clinical trial site to the sponsor of the trial so that the safety profile of the drug can be updated if necessary and appropriate actions can be taken.

Key Risk Indicators

Statistical monitoring of clinical trial sites typical employs several key risk indicators which are operational metrics derived from activities at the clinical trial sites. The ratio of AEs is typically one of them, the ratio of issues occurring when treating and examininig patients according to the study protocol might be another one.

Release 0.5.0

{simaerep} was originally developed to detect sites that are under-reporting Adverse Events (AE) in clinical trials using a nonparametric bootstrap algorithm. With the release of the most current version v0.5.0 {simaerep} can also be used to detect over-reporting. With this update the algorithm can be used to detect under and over-reporting for all patient-related clinical events at site-level. From a quality perspective under-reporting is of a greater interest than over-reporting of AE. When it comes to other events such as site and subject issues, protocol deviations and data queries over- or high-reporting can be of the same importance as under-reporting. As high reporting can be indicative of site problems when executing the study protocol, low- or under-reporting can be indicative of failure of reporting such issues. A detailed guideline on how to obtain both over and under-reporting probabilities has been added to the package documentation.

Comparison to other KRI flagging methods

The open-source landscape for R packages for clinical study quality monitoring and assurance has lately been extended with the release of the {gsm} R package which provides streamlined reporting for monitoring of key risk indicators that directly connects to various standardized clinical data formats. It also implements a funnel plot outlier flagging method which holds various advantages over percentile-based flagging methods. Nevertheless we could show that the algorithm used by simarep can flag under-reporting sites more reliably mostly because event rates are not constant over-time. This shows that the redistribution/restratification of patients to sites can be used to get more reliable flags than flagging methods that make assumptions about the underlying process distributions.

Demo

suppressPackageStartupMessages(library(simaerep))
suppressPackageStartupMessages(library(tidyverse))
suppressPackageStartupMessages(library(knitr))

set.seed(1)

df_visit <- simaerep::sim_test_data_study(
  n_pat = 1000, # number of patients in study
  n_sites = 100, # number of sites in study
  frac_site_with_ur = 0.05, # fraction of sites under-reporting
  ur_rate = 0.4, # rate of under-reporting
  ae_per_visit_mean = 0.5 # mean AE per patient visit
)

df_visit$study_id <- "A"

df_visit %>%
  select(study_id, site_number, patnum, visit, n_ae) %>%
  head(10) %>%
  knitr::kable()

aerep <- simaerep::simaerep(df_visit)

plot(aerep, study = "A")

IMPALA

simaerep has been published as workproduct of the Inter-Company Quality Analytics (IMPALA) consortium. IMPALA aims to engage with Health Authorities inspectors on defining guiding principles for the use of advanced analytics to complement, enhance and accelerate current QA practices. simaerep has initially been developed at Roche but is currently evaluated by other companies across the industry to complement their quality assurance activities (see testimonials).

Ressources

• github
• website

Publications

Koneswarakantha, B., Adyanthaya, R., Emerson, J. et al. An Open-Source R Package for Detection of Adverse Events Under-Reporting in Clinical Trials: Implementation and Validation by the IMPALA (Inter coMPany quALity Analytics) Consortium. Ther Innov Regul Sci (2024). https://doi.org/10.1007/s43441-024-00631-8

Koneswarakantha, B., Barmaz, Y., Ménard, T. et al. Follow-up on the Use of Advanced Analytics for Clinical Quality Assurance: Bootstrap Resampling to Enhance Detection of Adverse Event Under-Reporting. Drug Saf (2020).
https://doi.org/10.1007/s40264-020-01011-5