New arXiv working paper introducing conditional method agreement trees (COAT) which can capture the dependency of a Bland-Altman analysis on covariates. It is ccompanied by an R implementation in the CRAN package coat.

Citation

Siranush Karapetyan, Achim Zeileis, André Henriksen, Alexander Hapfelmeier (2023). “Tree Models for Assessing Covariate-Dependent Method Agreement.” arXiv.org E-Print Archive arXiv:2306.04456 [stat.ME]. doi:10.48550/arXiv.2306.04456

Abstract

Method comparison studies explore the agreement of measurements made by two or more methods. Commonly, agreement is evaluated by the well-established Bland-Altman analysis. However, the underlying assumption is that differences between measurements are identically distributed for all observational units and in all application settings. We introduce the concept of conditional method agreement and propose a respective modeling approach to alleviate this constraint. Therefore, the Bland-Altman analysis is embedded in the framework of recursive partitioning to explicitly define subgroups with heterogeneous agreement in dependence of covariates in an exploratory analysis. Three different modeling approaches, conditional inference trees with an appropriate transformation of the modeled differences (CTreeTrafo), distributional regression trees (DistTree), and model-based trees (MOB) are considered. The performance of these models is evaluated in terms of type-I error probability and power in several simulation studies. Further, the adjusted rand index (ARI) is used to quantify the models’ ability to uncover given subgroups. An application example to real data of accelerometer device measurements is used to demonstrate the applicability. Additionally, a two-sample Bland-Altman test is proposed for exploratory or confirmatory hypothesis testing of differences in agreement between subgroups. Results indicate that all models were able to detect given subgroups with high accuracy as the sample size increased. Relevant covariates that may affect agreement could be detected in the application to accelerometer data. We conclude that conditional method agreement trees (COAT) enable the exploratory analysis of method agreement in dependence of covariates and the respective exploratory or confirmatory hypothesis testing of group differences. It is made publicly available through the R package coat.

Read full paper ›

Links

R package: https://CRAN.R-project.org/package=coat

Presentation slides: Psychoco 2023

Illustration

The paper presents an illustration in which measurements of activity energy expenditure (in 24 hours) from two different accelerometers (ActiGraph vs. Actiheart) are compared and their dependence on age, gender, weight, etc. is assessed. As the data is not freely available, we show below another illustration taken from the MethComp package.

The scint data provides measurements of the relative kidney function (renal function, percent of total) for 111 patients. The reference method is DMSA static scintigraphy and it is compared here with DTPA dynamic scintigraphy. The question we aim to answer using the new COAT method is:

Does the agreement between DTPA and DMSA depend on the age and/or the gender of the patient?

First, the package and data are loaded and reshaped to wide format:

library("coat")
data("scint", package = "MethComp")
scint_wide <- reshape(scint, v.names = "y",
  timevar = "meth", idvar = "item", direction = "wide")

Then, COAT can be applied using the coat() function, by default leveraging ctree() from the partykit in the background:

tr1 <- coat(y.DTPA + y.DMSA ~ age + sex, data = scint_wide)
print(tr1)
## Conditional method agreement tree (COAT)
## 
## Model formula:
## y.DTPA + y.DMSA ~ age + sex
## 
## Fitted party:
## [1] root
## |   [2] age <= 35: Bias = -0.49, SD =  3.42
## |   [3] age > 35: Bias = 0.25, SD = 7.04
## 
## Number of inner nodes:    1
## Number of terminal nodes: 2

This shows that the measurement differences between the two scintigraphies vary clearly between young and old patients. While the average difference between the measurements (bias) is close to zero for both age groups, the corresponding standard deviation (SD) is substantially larger (and hence the limits of agreement wider) for the older subgroup. This is better brought out graphically by the corresponding tree display with the classical Bland-Altman plots in the terminal nodes.

plot(tr1)

As the Bland-Altman plot for the older subgroup suggests that the bias between the methods may also depend on the mean measurement, we fit a second COAT tree. In addition to age and gender we also include the mean renal function measurement from DTPA and DMSA as a third potential split variable.

tr2 <- coat(y.DTPA + y.DMSA ~ age + sex, data = scint_wide, means = TRUE)
print(tr2)
## Conditional method agreement tree (COAT)
## 
## Model formula:
## y.DTPA + y.DMSA ~ age + sex
## 
## Fitted party:
## [1] root
## |   [2] means(y.DTPA, y.DMSA) <= 31: Bias = 4.80, SD = 6.61
## |   [3] means(y.DTPA, y.DMSA) > 31
## |   |   [4] means(y.DTPA, y.DMSA) <= 53.5: Bias = -0.38, SD =  3.33
## |   |   [5] means(y.DTPA, y.DMSA) > 53.5: Bias = -4.27, SD =  3.90
## 
## Number of inner nodes:    2
## Number of terminal nodes: 3
plot(tr2)

This tree reveals three subgroups where only the middle group (with renal function between 31 and 53.5 percent) has both small bias and standard deviation for the scintigraphy differences while for the other two subgroups bias and/or standard deviation are larger.