What will happen if I change this a little— introducing ArenaR 0.2.0

Arena is a modern tool for exploring and comparing machine learning models. It delivers explainable AI, model performance, and data analysis charts to x-ray your models from different perspectives.

A new version of Arena’s connector for R just landed on CRAN. It brings a bunch of new plots and features.

Attributes and modifying observations

Prior to this release pushed observations were only used to calculate local explanations. To distinct them, you have had to set descriptive row names. Now thanks to the developed mechanism of attributes Arena has a panel showing details of each observation. Moreover, in live mode, you can modify them and watch how charts are changing.

Keep in mind, that push_observations method can take not only data set with required columns to make a prediction. You can pass more auxiliary variables to accelerate the exploration of a single observation. Take a look at three examples of FIFA Players.

The first four variables are not editable because they are not used by the models. In the static mode modifying is not possible and all variables look like them.

Attributes are now enabled by default. To disable them or to disable custom observations use these arguments

create_arena(
    enable_attributes = FALSE, # disable attributes at all
    enable_custom_params = FALSE # disable only modifying observations
)

Fairness

Fairness is quickly gaining attention within ML community, therefore there is nothing surprising that now Arena supports various fairness metrics. It supports some of the most popular features of R package fairmodels. You can now compare fairness metrics for different models, and detect which one is biased.

There are two types of plots implemented:
Fairness Check — core functionality of fairmodels. Basically, the lollipops should not reach read zones. The longer one gets the bigger the bias. If a lollipop is leaning towards the left red zone than it means that there is a bias against unprivileged subgroups and the contrary happens when it is reaching the right red zone.
Metric Scores — sometimes it is necessary to know the actual values of metrics. Not just to assert yourself of the unfairness but to better understand the ratios of the metrics. Especially when dealing with small scores, metrics might have really big values.

But there is more! Arena’s interactive interface allows for changing the privileged subgroup and adjusting cutoff to your needs! In Arena there is no area for biased models.

Exploratory data analysis

New category of plots comes to the Arena. We believe that simple data analysis should give you next perspective on what models have learned.

Two kinds of plots are now available. Distribution of variables and contrasting two variables. The second kind visible below can be scatter plot, box plot, or heat map depending on variables’ types.

To make use of just introduced charts you need to add one line to your arenar pipe.

arena <- create_arena() %>%
    push_model(model1) %>%
    push_model(model2) %>%
    push_model(model3) %>%
    push_observations(observations) %>%
    # Add dataset for EDA charts
    push_dataset(data_frame, target="m2.price", label="Dataset")

The first argument is the data frame containing the target variable (its name is set in the next argument). We suggest not to push data set with more variables than these used for training. However, all numerical and categorical variables will work.

Check model performance on the subset

Although the funnel plot was available for two months, let me characterize it with just released Subset Performance plot. Both are similar in construction, but the usage is different.
The key point behind them is to split the testing dataset based on subsets of a variable. For example, for Age, we can create subsets :

[0, 5), [5, 10), [10, 20), [20, 40), [40, 100)

In each group, we can now calculate different metrics like MSE or Accuracy.

Subset Performance shows the raw values of the selected measure for each subset. Lollipops start in the lines representing the value of the metric for the whole dataset. You can check on which observations your model performs better or worse.
Funnel Plot is a little different because you need to have some primary model. Performance in subsets is presented in contrast to the performance of the primary model. This chart is useful when you want to test white-box models in contrast to some high-performance black box.

Enjoy new features

# stable version
install.packages('arenar')
# latest version
remotes::install_github('ModelOriented/Arenar')

If you are interested in other posts about explainable, fair, and responsible ML, follow #ResponsibleML on Medium.

In order to see more R related content visit https://www.r-bloggers.com

What will happen if I change this a little— introducing ArenaR 0.2.0 was originally published in ResponsibleML on Medium, where people are continuing the conversation by highlighting and responding to this story.