How to Handle Missing Data

This article is part of a R-Tips Weekly, a weekly video tutorial that shows you step-by-step how to do common R coding tasks.

Missing values used to drive me nuts… until I learned how to impute them! In 10-minutes, learn how to visualize and impute in R using ggplot dplyr and 3 more packages to simple imputation.

Here are the links to get set up. ????

• Get the Code
• YouTube Tutorial

Handling missing values

We’re going to kick the tires on 3 key packages:

• visdat – For quickly visualizing data
• naniar – For working with NA’s (missing data)
• simputation – For simple imputation (converting missing data to values)

So let’s get started!

Visualizing Missing Data
Using vis_miss(), gg_miss_upset() and geom_miss_point()

Quickly Skim Missing Data

It doesn’t get any easier than this. Simply use visdat::vis_miss() to visualize the missing data. We can see Ozone and Solar.R are the offenders.

Identify Interactions in Column Missingness

Use Case: It often makes sense to evaluate the interactions between columns containing missing data. We can use an “upset” plot for this.

Start with a good question:

“Is it often that we have both Ozone and Solar.R missing at the same time?”

We can answer this with gg_miss_upset(). We can see that 2 of 5 Solar.R (40%) happen at the same observation that Ozone is missing. Might want to check for IOT sensor issues!

Visualize Missing Observations in a Scatter Plot

Use Case: This is a great before/after visual.

For our final exploratory plot, let’s plot the missing data using geom_miss_point(). It works just like geom_point(), but plots where the missing data are located in addition to the non-missing data.

Imputation
impute_rf()

The simputation library comes with a host of impute*()_ functions. We’ll focus on impute_rf(), which implements a random forest to do the imputation.

This imputes the NA’s, replacing the missing Ozone and Solar.R data. We can see the missing data follows the distribution of the non-missing data in the updated scatter plot.

Reminders:

1. Full code in the Github Repository.
2. Watch the YouTube Video for detailed instructions.

But if you really want to improve your productivity…

SETUP R-TIPS WEEKLY PROJECT

1. Sign Up to Get the R-Tips Weekly (You’ll get email notifications of NEW R-Tips as they are released): https://mailchi.mp/business-science/r-tips-newsletter

2. Set Up the GitHub Repo: https://github.com/business-science/free_r_tips

3. Check out the setup video (https://youtu.be/F7aYV0RPyD0). Or, Hit Pull in the Git Menu to get the R-Tips Code

Once you take these actions, you’ll be set up to receive R-Tips with Code every week. =)