Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Source: https://bit.ly/3g9sNgK
Quick Overview
Exploring-Data is a place where I share easily digestible content aimed at making the wrangling and exploration of data more efficient (+fun).
Sign up Here to join the many other subscribers who also nerd out on new tips and tricks ????
And if you enjoy the post be sure to share it
TweetLet’s Dive into an Example
I’m a nerd ???? when it comes to programming (esp. in R) and love learning about new functions.
Matt Dancho, early in his Data Science Foundations with R Course, teaches students the importance of examining data before diving into wrangling and exploring it further.
One of the functions I’ve adopted from the course is the glimpse() function – it’s awesome and I use it all the time!
The glimpse() function
When your data has a small number of columns it’s easy to print + view them in the RStudio console; however, when there are many columns it’s difficult to digest the view returned.
Let’s look at an example so I can stress the value in using the dplyr::glimpse() function when examining your data.
Let’s Get Some Data
The Tidy Tuesday Project is an awesome repository of useful data for practicing your Data Wrangling skills.
We will work with the Volcano Eruptions data as a case study for using the glimpse() function for examining your data prior to further exploration.
For simplicity, we are only going to focus on the Volcano data from the repository.
# Load Libraries
library(tidyverse)
library(tidyquant)
library(rmarkdown)
library(leaflet)
library(tmap)
library(DataExplorer)
# Import Data
volcano_raw_tbl <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-05-12/volcano.csv')
Data Examination with print()
The tidyverse version of a data.frame is a tibble and when printed to the RStudio console an overview of your data is returned.
When our data has only a few columns the tibble printed to the console is awesome! The user gets a quick overview of their data e.g., data-types, column count, etc.
Let’s filter our volcano data to see what this looks like in action.
# Filter + print data print(volcano_raw_tbl[ , 0:4], n = 3) ## # A tibble: 958 x 4 ## volcano_number volcano_name primary_volcano_type last_eruption_year ## <dbl> <chr> <chr> <chr> ## 1 283001 Abu Shield(s) -6850 ## 2 355096 Acamarachi Stratovolcano Unknown ## 3 342080 Acatenango Stratovolcano(es) 1972 ## # … with 955 more rows
That’s awesome!
A clean view is returned with information that allows the user to quickly examine their data. But what about if our data has many columns as is the case with the Volcanos Data?
print(volcano_raw_tbl, n = 5) ## # A tibble: 958 x 26 ## volcano_number volcano_name primary_volcano… last_eruption_y… country region ## <dbl> <chr> <chr> <chr> <chr> <chr> ## 1 283001 Abu Shield(s) -6850 Japan Japan… ## 2 355096 Acamarachi Stratovolcano Unknown Chile South… ## 3 342080 Acatenango Stratovolcano(e… 1972 Guatem… Méxic… ## 4 213004 Acigol-Nevs… Caldera -2080 Turkey Medit… ## 5 321040 Adams Stratovolcano 950 United… Canad… ## # … with 953 more rows, and 20 more variables: subregion <chr>, latitude <dbl>, ## # longitude <dbl>, elevation <dbl>, tectonic_settings <chr>, ## # evidence_category <chr>, major_rock_1 <chr>, major_rock_2 <chr>, ## # major_rock_3 <chr>, major_rock_4 <chr>, major_rock_5 <chr>, ## # minor_rock_1 <chr>, minor_rock_2 <chr>, minor_rock_3 <chr>, ## # minor_rock_4 <chr>, minor_rock_5 <chr>, population_within_5_km <dbl>, ## # population_within_10_km <dbl>, population_within_30_km <dbl>, ## # population_within_100_km <dbl>
Printing that many columns into the console makes the process of examining the data cumbersome – the columns being wrapped show the data-type for the columns but it’s difficult to extract that info. Also, for the wrapped columns you don’t see example values.
So why not just use the tibble::view() function and open the data in the RStudio Data Window? That’s a great idea but even before we go there we really want to get a sense of our data and the associated data-types.
Data Examination with glimpse()
I recommend using the glimpse() function and using it early ????
NOTE: On a mobile device (esp. a phone) this will not look good; check it out on a laptop or desktop ????
volcano_raw_tbl %>% glimpse() ## Rows: 958 ## Columns: 26 ## $ volcano_number <dbl> 283001, 355096, 342080, 213004, 321040, 2831… ## $ volcano_name <chr> "Abu", "Acamarachi", "Acatenango", "Acigol-N… ## $ primary_volcano_type <chr> "Shield(s)", "Stratovolcano", "Stratovolcano… ## $ last_eruption_year <chr> "-6850", "Unknown", "1972", "-2080", "950", … ## $ country <chr> "Japan", "Chile", "Guatemala", "Turkey", "Un… ## $ region <chr> "Japan, Taiwan, Marianas", "South America", … ## $ subregion <chr> "Honshu", "Northern Chile, Bolivia and Argen… ## $ latitude <dbl> 34.500, -23.292, 14.501, 38.537, 46.206, 37.… ## $ longitude <dbl> 131.600, -67.618, -90.876, 34.621, -121.490,… ## $ elevation <dbl> 641, 6023, 3976, 1683, 3742, 1728, 1733, 125… ## $ tectonic_settings <chr> "Subduction zone / Continental crust (>25 km… ## $ evidence_category <chr> "Eruption Dated", "Evidence Credible", "Erup… ## $ major_rock_1 <chr> "Andesite / Basaltic Andesite", "Dacite", "A… ## $ major_rock_2 <chr> "Basalt / Picro-Basalt", "Andesite / Basalti… ## $ major_rock_3 <chr> "Dacite", " ", " ", "Basalt / Picro-Basalt",… ## $ major_rock_4 <chr> " ", " ", " ", "Andesite / Basaltic Andesite… ## $ major_rock_5 <chr> " ", " ", " ", " ", " ", " ", " ", " ", " ",… ## $ minor_rock_1 <chr> " ", " ", "Basalt / Picro-Basalt", " ", "Dac… ## $ minor_rock_2 <chr> " ", " ", " ", " ", " ", "Basalt / Picro-Bas… ## $ minor_rock_3 <chr> " ", " ", " ", " ", " ", " ", " ", "Andesite… ## $ minor_rock_4 <chr> " ", " ", " ", " ", " ", " ", " ", " ", " ",… ## $ minor_rock_5 <chr> " ", " ", " ", " ", " ", " ", " ", " ", " ",… ## $ population_within_5_km <dbl> 3597, 0, 4329, 127863, 0, 428, 101, 51, 0, 9… ## $ population_within_10_km <dbl> 9594, 7, 60730, 127863, 70, 3936, 485, 6042,… ## $ population_within_30_km <dbl> 117805, 294, 1042836, 218469, 4019, 717078, … ## $ population_within_100_km <dbl> 4071152, 9092, 7634778, 2253483, 393303, 502…
glimpse() Packs a Punch ????
This view (if on a large enough screen) is awesome!
We’ve now got a glimpse() of an incredible amount of detail to help inform our next steps in examining and exploring these data. I love that in one view we can quickly see row/column counts, each column printed cleanly, the data-type of each column, and a handful of example values for each column.
We immediately gain insights into potential issues such as those we are seeing with the empty values being spaces (e.g., " ") instead of NA values.
We also see that these data are not tidy because headers of our columns in some instances are values, not names of variables – this is the case with the rock features and the population features.
The glimpse() function allowed us to quickly examine our data – now let’s have fun!
Subsetting Data for Exploration
I’ve done GIS work before and so I’m really interested in mapping some of these data.
The population counts based on proximity to volcanoes sound really interesting so let’s start there.
volcano_raw_subset_tbl <- volcano_raw_tbl %>%
select(volcano_name, country,
contains('tude'), contains('population'))
print(volcano_raw_subset_tbl, n = 5)
## # A tibble: 958 x 8
## volcano_name country latitude longitude population_with… population_with…
## <chr> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 Abu Japan 34.5 132. 3597 9594
## 2 Acamarachi Chile -23.3 -67.6 0 7
## 3 Acatenango Guatem… 14.5 -90.9 4329 60730
## 4 Acigol-Nevs… Turkey 38.5 34.6 127863 127863
## 5 Adams United… 46.2 -121. 0 70
## # … with 953 more rows, and 2 more variables: population_within_30_km <dbl>,
## # population_within_100_km <dbl>
Get another glimpse()
We’ve reduced our data down to just 7 columns but it’s still a bit difficult to get a full view of what it contains.
Let’s use the glimpse() function again to get a better view ????
glimpse(volcano_raw_subset_tbl) ## Rows: 958 ## Columns: 8 ## $ volcano_name <chr> "Abu", "Acamarachi", "Acatenango", "Acigol-N… ## $ country <chr> "Japan", "Chile", "Guatemala", "Turkey", "Un… ## $ latitude <dbl> 34.500, -23.292, 14.501, 38.537, 46.206, 37.… ## $ longitude <dbl> 131.600, -67.618, -90.876, 34.621, -121.490,… ## $ population_within_5_km <dbl> 3597, 0, 4329, 127863, 0, 428, 101, 51, 0, 9… ## $ population_within_10_km <dbl> 9594, 7, 60730, 127863, 70, 3936, 485, 6042,… ## $ population_within_30_km <dbl> 117805, 294, 1042836, 218469, 4019, 717078, … ## $ population_within_100_km <dbl> 4071152, 9092, 7634778, 2253483, 393303, 502…
Data Examination Cont.
Before exploring this data further, let’s examine it a little closer. Let’s see if there are any missing data in our subset of columns
Any Missing Records?
volcano_raw_subset_tbl %>%
DataExplorer::profile_missing() %>%
arrange(desc(num_missing)) %>%
rmarkdown::paged_table()
Nothing missing, that’s a good sign. And based on our earlier glimpse() all of the data-types looked good too.
Volcano Counts by Country: Top 5
Let’s see which countries have the most volcanoes.
volcano_raw_subset_tbl %>%
count(country, sort = T) %>%
top_n(5) %>% rmarkdown::paged_table()
I’m shocked that the United States has the most (in this data-set). While that’s interesting, a quick Google-search search shows that Indonesia is high in the ranks for countries with the most active volcanoes.
Let’s focus on Indonesia.
Data Wrangling
Let’s wrangle these data and transform them from a wide to long format – Tidy-Data ????
# wrangle + tidy data
indonesia_tidy_tbl <- volcano_raw_subset_tbl %>%
# filter to country of interest
filter(country == 'Indonesia') %>%
# pivot data into 'tidy' format
pivot_longer(cols = contains('pop'),
names_to = 'distance_km',
values_to = 'population') %>%
# reorder columns
select(volcano_name, country, distance_km,
population, everything())
# view tidy data
indonesia_tidy_tbl %>%
# format table
rmarkdown::paged_table(options = list(rows.print = 5))
Our tidy-data table is coming together but it still has a glaring issue: there are text combined with numeric values in our distance_km column.
I’ve got just the trick for that: readr::parse_number()
Another Awesome Function
The readr::parse_number() function extracts numeric values from text – Simply Awesome!
# tidy up the distance feature
indonesia_tidy_tbl <- indonesia_tidy_tbl %>%
mutate(distance_km = parse_number(distance_km))
# format and view table
indonesia_tidy_tbl %>% paged_table(list(rows.print = 5))
Am I the only one who geeks out when they learn new functions?
Now that we’ve got our data wrangled let’s see if we can generate a quick map. I’ve never done this in R and so this ought to be fun ????
Map Indonesia Volcanoes
Let’s take a look at all volcanoes that have more than 10,000 people living within 5km (~3mi).
Click on a Volcano to see its name and population ????
indonesia_tidy_tbl %>%
# filter data
filter(country == 'Indonesia', distance_km == 5,
population > 10000) %>%
# build text for popup
mutate(popup_text = str_glue(
"<b>Volcano Name:</b> {volcano_name}</p>
<b>Population:</b> {scales::comma(population)}")) %>%
# build awesome volcano map
leaflet(height = 300) %>%
addTiles() %>%
addCircleMarkers(
lat = ~ latitude,
lng = ~ longitude,
radius = 5, color = 'red',
stroke = F,
fillOpacity = 0.5,
popup = ~ popup_text) # hover over to activate
In these data there are 28 volcanoes that have more than 10,000 people living within 3 miles.
I wonder how many of those volcanoes are active. At UC Davis I actually took a class on volcanoes and remember that cities living near active volcanoes have extensive evacuation plans.
Wrap-Up
That’s it for today!
We used glimpse() to quickly examine our data prior to Wrangling and Exploring it further.
Get the code here: Github Repo.
Subscribe + Share
Stay tuned for more posts on Wrangling + Exploring data.
Enter your Email Here to get the latest from Exploring-Data in your inbox.
PS: Be Kind and Tidy your Data ????
Learn R Fast ????
Interested in expediting your R learning path?
Head on over to Business Science and join me on the journey.
Link to my favorite R course: Data Science for Business 101
FREE Jumpstart Data-Science Course (opened for a limited time)
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.