New package: GetQuandlData
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Introduction
Quandl is one of the best platforms for finding and downloading financial and economic time series. The collection of free databases is solid and I’ve used it intensively in my research and class material.
But, a couple of things from the native package Quandl
always bothered me:
- Multiple data is always returned in the wide (column oriented) format (why??);
- No local caching of data;
- No control for importing error and status;
- Not easy to work within the
tidyverse
collection of packages
As you suspect, I decided to tackle the problem over the weekend. The result is package GetQuandlData
. This is what it does differently:
- It uses the json api (and not the Quandl native function), so that some metadata is also returned;
- The resulting dataframe is always returned in the long format, even for multiple series;
- Users can set custom names for input series. This is very useful when using along
ggplot
or making tables; - Uses package
memoise
to set a local caching system. This means that the second time you ask for a particular time series, it will grab it from your hard drive (and not the internet); - Always compares the requested dates against dates available in the platform.
Installation
# not in CRAN yet (need to test it further) #install.packages('GetQuandlData') # from github devtools::install_github('msperlin/GetQuandlData')
Example 01 – Inflation in the US
Let’s download and plot information about inflation in the US:
library(GetQuandlData) library(tidyverse) my_id <- c('Inflation USA' = 'RATEINF/INFLATION_USA') my_api <- readLines('~/Dropbox/.quandl_api.txt') # you need your own API (get it at https://www.quandl.com/sign-up-modal?defaultModal=showSignUp>) first_date <- '2000-01-01' last_date <- Sys.Date() df <- get_Quandl_series(id_in = my_id, api_key = my_api, first_date = first_date, last_date = last_date, cache_folder = tempdir()) glimpse(df) ## Observations: 236 ## Variables: 4 ## $ series_name <chr> "Inflation USA", "Inflation USA", "Inflation USA", "… ## $ ref_date <date> 2019-08-31, 2019-07-31, 2019-06-30, 2019-05-31, 201… ## $ value <dbl> 1.750, 1.811, 1.648, 1.790, 1.996, 1.863, 1.520, 1.5… ## $ id_quandl <chr> "RATEINF/INFLATION_USA", "RATEINF/INFLATION_USA", "R…
As you can see, the data is in the long format. Let’s plot it:
p <- ggplot(df, aes(x = ref_date, y = value/100)) + geom_col() + labs(y = 'Inflation (%)', x = '', title = 'Inflation in the US') + scale_y_continuous(labels = scales::percent) p
Beautiful!
Example 02 - Inflation for many countries
Next, lets have a look into a more realistic case, where we need inflation data for several countries:
First, we need to see what are the available datasets from database RATEINF
:
library(GetQuandlData) library(tidyverse) db_id <- 'RATEINF' my_api <- readLines('~/Dropbox/.quandl_api.txt') # you need your own API df <- get_database_info(db_id, my_api) knitr::kable(df)
code | name | description | refreshed_at | from_date | to_date | quandl_code | quandl_db |
---|---|---|---|---|---|---|---|
CPI_ARG | Consumer Price Index - Argentina | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:58 | 1988-01-31 | 2013-12-31 | RATEINF/CPI_ARG | RATEINF |
CPI_AUS | Consumer Price Index - Australia | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1948-09-30 | 2019-06-30 | RATEINF/CPI_AUS | RATEINF |
CPI_CAN | Consumer Price Index - Canada | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1989-01-31 | 2019-08-31 | RATEINF/CPI_CAN | RATEINF |
CPI_CHE | Consumer Price Index - Switzerland | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:58 | 1983-01-31 | 2019-08-31 | RATEINF/CPI_CHE | RATEINF |
CPI_DEU | Consumer Price Index - Germany | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1991-01-31 | 2019-08-31 | RATEINF/CPI_DEU | RATEINF |
CPI_EUR | Consumer Price Index - Euro Area | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1990-01-31 | 2019-08-31 | RATEINF/CPI_EUR | RATEINF |
CPI_FRA | Consumer Price Index - France | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1990-01-31 | 2019-08-31 | RATEINF/CPI_FRA | RATEINF |
CPI_GBR | Consumer Price Index - UK | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:58 | 1988-01-31 | 2019-08-31 | RATEINF/CPI_GBR | RATEINF |
CPI_ITA | Consumer Price Index - Italy | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 2001-01-31 | 2019-08-31 | RATEINF/CPI_ITA | RATEINF |
CPI_JPN | Consumer Price Index - Japan | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1970-01-31 | 2019-08-31 | RATEINF/CPI_JPN | RATEINF |
CPI_NZL | Consumer Price Index - New Zealand | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1988-03-31 | 2019-06-30 | RATEINF/CPI_NZL | RATEINF |
CPI_RUS | Consumer Price Index - Russia | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1995-01-31 | 2019-07-31 | RATEINF/CPI_RUS | RATEINF |
CPI_USA | Consumer Price Index - USA | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1913-01-31 | 2019-08-31 | RATEINF/CPI_USA | RATEINF |
INFLATION_ARG | Inflation YOY - Argentina | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:58 | 1989-01-31 | 2013-12-31 | RATEINF/INFLATION_ARG | RATEINF |
INFLATION_AUS | Inflation YOY - Australia | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1949-03-31 | 2019-06-30 | RATEINF/INFLATION_AUS | RATEINF |
INFLATION_CAN | Inflation YOY - Canada | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1990-01-31 | 2019-08-31 | RATEINF/INFLATION_CAN | RATEINF |
INFLATION_CHE | Inflation YOY - Switzerland | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1984-01-31 | 2019-08-31 | RATEINF/INFLATION_CHE | RATEINF |
INFLATION_DEU | Inflation YOY - Germany | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1992-01-31 | 2019-08-31 | RATEINF/INFLATION_DEU | RATEINF |
INFLATION_EUR | Inflation YOY - Euro Area | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1991-01-31 | 2019-08-31 | RATEINF/INFLATION_EUR | RATEINF |
INFLATION_FRA | Inflation YOY - France | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1991-01-31 | 2019-08-31 | RATEINF/INFLATION_FRA | RATEINF |
INFLATION_GBR | Inflation YOY - UK | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1989-01-31 | 2019-08-31 | RATEINF/INFLATION_GBR | RATEINF |
INFLATION_ITA | Inflation YOY - Italy | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 2002-01-31 | 2019-08-31 | RATEINF/INFLATION_ITA | RATEINF |
INFLATION_JPN | Inflation YOY - Japan | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1971-01-31 | 2019-08-31 | RATEINF/INFLATION_JPN | RATEINF |
INFLATION_NZL | Inflation YOY - New Zealand | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 2001-03-31 | 2019-06-30 | RATEINF/INFLATION_NZL | RATEINF |
INFLATION_RUS | Inflation YOY - Russia | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1996-01-31 | 2019-07-31 | RATEINF/INFLATION_RUS | RATEINF |
INFLATION_USA | Inflation YOY - USA | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2019-09-28 02:19:59 | 1914-01-31 | 2019-08-31 | RATEINF/INFLATION_USA | RATEINF |
Nice. Now we only need to filter the series with YOY inflation:
idx <- stringr::str_detect(df$name, 'Inflation YOY') df_series <- df[idx, ]
and grab the data:
my_id <- df_series$quandl_code names(my_id) <- df_series$name first_date <- '2010-01-01' last_date <- Sys.Date() df_inflation <- get_Quandl_series(id_in = my_id, api_key = my_api, first_date = first_date, last_date = last_date) glimpse(df_inflation) ## Observations: 897 ## Variables: 4 ## $ series_name <chr> "Inflation YOY - Argentina", "Inflation YOY - Argent… ## $ ref_date <date> 2013-12-31, 2013-11-30, 2013-10-31, 2013-09-30, 201… ## $ value <dbl> 10.95, 10.54, 10.55, 10.49, 10.55, 10.61, 10.46, 10.… ## $ id_quandl <chr> "RATEINF/INFLATION_ARG", "RATEINF/INFLATION_ARG", "R…
And, finally, an elegant plot:
p <- ggplot(df_inflation, aes(x = ref_date, y = value/100)) + geom_col() + labs(y = 'Inflation (%)', x = '', title = 'Inflation in the World', subtitle = paste0(first_date, ' to ', last_date)) + scale_y_continuous(labels = scales::percent) + facet_wrap(~series_name) p
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.