Introducing the ‘gimms’ package
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
This is a guest post by Florian Detsch
What it is all about
With the most recent update of the AVHRR GIMMS data collection to NDVI3g (Pinzon and Tucker, 2014), we decided to create a package from all functions we have written so far to download and process GIMMS binary files from the NASA ECOCAST server. The package is called gimms and features a collection of fundamental work steps required to get the data into R:
updateInventory
to list all GIMMS files available online andrearrangeFiles
to sort (online or local) files by date,downloadGimms
to download selected files,rasterizeGimms
to import the binary data as 'Raster*' objects into R andmonthlyComposite
to aggregate the bi-monthly datasets to monthly value
composites.
How to install
The gimms package (version 0.1.1) is now officially on CRAN and can be installed directly via
## install 'gimms' package install.packages("gimms") ## load 'gimms' package library(gimms)
In order to use the development version (no liability assumed), please refer to the 'develop' branch hosted at GitHub. There, you will also find the latest news and updates concerning the package.
library(devtools) install_github("environmentalinformatics-marburg/gimms", ref = "develop")
List available files
updateInventory
imports the latest version of the online file inventory as 'character' vector into R. By setting sort = TRUE
, it is at the same time a wrapper around rearrangeFiles
as the output vector will be sorted by date rather than in alphabetical order. The latter feature proves particularly useful when considering the GIMMS file naming convention, where e.g. 'geo13jul15a.n19-VI3g' means the first half of July 2013. In case no active internet connection is available, updateInventory
automatically imports the latest offline version of the file inventory.
gimms_files <- updateInventory(sort = TRUE) ## Trying to update GIMMS inventory from server... ## Online update of the GIMMS file inventory successful! gimms_files[1:5] ## [1] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81jul15a.n07-VI3g" ## [2] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81jul15b.n07-VI3g" ## [3] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81aug15a.n07-VI3g" ## [4] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81aug15b.n07-VI3g" ## [5] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81sep15a.n07-VI3g"
Download files
The next logical step of the gimms processing chain is to download selected (if not all) bi-monthly datasets. This can be achieved by running downloadGimms
which accepts various types of input parameters.
- 'missing' input → download entire collection
Specifying no particular input is possibly the most straightforward way of data acquisition. The function will automatically start to download the entire collection of files (currently July 1981 to December 2013) and write the data todsn
.
## download entire gimms collection downloadGimms(dsn = paste0(getwd(), "/data"))
- 'numeric' input → download temporal range
It is also possibly to specify a start year (x
) and/or end year (y
) to limit the temporal coverage of the datasets to be downloaded. In casex
(ory
) is missing, data download will automatically start from the first (or finish with the last) year available.
## download gimms data from 1998-2000 downloadGimms(x = 1998, y = 2000, dsn = paste0(getwd(), "/data"))
- 'character' input → download particular files
As a third and final possibility to rundownloadGimms
, it is also possible to supply a 'character' vector consisting of valid online filepaths. The latter can easily be retrieved fromupdateInventory
(as demonstrated above) and directly passed on to the input argumentx
.
## download manually selected files downloadGimms(x = gimms_files[769:780], dsn = paste0(getwd(), "/data"))
Rasterize downloaded data
rasterizeGimms
transforms the retrieved GIMMS data from native binary into common 'GeoTiff' format and makes the single layers available in R as ordinary 'Raster*' objects. Thereby, it is up to the user to decide whether or not to discard 'mask-water' values (-10,000) and 'mask-nodata' values (-5,000) (see also the official NDVI3g README) and apply the scaling factor (1/10,000). Since rasterizing usually takes some time, we highly recommend to make use of the filename
argument that automatically invokes raster::writeRaster
.
## list available files gimms_files <- rearrangeFiles(dsn = paste0(getwd(), "/data"), pattern = "^geo13", full.names = TRUE) ## rasterize files gimms_raster <- rasterizeGimms(gimms_files, filename = paste0(gimms_files, ".tif"))
With a little bit of effort and the help of RColorBrewer and sp, here is what we have created so far.
Figure 1.Global bi-monthly GIMMS NDVI3g images from July to December 2013.
Generate monthly composites
Sometimes, the user is required to calculate monthly value composites from the bi-monthly GIMMS datasets, e.g. to ensure temporal overlap with some other remote sensing product. For that purpose, gimms also features a function called monthlyComposite
which works both on vectors of filenames and entire 'RasterStack' objects (ideally returned by rasterizeGimms
) and calculates monthly values based on a user-defined function (e.g. fun = max
to create monthly MVC layers). The function is heavily based on stackApply
from the raster package and the required coding work is quite straightforward.
## 'GeoTiff' files created during the previous step gimms_files_tif <- sapply(gimms_raster@layers, function(i) attr(i@file, "name")) ## create monthly maximum value composites gimms_raster_mvc <- monthlyComposite(gimms_files_tif)
Figure 2.Global monthly composite GIMMS NDVI3g images from July to December 2013.
Final remark
A more comprehensive version of this short introduction to the gimms package including a collection of use cases (particularly in conjunction with R's parallel capabilities) can be found online at https://github.com/environmentalinformatics-marburg/gimms. Any comments on how to improve the package, possible bug-reports etc. are highly appreciated!
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.