V3 Station mash up

Posted on October 3, 2010 by Steven Mosher in R bloggers | 0 Comments

[This article was first published on Steven Mosher's Blog, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Hmm, lets see how a fusion table works with map maker. The process works like this. We read in the GHCN inventory file into R. ( see code below ) We then clean it up ( the names fields and missing values) and we output it to a *csv file. Then open the csv in excell and save it as XLS. ( I can avoid this step in the future by exporting xls direct ) Anyways, when you have an xls, you import that into a Fusion table.

http://tables.googlelabs.com/DataSource?dsrcid=267862

Then we can select or filter for Mountain valleys and export the KLM link:

View Larger Map


url_GhcnV3 <- "ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/ghcnm.latest.qcu.tar.gz"

v3Widths <- c(11,9,10,7,31,5,1,5,2,2,2,2,1,2,16,1)

v3Names <- c("Id","Lat","Lon","Altitude","Name","GridEl","Rural","Population",
"Topography","Vegetation","Coastal","DistanceToCoast","Airport",
"DistanceToTown","NDVI","Light_Code")

# then a bunch of utility code to download the  *gz, gunzip it, untar it and grep for the

# inventory file name. or you can unzip it by hand and just copy down the file name. When the whole

# process is automated i'll post complete code. Below find the function to clean the data up. It returns

# a data.frame which you merely write out as a  table or csv file.

setUpV3Inv <- function(filename){

ghcn_inventory=read.fwf(filename,widths=v3Widths,comment.char="",col.names=v3Names)

##################################################
# strip the commas out of names
# strip commas from other feilds
# write in  NA for -9 Values and -999 values
# Use True/False for airports
##################################################
ghcn_inventory$Name<-gsub(",","",ghcn_inventory$Name);
ghcn_inventory$Name<-gsub("/"," ",ghcn_inventory$Name)
ghcn_inventory$Name<-gsub("[(]"," ",ghcn_inventory$Name)
ghcn_inventory$Name<-gsub("[)]"," ",ghcn_inventory$Name)
ghcn_inventory$Name<-gsub("[']"," ",ghcn_inventory$Name)
ghcn_inventory$Name<-gsub("[.]"," ",ghcn_inventory$Name)
ghcn_inventory$Name<-gsub("[&]"," ",ghcn_inventory$Name)
ghcn_inventory$Name<-gsub("  ","",ghcn_inventory$Name)
ghcn_inventory$NDVI<-gsub(",","",ghcn_inventory$NDVI)
ghcn_inventory$NDVI<-gsub("[.]"," ",ghcn_inventory$NDVI)
ghcn_inventory$NDVI<-gsub("/"," ",ghcn_inventory$NDVI)
ghcn_inventory$Altitude<-gsub("-999","NA",ghcn_inventory$Altitude)
ghcn_inventory$Population<-gsub("-9","NA",ghcn_inventory$Population)
ghcn_inventory$Vegetation<-gsub("xx","NA",ghcn_inventory$Vegetation)
ghcn_inventory$DistanceToCoast<-gsub("-9","NA",ghcn_inventory$DistanceToCoast)
ghcn_inventory$Airport<-gsub("A","TRUE",ghcn_inventory$Airport)
ghcn_inventory$Airport<-gsub("x","FALSE",ghcn_inventory$Airport)
ghcn_inventory$DistanceToTown<-gsub("-9","NA",ghcn_inventory$DistanceToTown)

return(ghcn_inventory)

}