With the publication of Hansen2010 forthcoming it is critical to examine the subject afresh. The global temperature index product from NASA is known as GISSTEMP .GISSTEMP, like the temperature index from Hadley/CRU and NCDC attempts to estimate the average temperature of the globe using historical data archived in the GHCN ( Global Climate Historical Network) as well as some other other sources, namely SCAR and USHCN. Moshtemp also compiles an average using  GHCN data and a method derived from CRU.

One issue that generates a fair amount of controversy is the issue of UHI or the Urban Heat Island. As climate science tells us Urban areas tend to be warmer than non urban areas. The physical reasons behind this are related to the geometry of urban areas, the material properties of urban areas, and human activity. Briefly, the geometry of the urban landscape, buildings, influence the amount of sun that strikes the surface, the clear view of the sky at night ( radiation back to the sky) and it alters the airflow over the surface which impacts the  transport of heat skyward. The geometry can also lessen the UHI, for example, if we consider aspect such as tree canopies which cool the urban environment. According to that study the second most important regressor after canopy cover was building height. Canopy cover created “cool parks” while building height was instrumental in creating hot zones. The importance of building height is well established in the literature for some time

And the relationship here not simply a statistical correlation but rather has a physical explanation.Buildings change the flow of air  and they change the way heat is transported
“Air traversing rough warm urban areas thus encounters changes in surface and
atmospheric characteristics that alter existing mean and turbulent wind velocity fields
(Craig 2002).  In particular, UHIs cause pressure field deformations, buildings produce
barrier effects, and large urban surface roughness length values increase surface frictional
drag.  Bornstein and Johnson (1977) showed that wind speeds over NYC were decreased
below (increased above) observed rural values during high (low) regional-speed periods.
Loose and Bornstein (1977), and Gaffen and Bornstein (1988) also showed that NYC re-
tards synoptic fronts as they pass over the city during non-UHI periods, while such fronts
were accelerated over the city during periods of well-developed UHIs; similar effects on
sea breeze movement over NYC were found by Bornstein and Thompson (1981).
Urban areas also alter wind directions, e.g., in otherwise calm synoptic flows, UHIs
produce inward directed “country breezes” that result in speed maxima and confluence
over the city.  With non-zero synoptic flows, the UHI is advected downwind and urban- induced flows are superimposed onto prevailing winds, the combination of which results
in displacement of urban effects to the downwind urban edge.
During non-UHI periods, building-barrier induced diffluence at the upwind urban
edge divides regional flows as they approach a city, especially during stable nocturnal
conditions (Bornstein and LeRoy 1990).  The diffluence is followed by downstream cyc-
lonic-turning on the right-hand urban edge (looking downstream) and anticyclonic turn-
ing on the left hand urban edge (Bornstein et al. 1993).  Barriers also produce confluence
at both lateral urban edges (where air deflected around the city converges with the undis-
turbed prevailing flow) and downwind of the city (where the flow re-unites).  These hori-
zontal convergence/divergence effects produce urban vertical velocity patterns consistent
with 3-D mass consistency and are larger during unstable daytime hours than stable
nighttime hours. “

(MODELING THE EFFECTS OF LAND-USE/LAND-COVER MODIFICATIONS ON
THE URBAN HEAT ISLAND PHENOMENA IN HOUSTON, TEXAS,2006 )

My intention here is not to explain the phenomena of UHI, suffice it to say that the existence of UHi is generally accepted and studied in detail. The effect is not uniform as this crosssection of Budapest indicates:

The effect can be modulated by wind speed, cloudiness, and precipitation. In addition, Oke suggests that the effect is modulated by the wetness of the surrounding rural environment. Lastly, the effect is  a function of the number of people. In general, more people means more buildings, more water taken from surrounding rural areas, more economic activity, more surfaces that retain more heat. Consequently, some view population as a good proxy for the actual physical differences that cause an urban environment to be warmer. However,

“The use of urban population as a predictor of the amount of UHI bias has been studied (Oke 1973; Oke 1982; Karl et al. 1988). Oke (1973) examined relationships between the urban and rural temperature bias and population statistics of North American and European cities and identified two different relationships for the two continents. Other limitations to the use of population statistics as an estimator of the UHI bias include the lack of globally consistent statistics. Additionally, population data given for a geographically arbitrary boundary can be difficult to relate to the population in the immediate vicinity of a weather station. Thus, population alone does not appear to be a globally applicable method for evaluating and removing the UHI bias.”

(The Use of NOAA AVHRR Data for Assessment of the Urban Heat Island Effect, Gallo, Karl, et al)

Nevertheless, Gisstemp uses  a proxy of population, nightlights, to categorize temperature stations into catagories of urban and rural throughout the world. Until recently Gisstemp had just used Nightlights for the US stations and population for the ROW. That has changed, they now use nightlights for the entire globe

The Nightlights data are used as a proxy for population and population is used as a proxy for the actual physical changes that cause UHI. For present purposes we will accept this notion and our question will be instead “how accurate are nightlights and how well can they separate the urban from the rural. This is simply a data accuracy issue and an uncertainty issue. Namely, when GISS use Nightlights to classify a station as “lit” is that determination error free? and if there are errors how widespread are they and finally what if anything do these errors mean in the determination of a global temperature.

For the purposes of this exercise we will accept the data that NASA relies on and we will accept their classification scheme. The accuracy of that radiance data is not examined here. Further, NASA defines the following categories: a radiance value of 0-10 is defined as rural,11-35 as periurban, and above 35 is defined as urban. In Giss code, rural stations are used to adjust periurban and urban stations. That adjustment will not be discussed. The very narrow question we will address is this:  Given a station at latitude X and longitude Y, how accurate is the giss assessment that a station is rural or not. The first part of this question is how accurate are the station locations. Giss station locations are found in the v2.inv file, which is derived from a similar file at GHCN. There are over 7000 stations in that file with and each has an associated altitude and longitude,as seen below in an example from greenland. the station at GODTHAB NUUK

View Larger Map

That station is listed as being at Lat: 64.17 Lon: -51.75. In actuality the site is here 64.191852,-51.675487. The error here appears small. .02 degrees in latitude and .08 degrees in longitude.

View Larger Map

Taking a larger view we can see that the location error is on the order of several km. Roughly, at the equator a degree is 111km and so .1 degrees will be on the order of 10km ( this varies with latitude). A cursory look at the stations in this way quickly reveals that the errors can be as large as .5degrees.  Nasa have been advised or these errors and they appear to be ignoring them. As Peter details here in his exchange with Jim hansen. Hansen’s claim that the station data is accurate to within .01 degrees is clearly wrong as 5 minutes with the station data and google earth will attest. This station in the inventory is a good case.  For more cases see Peter’s post above

 

This story is repeated over the ROW. In the USA, however, the station data is reported down to four decimal degrees. While there are still inaccuracies with that data, I’ll focus of the ROW.

So lets set the stage by stating the problem. The nightlights data is collected so that every pixel represents 2.7km of the earths surface. This data is then projected at 1km. Generally you can think of that as 1/100th of a degree. The station location data on the other hand is far less accurate, with errors up to .5deg or more. Simply, a station that says it is at 0.0,0.0 could actually be at .5,0. Now if nightlights are uniform over big patches, we have less of a problem. If every pixel with .5 degrees of a station is lit with the same light, then station location accuracy is not that critical. But if brite pixels and dark pixels are intertwined then the location aliasing is more critical.

In the posts that follow I will examine this location question in detail. The code for examining every station in detail over the whole globe is done, and we will start to plow through the data.

To the code:

The data sources. the station data and the nightlights data.

url_radianceCalibrated <- "ftp://ftp.ngdc.noaa.gov/DMSP/web_data/rad_cal/rad_cal.tar" url_Giss_Inv <-"ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/GISTEMP_sources/STEP1/input_files/v2.inv" download.GissInv<-function(url=url_Giss_Inv){ if(!file.exists(basename(url)))download.file(url,basename(url),mode="wb") gissInvSetUp() } download.Nightlights <- function(dir=DOWNLOADS_DIR,url=url_radianceCalibrated){ dzip <-file.path(DOWNLOADS_DIR,basename(url),fsep=.Platform$file.sep) if(!file.exists(dzip))download.file(url,dzip,mode="wb") fname <- file.path(DOWNLOADS_DIR,basename(url),fsep=.Platform$file.sep) untar(fname,exdir=DOWNLOADS_DIR) gzname <- file.path(DOWNLOADS_DIR,NightlightsGz,fsep=.Platform$file.sep) gunzip(gzname,overwrite=TRUE) tifFile <-file.path(DOWNLOADS_DIR,hiResNightlights,fsep=.Platform$file.sep) file.copy(tifFile, hiResNightlights ) }