Mapping Paranormal Manifestations in the British Isles

[This article was first published on Weird Data Science, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

In our last entry we analysed the relative frequency of paranormal manifestations in the British Isles according to the Paranormal Database. The results of that analysis showed that hauntings of various forms are by far the most commonly-reported paranormal encounter in the British Isles, followed by cryptozoological sightings.

This is, however, relatively unsatisfactory. It is much more interesting to know where such sightings and events occur. Are there particular haunts of restless spirits? Do mysterious beasts roam in particular regions more than others? To answer these questions, we need to delve into the specific locations of different reports.

The Paranormal Database does contain location information, but it is given very informally. To map this we can make use of Google’s Geolocation API to convert free text strings, such as “Felbrigg Hall, Norfolk” into usable latitude and longitude coordinates. (In this case: 52.907479, 1.259443.)

The geolocation is not perfect, but with sufficient manipulation of the service it was possible to produce geolocated coordinates for most of the entries in the database. In order to represent these meaningfully, we have also subdivided the entries into a different types. The original Paranormal Database data is subdivided into twenty categories, which we have reduced to six for easier presentation. This includes collapsing the various kinds of haunting, from poltergeists to ‘post-mortem manifestions’, simply to hauntings. Similarly, we combine alien big cats and shuck into the broader family of cryptozoology.

With this in place, we can see the overall distribution of paranormal events in the British Isles.

Paranormal manifestations in the British Isles. (PDF Version.)

As might be expected, London is a dark and sinister nexus of paranormal activity. Hauntings, as might be expected from their overall frequency, dominate the majority of the British Isles. Moving north, particularly as we reach the Scottish Highlands, cryptozoological sightings begin to challenge hauntings as the most common supernatural event. We can also see significant cryptozoology in the Hebrides, Orkney, and Shetland — the archipelagos that surround the Scottish mainland.

Both Wales, Ireland, and Cornwall are significantly less densely haunted in the Paranormal Database, with the majority of sightings falling in England.

This overall view, however, combines a number of very different phenomena. Where, for example, are we most likely to receive a visitation from a restless spectre as opposed to being pursued by a savage and unnatural beast?

By breaking down the sightings into different types, and plotting a heatmap of event density over each, we can identify the regions in which different manifestations cluster.

Density plot of paranormal manifestations in the British Isles. (PDF Version.)

This view highlights several points of interest.

Firstly, London’s preeminent position is not for all forms of paranormal activity. Hauntings are extremely dense in London, however the rest of England is also well-populated. As might be expected from the first diagram, the less population-dense regions further north have produced fewer sad echoes of mortality. Despite its general reputation Edinburgh, while noticeably haunted, cannot compare with many regions of England.

Cryptozoologically, however, London is far from dominant. Whilst unknown beings may lurk in the foetid sewers of the capital, they clearly prefer the wide open spaces — both the Norfolk and Suffolk Broads are rife with cryptids, as are the Hebrides, Orkney, and Shetland that were noticeable earlier. Finally, visitors to Cornwall will pass through areas of increasing monstrous activity.

UFO’s also appear to be attracted to East Anglia most strongly, and are otherwise most common in the large population centres of England. Less obviously, there is a noticeable density of UFO activity on the Pembrokeshire Coast, in the south-west of Wales.

Monsters, which in this classification includes werewolves, vampires, and dragons, produce a surprising cluster in North Wales, around Snowdonia. The most significant monstrous sightings, however, appear to be in Exmoor; again on the south-western tip of the British Isles.

The final categories of manifestation include legends, fairies, and a catch-all category of ‘other manifestations’ that include mysterious orbs, talking trees, bleeding stones, and the supernatural impressions left by the work of John Dee. As might be expected, this last category is more uniformly distributed across the country, matching high-density population areas. There is, however, another notable cluster in Cornwall for this category.

In conclusion, then, the British Isles are teeming with paranormal activity. Entities from beyond the grave lie close at all times, with twisted monstrosities roaming the wild spaces. UFOs descend from the night sky to terrorise the coastal regions.

From this analysis, ghost hunters should concentrate on London for the best chance of a sighting, although almost any of the large centres of population provide a reasonable chance of spectral apparitions. Cryptid researchers should concentrate in East Anglia or head north to the islands beyond Scotland. Those seeking contact with extraterrestrials should focus particularly on the east coast of Suffolk, or travel to the south-west of Wales. Paranormal investigators whose interests lie in legends or monsters, or less specific strange entities, would be well-advised to visit Cornwall.

Code for the plotting elements of this analyis are given below, following on from the scraping and parsing code in our previous post. The geolocation step required a more significant effort, and will be the focus of a future code-based post.

You can keep up to date with our latest paranormal data mining on Twitter at @WeirdDataSci.

Show analysis code

Data:

Other:

Combined Plot Code:

library(spatstat)

library(rgdal)
library(maptools)

library(tidyverse)
library(magrittr)
library(ggplot2)
library(ggthemes)
library(raster)
library(viridis)
library(scales)

library(sf) 

library(showtext)

library(grid)

library(cowplot)
library(magick)

# Function to combine certain paranormal report types into broader categories
combine_paranormal_types <- function( type ) {

	# levels( paranormal_tbl$type )
	# [1] "Alien Big Cat"                "Crisis Manifestation"        
	# [3] "Cryptozoology"                "Curse"                       
	# [5] "Dragon"                       "Environmental Manifestation" 
	# [7] "Experimental Manifestation"   "Fairy"                       
	# [9] "Haunting Manifestation"       "Legend"                      
	# [11] "Manifestation of the Living"  "Other"                       
	# [13] "Poltergeist"                  "Post-Mortem Manifestation"   
	# [15] "Shuck"                        "Spontaneous Human Combustion"
	# [17] "UFO"                          "Unknown Ghost Type"          
	# [19] "Vampire"                      "Werewolf"                   

	# Simple lookup
	case_when(
				 type %in% c("Alien Big Cat", "Cryptozoology", "Shuck") ~ "Cryptozoology",
				 type %in% c("Dragon", "Vampire", "Werewolf" ) ~ "Monster",
				 type %in% c("Crisis Manifestation", "Environmental Manifestation", "Experimental Manifestation", "Haunting Manifestation", "Manifestation of the Living", "Poltergeist", "Post-Mortem Manifestation", "Unknown Ghost Type") ~ "Haunting",
				 type %in% c("Legend", "Fairy", "Curse") ~ "Legend",
				 type %in% c("Spontaneous Human Combustion", "Other") ~ "Other",
				 type %in% c("UFO") ~ "UFO"
				 )

}

# Load font
font_add( "mapfont", "/usr/share/fonts/TTF/weird/JANCIENT.TTF" )
showtext_auto()

# Read world shapefile data and tranform to an appropriate projection.
# Limit to the UK, Ireland, and the Isle of Man
world <- readOGR( dsn='data/ne/10m_cultural', layer='ne_10m_admin_0_countries' )
world_subset <- world[ world$iso_a2 %in% c("GB","IE","IM"), ]
world_subset <- spTransform(world_subset,CRS("+init=epsg:4326"))
world_df <- fortify( world_subset )

# Read paranormal database
paranormal_tbl <- as.tibble( read.csv( file="data/paranormal_database.csv" ) )

# Convert the paranormal dataframe to a spatial dataframe that contains
# explicit longitude and latitude projected appropriately for plotting.
coordinates( paranormal_tbl ) <- ~lng+lat
proj4string( paranormal_tbl )<-CRS("+init=epsg:4326")
paranormal_tbl <- spTransform(paranormal_tbl,CRS(proj4string(paranormal_tbl)))

# Restrict paranormal_tbl to those points in the polygons defined by world_subset
paranormal_tbl_rows <- paranormal_tbl %>%
	over( world_subset ) %>%
	is.na() %>%
	not() %>%
	rowSums() %>%
	`!=`(0) %>%
	which


paranormal_tbl <- as.tibble( paranormal_tbl[ paranormal_tbl_rows, ] )

paranormal_tbl$combined_type <- paranormal_tbl$type %>%
	map( combine_paranormal_types ) %>%
	unlist

# Show the map
gp <- ggplot() + 

	geom_map( data = world_df, aes( map_id=id ), colour = "#3c3f4a", fill = "transparent", size = 0.5, map = world_df )

# Display each sighting as geom_point. Use a level of transparency to highlight
# more common areas. (On the 20180401 dataset, this reports that 16035 out of
# the original 19387 points lie in the appropriate area. Several are geolocated
# outside of the UK -- the geolocation should be run again with better bounds
# checking and region preference.)
gp <- gp + geom_point(data=paranormal_tbl, aes(x=lng, y=lat, colour=combined_type ), size=0.5, shape=17, alpha=0.9) +
	expand_limits(x = world_df$long, y = world_df$lat)  

gp <- gp +	
	
	# Theming 
	theme_map() + 
	theme( 
			plot.background = element_rect(fill = "transparent", colour = "transparent"),
			panel.border = element_blank(),
			plot.title = element_text( size=24, colour="#3c3f4a", family="mapfont" ),
			text = element_text( size=14, color="#3c3f4a", family="mapfont" ),
			) +

	theme( 
			legend.background = element_rect( colour ="transparent", fill = "transparent" ), 
			legend.key = element_rect(fill = "transparent", colour="transparent"),
			legend.position = "left",
			legend.justification = c(0,0)
			) +

	guides( fill = guide_colourbar( title.position="top", direction="vertical", barwidth=32, nrow=1 ) ) +
	guides(colour = guide_legend(override.aes = list(size=2)) ) +
	scale_colour_manual( 
		values = c("#00EA38","#417CCC","#B79F00","#F564E3","#00BFC4","#F8766D"), 
		breaks = c("Haunting", "Cryptozoology", "Monster", "Legend", "UFO", "Other" ),
		name = "Manifestation" ) +
	coord_fixed( ratio=1.2 )

# Cowplot trick for ggtitle
title <- ggdraw() + 
	draw_label("Paranormal Manifestations in the British Isles", fontfamily="mapfont", colour = "#3c3f4a", size=24, hjust=0, vjust=1, x=0.02, y=0.88) +
	draw_label("http://www.weirddatascience.net | @WeirdDataSci", fontfamily="mapfont", colour = "#3c3f4a", size=14, hjust=0, vjust=1, x=0.02, y=0.40)

data_label <- ggdraw() +
	draw_label("Data from: http://www.paranormaldatabase.com", fontfamily="mapfont", colour = "#3c3f4a", size=12, hjust=1, x=0.98 ) 

paranormal_legend <- get_legend(gp)

# Remove legend from internal plot
gp <- gp + theme(legend.position='none')

tgp <- plot_grid(title, gp, data_label, ncol=1, rel_heights=c(0.1, 1, 0.1)) 

vellum_plot <- ggdraw() +
	draw_image("img/vellum.jpg", scale=1.4 ) +
	draw_plot(tgp) +
	draw_plot( paranormal_legend, 0.02, 0.03 )


save_plot("output/paranormal.pdf", 
			vellum_plot,
			base_width = 16,
			base_height = 9,
			base_aspect_ratio = 1.78 )

Per-Manifestation Density Plot Code:

library(spatstat)

library(rgdal)
library(maptools)

library(tidyverse)
library(magrittr)
library(ggplot2)
library(ggthemes)
library(raster)
library(viridis)
library(scales)

library(sf) 

library(showtext)

library(grid)

library(cowplot)
library(magick)

# Function to combine certain paranormal report types into broader categories
combine_paranormal_types <- function( type ) {

	# levels( paranormal_tbl$type )
	# [1] "Alien Big Cat"                "Crisis Manifestation"        
	# [3] "Cryptozoology"                "Curse"                       
	# [5] "Dragon"                       "Environmental Manifestation" 
	# [7] "Experimental Manifestation"   "Fairy"                       
	# [9] "Haunting Manifestation"       "Legend"                      
	# [11] "Manifestation of the Living"  "Other"                       
	# [13] "Poltergeist"                  "Post-Mortem Manifestation"   
	# [15] "Shuck"                        "Spontaneous Human Combustion"
	# [17] "UFO"                          "Unknown Ghost Type"          
	# [19] "Vampire"                      "Werewolf"                   

	# Simple lookup
	case_when(
				 type %in% c("Alien Big Cat", "Cryptozoology", "Shuck") ~ "Cryptozoology",
				 type %in% c("Dragon", "Vampire", "Werewolf" ) ~ "Monster",
				 type %in% c("Crisis Manifestation", "Environmental Manifestation", "Experimental Manifestation", "Haunting Manifestation", "Manifestation of the Living", "Poltergeist", "Post-Mortem Manifestation", "Unknown Ghost Type") ~ "Haunting",
				 type %in% c("Legend", "Fairy", "Curse") ~ "Legend",
				 type %in% c("Spontaneous Human Combustion", "Other") ~ "Other Manifestation",
				 type %in% c("UFO") ~ "UFO"
				 )

}

# Load font
font_add( "mapfont", "/usr/share/fonts/TTF/weird/JANCIENT.TTF" )
showtext_auto()

# Read world shapefile data and tranform to an appropriate projection.
# Limit to the UK, Ireland, and the Isle of Man
world <- readOGR( dsn='data/ne/10m_cultural', layer='ne_10m_admin_0_countries' )
world_subset <- world[ world$iso_a2 %in% c("GB","IE", "IM"), ]
world_subset <- spTransform(world_subset,CRS("+init=epsg:4326"))
world_df <- fortify( world_subset )

# Read UK Major Cities datafile
#cities <- readOGR( dsn='data/ons_uk_cities', layer='Major_Towns_and_Cities_December_2015_Boundaries' )
#cities <- spTransform(cities,CRS("+init=epsg:4326"))
#cities_fortified <- fortify( cities )
#cities_centroids_tbl <- as.tibble( coordinates( cities ) )
#cities_tbl <- as.tibble( cbind( cities@data$tcity15nm, cities_centroids_tbl ))
#colnames(cities_tbl) <- c( "city", "lng", "lat" )


# As the polygons are more confusing than useful, instead label centroids

# Read paranormal database
paranormal_tbl <- as.tibble( read.csv( file="data/paranormal_database.csv" ) )

paranormal_tbl$combined_type <- paranormal_tbl$type %>%
	map( combine_paranormal_types ) %>%
	unlist


# Convert the paranormal dataframe to a spatial dataframe that contains
# explicit longitude and latitude projected appropriately for plotting.
coordinates( paranormal_tbl ) <- ~lng+lat
proj4string( paranormal_tbl )<-CRS("+init=epsg:4326")
paranormal_tbl_spatial <- spTransform(paranormal_tbl,CRS(proj4string(paranormal_tbl)))

# Restrict paranormal_tbl to those points in the polygons defined by world_subset
paranormal_tbl_rows <- paranormal_tbl_spatial %>%
	over( world_subset ) %>%
	is.na() %>%
	not() %>%
	rowSums() %>%
	`!=`(0) %>%
	which

#paranormal_tbl <- as.tibble( paranormal_tbl_spatial[ paranormal_tbl_rows, ] )

# Create window for spatial analysis
paranormal_owin <- as.owin.SpatialPolygons(world_subset)

# Function to plot density of a specific manifestation type.
# plot_resolution is for the density raster, and is mainly used for quick prototyping of the output.
density_plot <- function( paranormal_type, plot_resolution = 1024 ) {

	cat( paste0( "Plotting density: ", paranormal_type, "... " ) )

	paranormal_tbl_spatial <- paranormal_tbl_spatial[ which( paranormal_tbl_spatial$combined_type == paranormal_type ), ]

	paranormal_ppp <- 
		ppp( 	x=coordinates(paranormal_tbl_spatial)[,1], 
			 y=coordinates(paranormal_tbl_spatial)[,2], 
			 window = paranormal_owin )

	# This discards 'illegal' points outside of the window
	paranormal_ppp <- as.ppp(paranormal_ppp)

	paranormal_density <- density( paranormal_ppp, diggle=T, sigma=0.4, dimyx=c(plot_resolution,plot_resolution) )

	# Make density image object usable by ggplot as a raster
	paranormal_density_raster <- raster( paranormal_density )
	raster_tbl <- as.tibble( rasterToPoints( paranormal_density_raster ) )

	# Show the map
	gp <- ggplot() + 

		geom_map( data = world_df, aes( map_id=id ), colour = "#3c3f4a", fill = "transparent", size = 0.8, map = world_df ) +

		# Add density of sightings as raster.
		geom_raster( data = raster_tbl, alpha=0.8, aes( x=x, y=y, fill=layer), show.legend=TRUE ) +
		scale_fill_viridis( option="magma", direction = -1, name="Density" ) 

	gp <- gp +	

		# Theming 
		theme_map() + 

		theme( 
				plot.background = element_rect(fill = "transparent", colour = "transparent"),
				panel.border = element_blank(),
				plot.title = element_text( size=12, colour="#3c3f4a", family="mapfont" ),
				text = element_text( size=12, color="#3c3f4a", family="mapfont" ),
				) +

		theme( 
				legend.background = element_rect( colour ="transparent", fill = "transparent" ), 
				legend.key = element_rect(fill = "transparent", colour="transparent"),
				legend.position = c(0,0),
				legend.justification = c(0,0)
		) +
	
		guides( fill = guide_colourbar( title.position="top", direction="horizontal", barwidth=6, barheight=0.4 ) ) +

                # Fix the ratio of the plot to avoid distorting the UK
		coord_fixed( ratio=1.2 )

	cat("done.\n" )
	return(gp)
}

# Calculate densities for each phenomenon
phenomena <- unique( paranormal_tbl$combined_type )

gp_list <- phenomena %>%
	map( density_plot, 1024 )

# Plot as a grid with cowplot
theme_set(theme_cowplot(font_size=4, font_family = "mapfont" ) )
gp <- plot_grid( plotlist=gp_list,
			labels = phenomena,
			label_colour = "#3c3f4a" )

# Cowplot trick for ggtitle
title <- ggdraw() + 
	draw_label("Density of Paranormal Manifestations in the British Isles", fontfamily="mapfont", colour = "#3c3f4a", size=24, hjust=0, vjust=1, x=0.02, y=0.88) +
	draw_label("http://www.weirddatascience.net | @WeirdDataSci", fontfamily="mapfont", colour = "#3c3f4a", size=14, hjust=0, vjust=1, x=0.02, y=0.40)

data_label <- ggdraw() +
	draw_label("Data from: http://www.paranormaldatabase.com", fontfamily="mapfont", colour = "#3c3f4a", size=12, hjust=1, x=0.98 )
 
tgp <- plot_grid(title, gp, data_label, ncol=1, rel_heights=c(0.1, 1, 0.1)) 

vellum_plot <- ggdraw() +
	draw_image("img/vellum.jpg", scale=1.4 ) +
	draw_plot(tgp)

save_plot("output/paranormal-density.pdf", 
				vellum_plot,
				base_width = 16,
				base_height = 9,
			        base_aspect_ratio = 1.78 )

To leave a comment for the author, please follow the link and comment on their blog: Weird Data Science.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)