Superpixels in imager
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Superpixels are used in image segmentation as a pre-processing step. Instead of segmenting pixels directly, we first group similar pixels into “super-pixels”, which can then be processed further (and more cheaply).
(image from Wikimedia)
The current version of imager doesn’t implement them, but it turns out that SLIC superpixels are particularly easy to implement. SLIC is essentially k-means applied to pixels, with some bells and whistles.
We could use k-means to segment images based on colour alone. To get good results on colour segmentation the CIELAB colour space is appropriate, because it tries to be perceptually uniform.
library(tidyverse) library(imager) im <- load.image("https://upload.wikimedia.org/wikipedia/commons/thumb/f/fd/Aster_Tataricus.JPG/1024px-Aster_Tataricus.JPG") #Convert to CIELAB colour space, then create a data.frame with three colour channels as columns d <- sRGBtoLab(im) %>% as.data.frame(wide="c")%>% dplyr::select(-x,-y) #Run k-means with 2 centers km <- kmeans(d,2) #Turn cluster index into an image seg <- as.cimg(km$cluster,dim=c(dim(im)[1:2],1,1)) plot(im,axes=FALSE) highlight(seg==1)
We mostly manage to separate the petals from the rest, with a few errors here and there.
SLIC does pretty much the same thing, except we (a) use many more centers and (b) we add pixel coordinates as features in the clustering. The latter ensures that only adjacent pixels get grouped together.
The code below implements SLIC. It’s mostly straightforward:
#Compute SLIC superpixels #im: input image #nS: number of superpixels #ratio: determines compactness of superpixels. #low values will result in pixels with weird shapes #... further arguments passed to kmeans slic <- function(im,nS,compactness=1,...) { #If image is in colour, convert to CIELAB if (spectrum(im) ==3) im <- sRGBtoLab(im) #The pixel coordinates vary over 1...width(im) and 1...height(im) #Pixel values can be over a widely different range #We need our features to have similar scales, so #we compute relative scales of spatial dimensions to colour dimensions sc.spat <- (dim(im)[1:2]*.28) %>% max #Scale of spatial dimensions sc.col <- imsplit(im,"c") %>% map_dbl(sd) %>% max #Scaling ratio for pixel values rat <- (sc.spat/sc.col)/(compactness*10) X <- as.data.frame(im*rat,wide="c") %>% as.matrix #Generate initial centers from a grid ind <- round(seq(1,nPix(im)/spectrum(im),l=nS)) #Run k-means km <- kmeans(X,X[ind,],...) #Return segmentation as image (pixel values index cluster) seg <- as.cimg(km$cluster,dim=c(dim(im)[1:2],1,1)) #Superpixel image: each pixel is given the colour of the superpixel it belongs to sp <- map(1:spectrum(im),~ km$centers[km$cluster,2+.]) %>% do.call(c,.) %>% as.cimg(dim=dim(im)) #Correct for ratio sp <- sp/rat if (spectrum(im)==3) { #Convert back to RGB sp <- LabtosRGB(sp) } list(km=km,seg=seg,sp=sp) }
Use it as follows:
#400 superpixels out <- slic(im,400) #Superpixels plot(out$sp,axes=FALSE) #Segmentation plot(out$seg,axes=FALSE) #Show segmentation on original image (im*add.colour(abs(imlap(out$seg)) == 0)) %>% plot(axes=FALSE)
The next step is to segment the superpixels but I’ll keep that for another time.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.