Image Annotation

This week, I uploaded a newer version of the R package recogito to CRAN.

The recogito R package provides tools to manipulate and annotate images and text in shiny. It is a htmlwidgets R wrapper around the excellent recogito-js and annotorious javascript libraries as well as it’s integration with openseadragon.
You can use the package to set up shiny apps which

• annotate areas of interest (rectangles / polygons) in images with specific labels
• annotate text using tags and relations between these tags (for entity labelling / entity linking).

The video below shows the image manipulation functionality in action in a shiny app which allows to align image areas with chunks of transcribed handwritten texts.

< video controls="controls" width="450" height="600">< source src="https://user-images.githubusercontent.com/1710810/181257966-2e08027c-b039-413f-a310-75bee2d6088c.mp4" type="video/mp4" /> Your browser does not support the video tag.

Although the package was orginally designed to extract information from handwritten text documents from the 18th-19th century, you can probably use it in other domains as well.
To get you started install the package from CRAN and read the README.

install.packages("recogito")

The following code shows an example app which shows an url and allows you to annotate areas of interest. Enjoy.

library(shiny)
library(recogito)
url <- "https://upload.wikimedia.org/wikipedia/commons/a/a0/Pamphlet_dutch_tulipomania_1637.jpg"
ui <- fluidPage(openseadragonOutput(outputId = "anno", height = "700px"),
                tags$h3("Results"),
                verbatimTextOutput(outputId = "annotation_result"))
server <- function(input, output) {
 current_img <- reactiveValues(url = url)
 output$anno <- renderOpenSeaDragon({
   annotorious(inputId = "results", src = current_img$url, tags = c("IMAGE", "TEXT"), type = "openseadragon")
 })
 output$annotation_result <- renderPrint({
   read_annotorious(input$results)
 })
}
shinyApp(ui, server)