Systematic error messages

Anyone writing code for use in data processing systems needs to have a well thought-out protocol for generating error messages and logs. When a complex pipeline breaks, good logs and recognizable error messages are key to debugging the problem. This post describes improvements to the MazamaCoreUtils package that help you create systematic error messages that can be better handled by calling functions.

Easy Error Handling

Error handling in the MazamaCoreUtils package has been described in previous blog posts:

• Basic Error Handling with tryCatch
• Easier Error Handling in R with try()
• Logging and error handling in operational systems

We are a little obsessed with logging and error handling, but for a very good reason: Whenever anyone complains that one of our automated data processing pipelines isn’t working properly, a quick look at the log files tells us what, if anything, is wrong with our code or whether the problem lies further upstream with the data we ingest.

< mark class="wp-block-coblocks-highlight__content">Good error messages and detailed logging can save you LOTS of time!

In an effort to make error handling as easy as possible, our previous recommendation was to put any code that might fail into a try block and test the result:

result <- try({
  # ...
  # lines of R code
  # ...
}, silent = FALSE)
stopOnError(result)

The stopOnError() function tests whether result is of class “try-error” and stops with the contents of geterrmessage() or a custom error message provided by the user. If logging is enabled, this message is also logged at the ERROR level. This has been a very useful construct over the years.

Improvements to stopOnError()

After many months using this function operationally in our data processing pipelines, a few improvements have been added and are described below. The new function signature is:

stopOnError <- function(
  result,
  err_msg = "",
  prefix = "",
  maxLength = 500,
  truncatedLength = 120,
  call. = FALSE
) {

result

As in the previous version, this is the result of the try({ ... })block.

err_msg

As in the previous version, a custom error message will be used if provided.

prefix

Sometimes it is nice to retain the detailed information obtained with geterrmessage() while assigning this message to a particular type. Providing a prefix allows developers to approach python-style exceptions by prefixing an error message with their own information, e.g.:

stopOnError(result, prefix = "USER_INPUT_ERROR")

maxLength, truncatedLength

We have found that the contents of geterrmessage() can sometimes include huge amounts of text. For example, when requesting data from a web service, we might get back the html contents of an error status page. When this is written to a log file, that log file becomes much more difficult for a human to scan.

Truncating messages to some reasonable length ensures that we get useful information without wrecking the readability of our log files.

call.

The previous version of stopOnError() always stopped with stop(err_msg, call. = FALSE). This restriction is not necessary so we elevated this argument into the stopOnError() function signature.

Example Usage

Our new, preferred style of error handling looks like this:

library(MazamaCoreUtils)

# A function that might fail
myFunc <- function(x) { return(log(x)) }

# Bad user input
userInput <- "10"

# Default error message
try({
  myFunc(x = userInput)
}, silent = TRUE) %>%
  stopOnError(result)

# Custom error message
try({
  myFunc(x = userInput)
}, silent = TRUE) %>%
  stopOnError(result, err_msg = "Unable to process user input")

# Default error message with prefix
try({
  myFunc(x = userInput)
}, silent = TRUE) %>%
  stopOnError(result, prefix = "USER_INPUT_ERROR")

# Truncating prefixed default message
try({
  myFunc(x = userInput)
}, silent = TRUE) %>%
  stopOnError(
    result,
    prefix = "USER_INPUT_ERROR",
    maxLength = 40,
    truncatedLength = 32
  )

Best wishes for better error handling in all your code.