Yosh is writing

h1 Error Handling Survey
- 2019-11-13

Introduction

Rust’s error handling is a pleasure to use thanks to the Result type. It ensures Rust’s error handling is always correct, visible, and performant. And with the addition of the ? operator in Rust 1.13, and the addition of return types from main in Rust 1.26 Rust’s error handling has only kept improving.

However it seems we haven’t quite reached the end of the road yet. The Crates.io ecosystem has seen a recent addition of a fair number of error-oriented libraries that try and improve upon the status quo of Rust’s error handling.

This post is a survey of the current crates.io error library landscape. 1

1

This is not to say I don’t have any preferences in terms of error handling, but I think presenting an overview of error handling in userland is more valuable. Any progress on this topic from the Rust teams will first understanding of which problems currently exist with error handling and how they’re already being solved.

Libraries

error-chain

error-chain is one of the earlier error handling libraries and was released in April of 2016. A lot of what’s proposed in this library has become part of std over the years. In particular it introduced the following features:

// Fallible main function.
quick_main!(|| -> Result<()> {
    // Chain errors to provide context.
    let res: Result<()> = do_something().chain_err(|| "something went wrong");

    // Error-based assertions.
    ensure!(2 > 1, "num too big");

    // Return ad-hoc errors from strings.
    bail!("error")
});

Looking at this library it’s quite obvious it’s aged a bit. But in a good way; because many of its innovations have become part of everyday error handling in Rust, and the need for the library has lessened.

2

These are assertions that instead of panicking return a Result::Err. For example ensure!(list.len() < 12, "list is too long").

failure

Failure is the spiritual successor to error-chain, and was released in November of 2017. It comes with an analysis of the limitations of the error trait, and introduced a new trait, Fail, that served as a prototype of how to overcome those limitations. In particular it came with the following features:

//! A basic example of failure at work.

use failure::{ensure, Error, ResultExt};

fn check(num: usize) -> Result<(), Error> {
    ensure!(num < 12, "number exceeded threshold");
    Ok(())
}

fn main() -> Result<(), Error> {
    check(6).context("Checking number failed.")?;
    check(13).context("Checking number failed.")?;
    Ok(())
}

It seems as failure was designed in anticipation of main being able to return Result, and so unlike error-chain it doesn’t bother with that. Also it brings back the ability to define an [Error + ErrorKind]. It feels like the library was equal parts trying to improve defining new errors for use in libraries, as ad-hoc errors for use in applications.

[#[derive(Fail)]]: https://docs.rs/failure_derive/0.1.5/failure_derive/derive.Fail.html

context-attribute

context-attribute was a library that I wrote in May of 2019 to solve the locality problem of failure’s ResultExt::context method.

Namely when you’re applying .context onto a function, what you’re really doing is describing what that function does. This is so that when an error occurs from that function, we’re provided with human-readable context of what we were trying to do. But instead of that context being provided by the function we’re calling, the context needs to be set by the caller.

Not only does it feel like the context and the error site are detached when using .context(), it makes the calling code harder to read because of all the inline documentation strings.

//! Error context being provided at call site.

fn square(num: usize) -> Result<usize, failure::Error> {
    ensure!(num < 10, "Number was too large");
    Ok(num * num)
}

fn main() -> Result<(), failure::Error> {
    square(2 * 2).context("Squaring a number failed")?;
    square(10 * 10).context("Squaring a number failed")?;
    Ok(())
}

// Error context being provided during definition.

/// Square a number if it's less than 10.
#[context]
fn square(num: usize) -> Result<usize, failure::Error> {
    ensure!(num < 10, "Number was too large");
    Ok(num * num)
}

fn main() -> Result<(), failure::Error> {
    square(2 * 2)?;
    square(10 * 10)?;
    Ok(())
}

err-derive

err-derive is a failure-like derive macro for std::error::Error first released in December of 2018. It’s motivation for existing is that since std::error::Error is going to be gaining many of the benefits of failure::Fail, there should be a macro to use just that.

Features it provides are:

[#[derive(Error)]]: https://docs.rs/err-derive/0.1.6/err_derive/derive.Error.html

//! Error-derive in action.

use std::error::Error;

#[derive(Debug, derive_error::Error)]
pub enum LoadingError {
    #[error(display = "could not decode file")]
    FormatError(#[error(cause)] FormatError),
    #[error(display = "could not find file: {:?}", path)]
    NotFound { path: PathBuf },
}

fn print_error(e: &dyn Error) {
    eprintln!("error: {}", e);
    let mut cause = e.source();
    while let Some(e) = cause {
        eprintln!("caused by: {}", e);
        cause = e.source();
    }
}

auto_enums

auto-enums is a proc macro prototype of a language feature to enable more flexible impl Trait return types. But since Error is a trait, it’s uses extend to error handling too. It was first released in December of 2018.

auto-enums unfortunately doesn’t allow anonymous impl Error yet inside Result, but it can create an auto-derive for Error if all variants’ inner values in an enum implement Error.

//! Quickly create a new error type that's the sum of several other error types.

use auto_enums::enum_derive;
use std::io;

#[enum_derive(Error)]
enum Error {
    Io(std::io::Error),
    Fmt(std::fmt::Error),
}

fn foo() -> Result<(), Error> {
    let err = io::Error::new(io::ErrorKind::Other, "oh no");
    Err(Error::Io(err))
}

fn main() -> Result<(), Error> {
    foo()?;
    println!("Hello, world!");
    Ok(())
}

But presumably if impl Trait in auto_enums would work for Error the way it does for other traits, the error enum would be anonymous and created on the fly. Which would allow us to do multiple operations with different error types in a single function without declaring a new error type.

use std::error::Error;
use std::fs;

fn main() -> Result<(), impl Error> {
    let num = i8::from_str_radix("A", 16)?;
    let file = fs::read_to_string("./README.md")?;
    Ok(())
}

snafu

snafu is an error handling library first released in January 2019. The purpose of it is: “to easily assign underlying errors into domain-specific errors while adding context”.

It seems to draw from experiences of using failure, and ships a comparison as part of the provided documentation. A primary difference seem to be that failure defines a new trait Fail, while snafu uses std::error::Error. Snafu also don’t thinks having shorthands to create errors from strings is very important, stating: “It’s unclear what benefit Failure provides here.”

Snafu provides the following features:

// A small taste of defining & using errors with snafu.

#[derive(Debug, Snafu)]
enum Error {
    #[snafu(display("Could not open config from {}: {}", filename.display(), source))]
    OpenConfig {
        filename: PathBuf,
        source: std::io::Error,
    },
    #[snafu(display("The user id {} is invalid", user_id))]
    UserIdInvalid { user_id: i32, backtrace: Backtrace },
}

fn log_in_user (filename: Path, id: usize) -> Result<bool, std::error::Error> {
    let config = fs::read(filename).context(Error::OpenConfig { filename })?;
    ensure!(id == 42, Error::UserIdInvalid { user_id });
    Ok(true)
}

[#[derive(Snafu)]]: https://docs.rs/snafu/0.6.0/snafu/derive.Snafu.html [OptionExt]: https://docs.rs/snafu/0.6.0/snafu/trait.OptionExt.html

fehler

fehler is a new error library by the inimitable Boats, released in September of 2019 (the same author as failure). It feels like an experimental attempt to answer the question of: “What if we introduced checked exception syntactic sugar for Rust’s error handling”. It seems to propose that what async/.await is to Future, throw/throws would be to Result.

This library was only released after the backtrace feature landed on nightly, and Error::source landed in Rust 1.30.

The innovation it provides here is the removal of the need to Ok wrap each value at the end of a statement. Meaning Ok(()) is no longer needed. Additionally creating new errors can be done using the error! macro. And returning errors from the function (“throwing”) errors can done using the throw! macro.

Each function is annotated using the fehler::throws attribute. It optionally takes an argument for the error type, such as io::Error. If the error type is omitted, it assumes Box<dyn Error + Send + Sync + 'static>.

edit: the following section is slightly out of date as fehler merged a radical simplification to their API 4 hours ago. It now only exposes throw and throws, and allow specifying your own default error type.

Unlike failure, this library only defines methods for propagating errors and creating string errors. It doesn’t have any of the extra error-definition utilities failure had. In particular it provides:

//! Error handling relying only on stdlib

use std::error::Error;

async fn square(x: i32) -> Result<i32, Box<dyn Error + Send + Sync + 'static>> {
    if x > 12 {
        Err(format!("Number is too large"))?;
    }
    Ok(x * x)
}

//! An example of Fehler in use.

use fehler::{throws, throw, error};

#[throws]
async fn square(x: i32) -> i32 {
    if x > 12 {
        throw!(error!("Number is too large"));
    }
    x * x
}

//! If Fehler's proc macros were directly implemented as lang features.

async fn square(x: i32) -> i32 throws {
    if x > 12 {
        throw "Number is too large";
    }
    x * x
}

anyhow

anyhow is an error-handling library for applications, and was first released on October 7th, 2019. It was built to make it easier to propagate errors in applications, much like Fehler. But instead of prototyping new language features, it defines a few new types and traits; with zero dependencies. Its tag line is: “A better Box<dyn Error>”.

This library was only possible after the implementation of RFC 2504 landed, and it requires Rust 1.34+ in order to function. The main features it provides are:

//! An example using anyhow.

use anyhow::{Context, Result};

fn main() -> Result<()> {
    ...
    it.detach().context("Failed to detach the important thing")?;

    let content = std::fs::read(path)
        .with_context(|| format!("Failed to read instrs from {}", path))?;
    ...
}

thiserror

thiserror was written by the same author, dtolnay, and released in the same week as anyhow. Where both fehler and anyhow cover dynamic errors, thiserror is focused on creating structured errors.

In a way, anyhow and thiserror feel like they have taken the wide ranger of features failure introduced in 2017, and split the domain between them. thiserror only contains a single trait to define new errors through:

use thiserror::Error;

#[derive(Error, Debug)]
pub enum DataStoreError {
    #[error("data store disconnected")]
    Disconnect(#[from] io::Error),
    #[error("the data for key `{0}` is not available")]
    Redaction(String),
    #[error("invalid header (expected {expected:?}, found {found:?})")]
    InvalidHeader {
        expected: String,
        found: String,
    },
    #[error("unknown data store error")]
    Unknown,
}

thiserror’s derive formatting was fairly unique at the time of release, and has since been extracted into the displaydoc crate to implement #[derive(Display)] for structs and enums.

std::error::Error

And last but not least, std’s Error trait has come a long way since the early days. A notable few changes have happened over the past few years:

In addition with Result from main, exit codes, and the ? operator we’ve definitely come a long way from the early days of Rust’s error handling.

Features

So far we’ve taken a look at 9 different error handling libraries, and you might have noticed that they have a lot in common. In this section we’ll try and break down which features libraries introduce, and what their differences are.

Returning Result from main

error-chain introduces the ability to return Result from main, and this was added to Rust in 1.26.

Error-based assertions

In the stdlib the assert! and assert_eq! macros exist. These are convenient shorthands to check a condition, and panic if it doesn’t hold. However, it’s quite common to check a condition, and want to return an error if it doesn’t hold. To that extent error-chain, failure, snafu, and anyhow introduce a new macro: ensure!.

The largest difference between the macros is which values are valid in the right-hand side of the macro. In snafu only structured errors are valid, in failure only strings are valid, and in anyhow both are valid.

//! Strings in the right-hand side to create new, anonymous errors.
ensure!(user == 0, "only user 0 is allowed");

//! Structured errors in the right-hand side.
use std::io::{Error, ErrorKind}

ensure!(depth <= MAX_DEPTH, Error::new(ErrorKind::Other, "oh no"));

Causes

Many libraries implement the ability to recursively iterate over Error::source (formerly Error::cause). These libraries include error-chain, failure, fehler, and anyhow.

In addition, fehler and anyhow will print the list of causes when returned from fn main.

Error: Failed to read instrs from ./path/to/instrs.json

Caused by:
    No such file or directory (os error 2)

err-derive doesn’t provide any facilities for this, but recommends using a pattern to print errors. This could be used with other libraries such as snafu as well. But it misses the “print from main” functionality of fehler and anyhow. And it doesn’t provide the same convenience anyhow::Chain does either by implementing DoubleEndedIterator and ExactSizeIterator.

for cause in error.chain() {
    println!("error: {}", cause)
}

Backtraces

error-chain, failure, and snafu add backtrace support to their error types. Backtrace support is currently available on nightly, and fehler, and anyhow make use of this. It seems that if this feature becomes available on stable, backtrace support for errors will mostly have been covered.

Creating errors from strings

failure, and anyhow provide a format_err! macro that allows constructing a boxed error from a string. anyhow has a type alias called anyhow with the same functionality.

fehler provides a macro error! to similarly construct new errors from strings.

let num = 21;
return Err(format_err!("{} kitten paws", num));

snafu does not provide a dedicated macro for creating errors from strings, but instead recommends using a pattern.

fn example() -> Result<(), Box<dyn std::error::Error>> {
    Err(format!("Something went bad: {}", 1 + 1))?;
    Ok(())
}

Early returns with string errors

error-chain, failure, and anyhow provide a bail! macro that allows returning from a function early with an error constructed from a string.

fehler has a model where Err and Ok no longer are required, and provides a throw! macro to return errors. And error! macro to create new errors from strings.

if !has_permission(user, resource) {
    bail!("permission denied for accessing {}", resource);
}

snafu does not provide a dedicated macro for creating errors from strings, but instead recommends using the Err(err)? pattern.

Context on Result

failure, snafu, fehler, and anyhow allow extending Result with context method that wraps an error in a new error that provides a new entry for source, effectively providing an extra layer of descriptions.

It’s unclear how each library goes about this, but especially anyhow seems to have gone to great lengths to prevent it from interacting negatively with downcasting.

fn do_it() -> Result<()> {
    helper().context("Failed to complete the work")?;
    // ...
}

error-context provides a way to move .context from the call site to the error definition through doc comments.

/// Complete the work
#[context]
fn helper() -> Result<()> {
    // ...
}

fn do_it() -> Result<()> {
    helper()?;
}

Custom errors

failure, err_derive, auto_enums, snafu, and thiserror provide derives to construct new errors. They each go about it slightly differently.

//! Failure

#[derive(failure::Fail, Debug)]
#[fail(display = "Error code: {}", code)]
struct RecordError {
    code: u32,
}

//! err_derive

/// `MyError::source` will return a reference to the `io_error` field
#[derive(Debug, err_derive::Error)]
#[error(display = "An error occurred.")]
struct MyError {
    #[error(cause)]
    io_error: io::Error,
}

//! snafu

#[derive(Debug, Snafu)]
enum Error {
    #[snafu(display("Could not open config from {}: {}", filename.display(), source))]
    OpenConfig {
        filename: PathBuf,
        source: std::io::Error,
    },
    #[snafu(display("The user id {} is invalid", user_id))]
    UserIdInvalid { user_id: i32, backtrace: Backtrace },
}

//! thiserror

#[derive(thiserror::Error, Debug)]
pub enum DataStoreError {
    #[error("invalid header (expected {expected:?}, found {found:?})")]
    InvalidHeader {
        expected: String,
        found: String,
    },
    #[error("the data for key `{0}` is not available")]
    Redaction(String),
}

#[enum_derive(Error)]
enum Enum<A, B> {
    A(A),
    B(B),
}

failure, err_derive, and snafu all seem to make use of format-string like interpolation inside inner attributes on structs to define custom error messages. thiserror is different, and provides a more inline style of formatting.

auto_enums is yet again different, in that it doesn’t provide the ability to define new error messages, but will delegate to the error implementation of its member’s inner values.

Other / Miscellaneous

This is a list of miscellaneous developments around errors that seemed somewhat related, and generally interesting.

Try blocks

try {} blocks are an unstable feature that introduces a new kind of block statement that can “catch” error propagation using the ? operator, and doesn’t require Ok wrapping at the end of the block.

This is useful for when for example wanting to handle multiple errors in a uniform way.

async fn read_file(file: Path) -> io::Result<()> {
    let file = try {
        path.is_file().await?;
        let file = fs::read_to_string(file).await?;
        file
    }.context("error reading the file")?;

    println!("first 5 chars {:?}", file[0..5]);
    Ok(())
}

This feature seems to have some overlap with fehler’s throw / throws syntax; and the interaction between the two language directions is definitely interesting.

Verbose IO errors

When reading a file from a path fails, you’ll often see an error like this:

Finished dev [unoptimized + debuginfo] target(s) in 0.56s
Running `target/debug/playground`
Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }

This is often quite unhelpful. Which file did we fail to read? What should we do to make it pass? We can usually find out with a bit of debugging. But if the error comes from a dependency, or a dependency’s dependency, figuring it out can often become a huge chore.

In async-std we’re in the process of adding “verbose errors” that contain this information, and should reduce compilation times. Because it’s off the hot path, and we already allocate regularly, it should hardly have any impact on performance.

Returning verbose errors by default in the stdlib may not be the right fit. But this is definitely something that has come up in the past, and will continue to come up in the future unless it’s addressed. But as always, the hardest question to answer is how.

Error pointer registers

Apparently Swift has really efficient error handling using try/catch. It reserves a dedicated register for error pointers, zeroes it, then checks if it’s still 0. The way it achieves this is through a dedicated swifterror instruction in LLVM.

It’s been mentioned that this is something that could be done for Rust’s ? as well, though nobody has the resources to.

Analysis

It seems like error-chain, and failure marked some big upsets in error handling for Rust. And since their release some of the value they’ve provided has been merged back into the language. Since error-chain Result from fn main has become possible. And since failure, Error::causes has been deprecated in favor of Error::source. And backtrace support has become available on nightly.

Utilities

However, throughout the years some features have been consistent among crates, but never found their way into the standard library. In particular format_err!, ensure!, and bail! are recurring in a lot of libraries, but haven’t made their way in. Having more readily available methods of creating and using string-based errors seems to be a desirable feature.

Creating errors

Another common theme is the availability of a context method on Result. There seems to be almost universal consensus among library authors that this is desirable, yet it has no counterpart in stdlib yet.

From there it seems libraries can roughly be divided into two categories: dynamic errors and structured errors. This split is best evidenced in the sibling libraries of anyhow, and thiserror.

In terms of dynamic error handling failure’s Fail type started the current trend, and fehler and anyhow seem to have modernized it with what’s been made available in stdlib since. In particular these libraries provide a better version of Box<dyn Error + Send + Sync + 'static>.

In terms of defining new errors, snafu and thiserror seem to have made the most progress since failure. They mainly differ in the way serialization of new errors works.

Language developments

In terms of language experimentations there have also been exciting developments. auto_enums hypothesises that if impl Trait would work, anonymous enums could be created for types that can delegate to their inner implementations. And fehler proposes syntactic sugar to remove Result from error use entirely, instead using checked exceptions through the throw and throws keywords.

The similarities between throw, throws, and try could plausibly help lower the barrier for learning Rust as they have much in common with mainstream programming languages such as JavaScript, Swift, and Java. But ideally further research would be conducted to help corroborate this.

Conclusion

In this post we’ve looked at 9 different error handling libraries, the features they propose, and laid out how they compare to other libraries in order to determine recurring themes.

There seems to be rough consensus in the ecosystem that we seem to need:

Additionally the functionality provided by ensure!, bail!, and format_err! has been exported as part of many of the libraries, and seems to be along the lines of: “stable, popular, and small” that is the bread and butter of std.

Since starting work on this post, two others (1, 2) have written about error handling. Which seems indicative error handling is something that’s on people’s mind right now.

This is all I have to share on error handling at the moment. I hope this post will be prove to be useful as people set out to improve Rust’s error handling.

Thanks to: Aleksey Kladov, Niko Matsakis, and Alex Crichton for reading reviewing this post and helping refine it.