Thoughts Heap

A Blog by Roman Gonzalez.-

Fast Haskell coding with cushions

Previously this week, the awesome “Matt Of Lambda” published a blog post about the benefits of using ghcid to develop Haskell applications; if you haven’t checked it out, it is an excellent read on how to get started with something like REPL driven development

Personally, I’ve always have used GHCi for development, from trivial tasks like checking my types and experiment with code to running my test suites and the main executable of my projects. This approach was usually way faster than the regular change/compile/run cycle that Haskellers developers typically do, but it came with some challenges:

  • Lack of resource de-allocation

Given that most applications are designed to run only once, there are no real incentives to develop our software in a way that can be cleaned up and started again without killing a process.

  • Code gets unusually slow after a few runs

Because of the previous point, every time you reload code in the REPL your application leaves resources hanging in memory (e.g., threads), and the REPL starts to get slower (and confusing) if dangling threads are spitting strings to stdout.

  • Port already allocated errors

Sometimes, you will get errors like a TCP Socket port is already bound, this happens when you reload your code, and your application leaves socket resources open after a reload.

Solving some of these problems

Given my tenure in Clojure land for a big chunk of the past four years, I had the opportunity to see how “REPL driven development” should be. Their tooling is oriented to have a REPL open at all times, and they run everything there, from unit-tests to executables. Following their example, I started to implement a few libraries that help me find this REPL nirvana.

teardown

Inspired by Reactive Extensions Disposable API, this library is a smart wrapper on top of IO sub-routines whose whole purpose is to perform a cleanup. This library will:

  • When a resource cleanup fails with an error, this API makes sure that this error does not affect the cleanup of other unrelated resources

  • It keeps track of the resource name and how much time it took to de-allocate; if the de-allocation fails, it also shows the error that made it fail

  • It ensures that de-allocation sub-routines are executed exactly once, avoiding errors of the “already de-allocated” nature.

You can take a look at the documentation site of this project, bear in mind though, this API is not a good fit for most applications.

componentm

Inspired by Clojure’s component, this library provides approaches for you to build all the resources of your applications transparently and safely. Some of the features this API offers:

  • It keeps track of initialization time for each declared component of your application

  • Builds a dependency graph for all your allocated components and then it guarantees they will be cleaned up in the right order (e.g., if component B depends on component A, the library makes sure that B is cleaned up before A)

  • If any of the resources fail on initialization (e.g., a Database is not running), this API will rollback all the previously allocated components in the right order and then throw an exception with detailed information

  • Makes use of teardown for resource cleanup, getting all the benefits of that API. This dependency is an implementation detail, so users of componentm don’t need to understand teardown.

  • It will execute the initialization of your components concurrently if components are built using an Applicative interface.

To create ComponentM value, you can use buildComponent:

buildDatabasePool :: LogFunc -> IORef Etc.Config -> ComponentM (Pool SqlBackend)
buildDatabasePool logFunc configRef = do
  config     <- readIORef configRef
  connString <- getConnString config
  poolSize   <- getPoolSize config

  pool <- buildComponent "databasePool"
    (runRIO logFunc (createPostgresqlPool connString poolSize))
    (destroyAllResources)

  runMigrations logFunc pool

  return pool

Although not all functions are defined in the previous example, we can see that buildComponent, which creates a Pool of PostgreSQL Database Connection and also, a call to destroyAllResources which de-allocates that pool resource.

Following is how the output of your program would look when it cannot connect to the database:

2018-05-22 07:44:02.317225: [error]
# Application Failed

Application failed on initialization, following are the exceptions that made it failed:

  * ComponentAllocationFailed "db migrations" - the following error was reported:
    |
    `- libpq: failed (could not connect to server: Connection refused
                Is the server running on host "localhost" (127.0.0.1) and accepting
                TCP/IP connections on port 5432?
        )

Following, we have the information of application resources cleanup:

✓ servant-playground (0.000036s)
  ✓ http-server (0.000033s)
  ✓ db migrations (empty)
  ✓ databasePool (0.000003s)
  ✓ logReloader (empty)
  ✓ config (empty)

You’ll notice there is an indication of the errors that our initialization script throws, as well as the cleanup of all resources that we have allocated so far. You have total visibility on the lifecycle of your program.

componentm-devel

To tie the knot, I created a library that provides an easy to use API that automatically reloads your application everytime you call it with the ComponentM builder of your application. It provides a single function runComponentDevel that receives the same parameters as runComponentM1

Conclusion

I like to say ComponentM implements a Monad to avoid the usual pyramid of withResourceX functions at the beginning of your application. It composes them all together once, so you don’t need to worry about it. If you like these APIs, make sure to check out a sibling library called etc, it provides an API to manage the configuration values of your app as a map with multiple sources in a declarative way.

You can check out a full-fledged example where we setup an app using servant and persistent-postgresql using the libraries mentioned in this blog post.

Shameless Plug

If you would like to learn more about how to build robust Haskell applications, be sure to join my workshop at LambdaConf 2018