Getting started with ZIO HTTP

ZIO HTTP is a library for building HTTP applications in Scala. It started as a ZIO wrapper over the Netty library, developed at Dream11 for their high throughput, low latency services. Revealed in 2021 at Functional Scala, it made a great impression on the attendants not only because of the great type-safe DSL but also because of its extraordinary benchmark results and the fact that it’s already used in production on a demanding system. It was later donated to the ZIO community and is now an important part of it as well as being one of its most actively developed projects. 

ZIO HTTP provides a type-safe and purely functional way to build HTTP servers and clients using ZIO’s effect system. It aims to provide a highly composable and expressive API for building HTTP applications while leveraging the benefits of functional programming.

It’s both fast and innovative. There are a lot of niceties that come from the power of ZIO (such as the error handling, thanks to ZIO’s error channel). Also included is a DSL for building documented JSON endpoints as well as built-in type-safe HTML templating.

The application

For this tutorial, we will be building a simple application for task tracking, commonly known as a TODO application. Essentially it will be a CRUD API with a simplified model that will allow us to look into different techniques offered by ZIO HTTP. The plan is to have this tutorial split into 3 parts with each one detailing one of the approaches. Part one will show the use of Routes to build JSON APIs, part two will be about the new Endpoints API making it easy to build documented endpoints, and part three will present HTML templates.

Building JSON APIs using Routes

Project setup

You can start however you like but for this purpose, we will bootstrap our new project using Scalac’s template for Scala 3 / ZIO applications (there is also a template for generating a fully working ZIO HTTP application with a relational database backend but for learning purposes, the vanilla Scala/ZIO template will be better). The only prerequisite is to have SBT up and running.

Once you have sbt installed run the command below:

sbt new ScalaConsultants/zio-dotty-quickstart.g8

Sbt will ask you a couple of questions that you can answer by just hitting enter (unless you want some customization). This will generate a new project directory with a project structure.

Now find a file called build.sbt in the main project directory and add these two lines to the libraryDependencies. These are the libraries we will need to build JSON APIs with ZIO HTTP.

libraryDependencies ++= Seq(
...
"dev.zio" %% "zio-http" % "3.0.0-RC4",
"dev.zio" %% "zio-json" % "0.6.2",
...
)

Starting the HTTP server

We will start with the simple task of starting the HTTP server at port 8090 and returning a text response to every request.

To do so, find the Main.scala file and replace its contents with the snippet below (you should leave the initial package <package.name> part that the sbt new command generated).

package my.project

import zio.ZIOAppDefault
import zio.http.*

object Main extends ZIOAppDefault:

  val app = Handler.text("Welcome to JSON APIs!").toHttpApp
    def run = Server.serve(app).provide(Server.defaultWithPort(8090))

The significant part is the last two lines where we create a Handler that ‘does something’, turn it into an HttpApp and serve it on a port provided as a configuration. This ‘ritual’ will be the same for the rest of the tutorial, only the Handlers will become more complex. 

We will not be delving into the configuration options in this tutorial as there are so many. Instead, I suggest you try reading the documentation and/or play with code suggestions your IDE provides (you will find most self-explanatory).

You can now run the server with a command:

sbt run

And then call any URL to receive a response:

> curl localhost:8090/abc
Welcome to JSON APIs!

Introducing Routes

As we can see, a Handler in the form we have used is detached from the request, in the sense that it’s used to respond to every request in the same way. That’s not a very common usage. As API developers we will usually create a handler per ‘route’ – the Route being a combination of HTTP Method and Path (which directly corresponds to the HTTP protocol request line, like

GET /todo HTTP/1.1

(only ignoring the protocol’s name/version part)

So right now our server responds with “Welcome to JSON APIs!” to every request we send it – whether it’s GET or POST or the path is “/” or “/whatever”. Let’s make it answer only to requests sent to localhost:8090/todo using the GET method. For this purpose, we will use Routes.

  val app =
    Routes(
      Method.GET / "todo" -> Handler.text("Welcome to JSON APIs!")
    ).toHttpApp

Now if we try the same call to /abc as before we will get 404 Not Found:

> curl -i localhost:8090/abc
HTTP/1.1 404 Not Found
warning: 404 ZIO HTTP /abc
content-length: 0

If we try a POST method to /todo URL we will also get 404 Not Found:

> curl -i -X POST localhost:8090/todo
HTTP/1.1 404 Not Found
warning: 404 ZIO HTTP /todo
content-length: 0

Finally, if we call GET /todo we will get our answer:

> curl -i localhost:8090/todo
HTTP/1.1 200 OK
content-type: text/plain
content-length: 21
Welcome to JSON APIs!

Ok, so now we know we can connect a handler to an HTTP method and path to create a Route. And what if we want to parametrize the path? After all, putting some dynamic information into the path, like entity IDs, is a common occurrence in APIs. Let’s try creating a route that will return a ToDo with a particular ID that would be a part of the URL.

val app =
  Routes(
    Method.GET / "todo" -> Handler.text("Welcome to JSON APIs!"),
    Method.GET / "todo" / string("id") ->
      handler { (id: String, req: Request) =>
        Response.text(s"This will show a TODO item with id: $id")
      }   ).toHttpApp

So now calling the new route will return:

> curl -i localhost:8090/todo/123
HTTP/1.1 200 OK
content-type: text/plain
content-length: 39

This will show a TODO item with id: 123

Notice that in the non-parameterized route, we’re using a smart constructor on the Handler object that creates a Handler from a provided String, while for the parameterized route we use a handler method defined on the http package object (as this one gives us access to the request context).

Now that we know how to create parameterized routes, we can make an initial attempt at all the routes we want in our application – including POST and DELETE methods.

val app =
  Routes(
    Method.GET / "todo" -> Handler.text("Welcome to JSON APIs!"),
    Method.POST / "todo" -> Handler.text("This will create a new TODO item"),     
    Method.GET / "todo" / string("id") ->
      handler { (id: String, req: Request) =>
        Response.text(s"This will show a TODO item with id: $id")
      }
    Method.DELETE / "todo" / string("id") ->
      handler { (id: String, req: Request) =>
        Response.text(s"This will remove a TODO item with id: $id")
      }
  ).toHttpApp

Simple model and storage

Until now, we’ve been responding to requests with simple text messages. To build some more meaningful JSON responses, let’s introduce some simple models and storage.

Our main model will be a ToDo class that will be stored in a Map (as this tutorial doesn’t cover database interaction). This will give us a simple way of retrieving ToDos by ID in the case of a GET request and adding more in the case of the POST request. For the latter, we will also need a way to represent the JSON request body. That is what the CreateToDoRequest class will be for.

case class ToDo(id: UUID, title: String, description: String)
case class CreateToDoRequest(title: String, description: String)

val database = List(
ToDo(UUID.fromString("612f2cdd-f7a7-4109-bcae-4e566e7e51fc"), "Buy groceries", "Blackberries, sardines, eggs"),
ToDo(UUID.fromString("0f16753c-077b-4d6e-b929-a4ad66830af3"), "Rent a movie", "Something scary"),
ToDo(UUID.fromString("5c2c5817-5d65-4537-ab26-f1447b196028"), "Order cat food", "Has to be monoprotein")
).map(todo => todo.id -> todo).to(collection.mutable.Map)

JSON responses

In order to be able to respond with JSON responses and receive JSON requests, we need to be able to encode and decode JSON messages. ZIO JSON is the library from the ZIO ecosystem that will suit our needs. It’s fast, safe, and easy to use. All we need to do is to declare Encoder and Decoder for our models and make them available implicitly in the code.

given JsonEncoder[ToDo]              = DeriveJsonEncoder.gen[ToDo]
given JsonDecoder[CreateToDoRequest] = DeriveJsonDecoder.gen[CreateToDoRequest]

The Encoder is needed for the ToDo class as it will encode the message as JSON with the use of .toJson method and the Decoder is needed for the request entity to decode it into our Scala case class (using .fromJson method). For more insights and complex examples look at the official ZIO JSON documentation.

Let’s try returning all ToDos in the JSON format:

val app =
  Routes(
    ...
    Method.GET / "todo" -> handler(Response.json(database.values.toJsonPretty)),
    ...
  ).toHttpApp

I’m using .toJsonPretty so the output is easier to read, but for production use .toJson is a better choice.

Now the request would yield:

> curl -i localhost:8090/todo
HTTP/1.1 200 OK
content-type: application/json
content-length: 408

[
  {
    "id" : "612f2cdd-f7a7-4109-bcae-4e566e7e51fc",
    "title" : "Buy groceries",
    "description" : "Blackberries, sardines, eggs"
  },
  {
    "id" : "5c2c5817-5d65-4537-ab26-f1447b196028",
    "title" : "Order cat food",
    "description" : "Has to be monoprotein"
  },
  {
    "id" : "0f16753c-077b-4d6e-b929-a4ad66830af3",
    "title" : "Rent a movie",
    "description" : "Something scary"
  }
]

So now we have our first JSON response with the correct content-type header.

We can do the same for other routes, the main difference being the operation on the ‘database’ that we perform.

val app =
  Routes(
    …
    Method.GET / "todo" / string("id") -> handler { (id: String, _: Request) =>
      val uuid = UUID.fromString(id)
      val todo = database.get(uuid)
      Response.json(todo.toJson)
    },
    …
  ).toHttpApp

val app =
  Routes(
    …          Method.DELETE / "todo" / string("id") -> handler { (id: String, _: Request) =>
      val uuid = UUID.fromString(id)
      database -= uuid
      Response.ok
    }     …
  ).toHttpApp

val app =
  Routes(
    …
    Method.POST / "todo" -> handler { (req: Request) =>
      for {
        body   <- req.body.asString.orDie
        entity <- ZIO.fromEither(body.fromJson[CreateToDoRequest])
        todo   = ToDo(UUID.randomUUID(), entity.title, entity.description)
        _      = database += (todo.id -> todo)
      } yield Response.json(todo.toJson)
    },
    …
  ).toHttpApp

Let’s stop here a bit, as the POST route and decoding the JSON body look more complicated than the GET and DELETE routes.

The first thing that deserves comment is the req.body.asString.orDie line. As you might have guessed, req.body.asString gives us the body in a String format. It returns ZIO[Any, Throwable, String] to be precise. This Throwable in the error position is something that we have to deal with now or later. In this case, the documentation says that “attempting to decode a large stream of bytes into a string could result in an out of memory error”. It’s not something we would expect from our small application model and not something we can deal with here in a meaningful way. This is where ZIO.orDie comes to the rescue – this is a signal for the ZIO that we don’t want to recover from this error and ZIO HTTP will interpret this as 500 Internal Server Error.

Another thing you might have noticed is the ZIO.fromEither(…) line. We’re using it to lift Either returned by body.fromJson so we can proceed with the for-comprehension. Also, the left part of the Either lands in the ZIO’s error channel, which will be useful for us later.

Exposing and handling errors

There is one more thing to do before the previous POST example will work correctly. If we take a look at the routes’ type signature it turns out to be Routes[Any, String]. That means that the routes can fail with a String error and it is an error that ZIO JSON will return in the case that the request body doesn’t fit our CreateToDoRequest model. We need to handle this error to get rid of it from the type signature and to be able to eventually call .toHttpApp. 

val app =
  Routes(
    …
  ).handleError(_ match
    case _: String => Response.badRequest("Wrong JSON payload")
  ).toHttpApp

This .handleError method turns Routes[Any, String] into Routes[Any, Nothing] and that allows us to call .toHttpApp without the compiler complaining.

It looks like we are done. We have our URLs and JSON requests and responses and we’ve handled the error that we knew about from the type. Unfortunately, our application can still fail in a not-so-obvious way, so we should make these cases visible and expand our error-handling clause.

Let’s take another look at our GET /todo/:id route:

val app =
  Routes(
    …
    Method.GET / "todo" / string("id") -> handler { (id: String, _: Request) =>
      val uuid = UUID.fromString(id)
      val todo = database.get(uuid)
      Response.json(todo.toJson)
    },
    …
  ).toHttpApp

There are two problems that we missed at the first take. One is the UUID.fromString method call. If we look into the documentation, it turns out this method throws an IllegalArgumentException in the case that the String passed as an argument is not a correctly formatted UUID. Thanks to the ZIO library, we can easily make it explicit:

val uuid =   ZIO.attempt(UUID.fromString(id))    .refineToOrDie[IllegalArgumentException]

The other problem is the call to database.get(uuid). It returns an Option because there’s a possibility that the ToDo with the given id doesn’t exist. The way our JSON decoder will handle it is to return a “null” String in the case that the Option is empty. That is not what we want. We want to respond with a 404 Not Found error response. One way to achieve it is, again, to expose the optionality in the error channel and later transform it in the handleError method along with other errors.

for {
  uuid <- ZIO.attempt(UUID.fromString(id)).refineToOrDie[IllegalArgumentException]
  todo <- ZIO.fromOption(database.get(uuid))
} yield Response.json(todo.toJson)

The not-so-nice thing about all of this is that the Routes now have a very unpleasant and confusing type of:

Routes[Any, Throwable | String | Option[Nothing]]

We can deal with this by mapping the generic errors returned from different frameworks and library APIs into our own error hierarchy. In our case this could be a simple enum:

enum Errors:
  case JsonPayloadError
  case IdFormatError
  case NotFoundError

The GET /todo/:id route would eventually look like this:

val app =
  Routes(
    ...
    Method.GET / "todo" / string("id") -> handler { (id: String, _: Request) =>
      for {
        uuid <- ZIO.attempt(UUID.fromString(id))
                  .refineToOrDie[IllegalArgumentException]
                  .mapError(_ => Errors.IdFormatError)
        todo <- ZIO.fromOption(database.get(uuid))
                  .mapError(_ => Errors.NotFoundError)
      } yield Response.json(todo.toJson)
    },
    ...
  )

And the POST route:

val app =
  Routes(
    …
    Method.POST / "todo" -> handler { (req: Request) =>
      for {
        body   <- req.body.asString.orDie
        entity <- ZIO.fromEither(body.fromJson[CreateToDoRequest]).mapError(_ => Errors.JsonPayloadError)
        todo   = ToDo(UUID.randomUUID(), entity.title, entity.description)
        _      = database += (todo.id -> todo)
      } yield Response.json(todo.toJson)
    },
    …
  )

So the Routes now have a clear type of:

Routes[Any, Errors]

And now we’re ready to handle the errors in the fashion we did before:

val app =
  Routes(
    …
  ).handleError(_ match
    case Errors.NotFoundError    => Response.notFound
    case Errors.JsonPayloadError => Response.badRequest("Wrong JSON payload")
    case Errors.IdFormatError    => Response.badRequest("Wrong id format (should be UUID)")
  ).toHttpApp

Summary

In this tutorial, we’ve gone through the whole process of setting up a server and creating JSON API with error handling using ZIO HTTP Routes. So what’s next? To build upon what we’ve learned here, I encourage you to look into the source code for Server, Handler and Response to get a feeling of what’s possible (and if not the source code then see the code suggestions of your IDE). I also suggest reading the official documentation and going over the examples in the official repository.

And this isn’t all. As I mentioned earlier, the latest releases have added a new DSL for creating JSON endpoints with Open API documentation (think Endpoints4s or Tapir). ZIO HTTP also has a built-in templating engine if you need to create web applications with server-side rendered pages and maybe with HTMX integration. I will do my best to present those features in the following parts of this tutorial.

Read more

• AI for programmers
• TestContainers in Scala: Use Integration Tests for building your services
• Scala 3 Data Transformation Library: Automating Data Transformations with ducktape
• How to implement streaming microservices with ZIO 2 and Kafka
• How to write a (completely lock-free) concurrent LRU Cache with ZIO STM