The digital transformation of Santa’s logistical nightmare - Part 4 http4s

Welcome to part four of the series:

In this blog post, we’d like to go through how Santa’s workshop makes use of http4s. We’ll go over how to setup a server and a client, some interesting things about the internals of httpApp, how we encode and decode, and as usual some excellent pictures of cats. Related repository

See the previous posts on this series

• Part 1: Introduction
• Part 2: Cats Effects
• Part 3: F[F[S]] Kafka! fs2-kafka

General things we want to say about http4s

There are many options when it comes to Web servers in Scala; we are most familiar with http4s. It works well with the rest of the Typelevel stack, which means it works well with Cats Effect.

As with most web servers you are provided with APIs to deal with sending and receiving HTTP requests, known respectively as the client side and the server side.

Http4s has a few options when it comes to backends — these are the underlying technologies used to implement the http4s core interfaces. Most people use Blaze, which is based on Netty. But the future is with Ember which is based on an FS2 wrapper around NIO. For the purposes of this toy example we chose Ember.

Setting up the server

Whilst the http4s API is not completely intuitive (I have to check the docs every time; might be just me tho) the requirements of the API do read well once they are written. To configure our server we do the following:

server <- EmberServerBuilder
  .default[IO]
  .withHost(ipv4"0.0.0.0")
  .withPort(port"8080")
  .withHttpApp(httpApp)
  .build

We use the builder pattern provided by our chosen http4s backend to create a server. Here we specify the interface and port to bind to (0.0.0.0 & 8080 respectively). As well as the http application to expose. Finally, build constructs our Resource[IO, Server] object that encapsulates how to start and safely shutdown our web server.

What’s this httpApp?

Towards the end of the code snippet above you will notice there is an httpApp parameter being passed to withHttpApp. If you inspect the type for httpApp you will find that it’s a Kleisli[IO, Request[IO], Response[IO]]. Kleisli is a case class wrapping Request[IO] => IO[Response[IO]]. Kleisli can be used to leverage a number of combinators and typeclasses provided by cats-core - for example SemigroupK for combining Routes.

For practical purposes and for this instance, you can think of Kleisli as a way to mount the routes, deal with some errors, etc. It is also possible to combine routes using the <+> combinator (from SemigroupK) between routes as seen in the example in the http4s docs.

If you’d like to learn more about Kleisli check out this video from Alex.

Routes

Our httpApp is composed of a combined set of routes that describe how Requests should be handled, as well as a fallback in the event no routes are matched orNotFound - as seen in the snippet below.

httpApp = Router(
  "/" -> SantasRoutes.ledger(
    addressBookService,
    consignmentService
  ),
  "/internal" -> StatusRoutes.route
).orNotFound

Router lets us combine routes under a specified prefix for each route. In this case we have all of the routes relating to Santa’s logistical problems under / and then we have some status routes in /internal. For example, the status of the app can be found by calling <host:port>/internal/status.

Lets look at an example of how to define a Route that matches a GET request. In SantasRoutes.scala we can find the snippet below.

def ledger(
    addressBookService: SantasAddressBookService,
    consignmentService: ConsignmentService
): HttpRoutes[IO] =
  HttpRoutes.of[IO] {
    case GET -> Root / "list" / lastName / firstName =>
      Ok(
        consignmentService.getConsignment(
          FullName(firstName, lastName)
        )
      )

There is a bit to unpack here — the pattern match is saying that we want to match GET requests for resources that match /list with an additional two trailing segments that we wish to capture as path variables.

The following lines then say that we want to respond with a 200 OK containing whatever we find when we call getConsignment with the full name provided in path variables. At this point you may be wondering how our IO[ChristmasConsignment] is converted into an actual HTTP response.

The http4s DSL expects an EntityEncoder to be available for whatever entity you pass to .apply — in our case it’s expecting a EntityEncoder[IO, List[FullName]]. This is being derived from our Circe json Encoder using the CirceEntityCodec.circeEntityEncoder method. Our Circe codecs are being derived semi-automatically in domain.scala using macros from circe-generic.

Setting Up The client

We’ll explore our testing approach in a future post but for now we’d like to direct you to NaughtyNiceReportSpec.scala to see an example of using an http4s client.

Setting up a client is not too different from the builder pattern used previously to create a server.

val clientResource: Resource[IO, Client[IO]] =
  EmberClientBuilder.default[IO].build

In the line above we create the Client resource and specify we are using IO for effects. When we .use the Client, it will result in the creation of a connection pool. This enables the reuse of connections for multiple requests. It is good practice to create one client and reuse it in your app. (But we haven’t done this for testing, probably because we are a bit lazy, after all we are 🐱🐱🐱 cats).

clientResource.use { client =>
  client.expect[List[FullName]](
    baseUri.withPath(
      Path.unsafeFromString(
        s"/house/${URLEncoder.encode(address.address, "UTF-8")}"
      )
    )
  )
}

When calling expect we are saying we expect the response to be decoded without errors to a List[FullName]. This version of expect that takes a Uri as a parameter will make a Get request. There are alternative versions of expect that accept a Request[IO] which lets us choose the HTTP method.

Under the hood, we are using Circe to decode the incoming data. As explained in the previous section, Circe is nicely integrated into http4s.

Conclusion

We hope that this series is helping you understand not just how to use these tools but why we wrote the code in the way we did. After all, there are many ways to solve any problem, and solutions have a context.

At the time of writing this, http4s is very close to 1.0 and after many years of using it, you can really feel the thought and care that has gone to it. Thanks to everyone that made that possible… so I guess all we can do is pay it with Cat Tax.

Cat tax

And finally the reason you are really here, to see pictures of us (Bones & Megachu)!

Megachu

Bones

Thanks to Noel Welsh for the feedback, much appreciated.