Create an API server with Clojure and Pedestal

Introduction

This article will walk you through building a web API using Pedestal, a Clojure web framework. Pedestal is focused on APIs first and exemplifies the very best possibilities of Clojure's functional paradigm.

You'll put together a basic project with the necessary dependencies and serve up some basic information about superheroes. In a later article, we'll also secure the API using a collection of best practices and Auth0!

Requirements

For this tutorial, you won't need much out of the box. Be sure you've:

Installed the JDK. This tutorial was written with the latest and greatest version of Java.
If you're on windows, set JAVA_HOME as an environment variable.
Wondered why JAVA_HOME isn't set by the JDK installation in 2021
Installed Clojure

Open up your command-line interface and check that you're good to go:

clj --version
 
Clojure CLI version 1.10.3.967

Setup a New Clojure Project

If you're brand new to Clojure, check out "Why Developers Should Look into Clojure" first.

Clojure's ecosystem provides several build and dependency management tools. While Leiningen and Boot provide this functionality and have been around a while, the Clojure team has provided the solution we'll be using for this article tools.deps.

Skipping Leiningen reduces the number of things we need to install. tools.deps is included with the Clojure CLI and satisfies all of the requirements for this article, so you should be set.

Like package.json for Node, pom.xml for Maven projects, and requirements.txt in Python, Clojure's CLI uses a single file to manage its dependencies. Create a root folder (clojure-pedestal) for your project on your system and, in it, create a file called deps.edn.

deps.edn contains dependencies (for different environments, like development or production), additional commands/scripts, and much more. We'll keep it simple for now - but do know that EDN stands for "Extensible Data Notation" and is used across the Clojure universe.

Add dependencies

For your Web API, you'll need to install a couple of Pedestal packages. In addition, we'll include some defacto-standard libraries for logging and accessing environment variables.

There's no "install dependency" command available in the Clojure CLI. Instead, we'll itemize what we need within deps.edn. When we invoke the Clojure CLI, it'll look into the file and download artifacts from our maven registry. You declare dependencies in the main hash-map, under the :deps keyword. Here's what your deps.edn should look like:

{:paths ["src"]
:deps {io.pedestal/pedestal.service {:mvn/version "0.5.9"}
       io.pedestal/pedestal.route {:mvn/version "0.5.9"}
       io.pedestal/pedestal.immutant {:mvn/version "0.5.9"}
       org.slf4j/slf4j-simple {:mvn/version "1.7.32"}
       environ/environ {:mvn/version "1.2.0"}}}

Add core.clj

Within your project folder, create two folders in sequence: src and app. Within app, add core.clj. your directory should look like this:

clojure-pedestal
|   deps.edn
|___src
    |
    |___app
        core.clj

Because of how the JVM constructs the classpath, it is important that the namespace and the path of the file match. For instance, if we have a namespace called app.core.models.user, the file will have to be src/app/core/models/users.clj.

Pedestal Fundamentals

Hello world

Let's walk through the fundamental steps to have an HTTP Web API listening for requests and returning data.

In core.clj, declare a namespace called app.core and require some of the dependencies we're using. Avoid spending time understanding what we're using each dependency for; it will be more apparent once we write some code.

(ns app.core (:require [io.pedestal.http :as http]
                       [environ.core :refer [env]]))

To create a functioning server, you'll need to:

Define the handler for a specific route
Define the routes and attach them to handlers
Create a service map containing some configuration options
Create a server instance and then run it

This is what some basic code to accomplish those goals should look like:

(defn hello-world [_] {:status 200 :body "Hello"}) ; Route handler
 
(def service-map {::http/routes #{["/" :get hello-world :route-name :hello-world]} ; Routes
             ::http/type   :immutant
             ::http/host   "0.0.0.0"
             ::http/join?  false
             ::http/port   (Integer. (or (env :port) 5000))}) ; Service map
 
(defn -main [_] (-> service-map http/create-server http/start)) ; Server Instance

That's it! With just this code and our packages, we can now run the server and try it out. From your root directory, run:

clj -X app.core/-main

...and then you can use curl to test the Web API ...

curl http://localhost:5000
Hello

Routing fundamentals

Next up is using Pedestal's intuitive routing features to return some data. We'll also filter and expand that data at different endpoints when necessary.

As you work through these instructions, start just under your namespace declaration.

Once again, in core.clj, define some static superhero data to play with:

(def heroes [{:name "Clark" :surname "Kent"  :hero "Superman"}
             {:name "Bruce" :surname "Wayne"  :hero "Batman"}
             {:name "James" :surname "Logan"  :hero "Wolverine"}
             {:name "Steve" :surname "Rogers" :hero "Captain America"}
             {:name "Bruce" :surname "Banner" :hero "Hulk"}])

Then define two additional route functions besides your root "Hello World". They should:

Return all the superheroes we have defined
Return the hero matching the name provided as a route parameter

(defn get-heroes [_] {:status 200 :body heroes})

(defn get-hero [{{:keys [hero]} :path-params
                {:keys [extended]} :query-params}]
 (if-let [hero (->> heroes
                    (filter #(= hero (:hero %)))
                    first)]
   {:status 200 :body (if extended hero (dissoc hero :hero))}
   {:status 404}))

Be sure to update your route map accordingly. While we're here, extract your routes to live outside of the API specification. That will keep our functions and definitions short:

(def routes #{["/" :get hello-world :route-name :hello-world] ;Routes
              ["/heroes/:hero" :get get-hero :route-name :get-hero]
              ["/heroes" :get get-heroes :route-name :get-heroes]})

(def service-map
  {::http/routes routes
   ::http/type :immutant
   ::http/host "0.0.0.0"
   ::http/join? false
   ::http/port (Integer. (or (env :port) 5000))}) ; Service map

While Pedestal does a lot for us and the code above is simple, it's worth dissecting it to explain some of the Clojure features that keep code concise.

Each route in Pedestal is technically a specialized Interceptor. Interceptors are a common web framework concept used in things like Angular as well as Pedestal. Simply put, they let you layer functions to create a "pipeline" to handle web requests. We will return to the concept of interceptors later.

Pedestal routes take a request map and return a response map with the response details.

(defn hello-world [request] {:status 200 :body "Hello"})

The request map contains query parameters under the :query-params key and path parameters under the path-params key. Typically, to get the data we need from a map, we need to access them by keys. Here's what that would look like in our hello-world method, attached to our root route:

(defn hello-world [request]
 (let [query-params (:query-params request)
       path-params (:path-params request)]) {:status 200 :body "Hello"})

From there, you could access the specified parameter with either a threading macro or using get-in...

(defn hello-world [request]
 (let [extend (get-in request [:query-params :extended] )
       path-params (get-in request [:path-params :hero])]) {:status 200 :body "Hello"})

This method is legit — but it's a little wordy. To fix that, Clojure supports destructuring in binding statements. Destructuring is syntactic sugar that saves lines of code when accessing maps or sequential data structures.

Consider a nested map like this one:

(def locations {:type "Office" :address {:street "Ocean Drive" :zip 90210 :city "Beverly Hills" }})

It is possible to destructure the map and access particular keys in several ways:

; Destructuring in a let statement
(let [{:keys office} locations] (println office))
 
; Destructuring in a function
(defn print-office [{:keys office}] (println office))
 
; It also works in nested maps
(let [{{:keys [city]} :address} locations] (println city))
 
; Destructuring Sequential structures, like our superhero data
(defn simple-heroes  ["Clark" "Kent" "Superman"])
(let [[name surname hero]] (println name " - " surname " - " hero))

Destructuring has a lot of facets we cannot explore here, but the official documentation contains a detailed guide on how best to use this feature. However, with this crash course, we should now understand what's happening at the beginning of our function get-hero:

(defn get-hero [{{:keys [hero]} :path-params
                {:keys [extended]} :query-params}] {})

(defn get-hero [{{:keys [hero]} :path-params
                {:keys [extended]} :query-params}]
                ;...

...which can be interpreted as saying: "From path params extract the hero key; from query, params extract the extended key."

In case you're thinking: "hey, that looks a bit like SQL query expressed as data structure" (I'm psychic) — you're onto something. Indeed, Datalog is a query language that expresses queries as data. It's used by Datomic, which we will explore in a future article.

As a counter-example where destructuring might not make sense, take a look at the filter statement we are using to find the hero by name in get-hero:

(filter #(= h (:hero %)))

With destructuring, you need to "fall back" to a regular fn definition instead of a "lambda". That means the code would look like this:

(filter (fn [{:keys hero}] (= h hero)))

To destructure, we need to "fall back" to a regular fn definition instead of a "lambda". There's not a dramatic improvement in readability, so I suggest accessing the key explicitly.

Serving Data

After applying the modifications above, our code should look like this:

(ns app.core (:require [io.pedestal.http :as http]
                       [environ.core :refer [env]]))

(def heroes [{:name "Clark" :surname "Kent" :hero "Superman"}
             {:name "Bruce" :surname "Wayne" :hero "Batman"}
             {:name "James" :surname "Logan" :hero "Wolverine"}
             {:name "Steve" :surname "Rogers" :hero "Captain America"}
             {:name "Bruce" :surname "Banner" :hero "Hulk"}])

(defn hello-world [request]
  (let [extend (get-in request [:query-params :extended])
        path-params (get-in request [:path-params :hero])]) {})

(defn get-hero [{{:keys [hero]} :path-params
                 {:keys [extended]} :query-params}]
  (if-let [hero (->> heroes
                     (filter #(= hero (:hero %)))
                     first)]
    {:status 200 :body (if extended hero (dissoc hero :hero))}
    {:status 404}))

(defn get-heroes [_] {:status 200 :body heroes})

(def routes #{["/" :get hello-world :route-name :hello-world]
              ["/heroes/:hero" :get get-hero :route-name :get-hero]
              ["/heroes" :get get-heroes :route-name :get-heroes]})

(def service-map
  {::http/routes routes
   ::http/type :immutant
   ::http/host "0.0.0.0"
   ::http/join? false
   ::http/port (Integer. (or (env :port) 5000))}) ; Service map

(defn -main [_] (-> service-map http/create-server http/start)) ; Server Instance

If we start the server again now and try to get some data, we should now be able to get results and test out our new endpoints:

curl http://localhost:5000/heroes
[{:name "Clark", :surname "Kent", :hero "Superman"} {:name "Bruce", :surname "Wayne", :hero "Batman"} {:name "James", :surname "Logan", :hero "Wolverine"} {:name "Steve", :surname "Rogers", :hero "Captain America"} {:name "Bruce", :surname "Banner", :hero "Hulk"}]
 
curl http://localhost:5000/heroes/Superman
{:name "Clark", :surname "Kent"}
 
curl "http://localhost:5000/heroes/Superman?extended=1"
{:name "Clark", :surname "Kent", :hero "Superman"}

You can indeed see the API is behaving as you'd expect. However, this response isn't JSON - which you can see from the syntax or by looking at the response details via curl -v:

$ curl localhost:5000/heroes/Superman -v
...
> Content-Type: application/edn

EDN is arguably a better format than JSON for several reasons, especially if you're using ClojureScript. However, the lingua franca of publicly exposed APIs and JavaScript applications is JSON. For this reason, you're going to modify our application to return data in JSON by default.

To do so, you need to modify our Service Map (where we specify our options) and add a new interceptor that will serialize the body to be JSON whenever it makes sense. The good news is that Pedestal provides an interceptor out of the box we can use.

Our code's http/create-server function will look for the ::http/interceptors key in the service map. If it does not exist there, it will automatically use some default interceptors that perform a series of common tasks. Since we need to add a new interceptor, we're going to need to update the interceptor chain by hand:

(def service-map (-> {::http/routes routes
                     ::http/type   :immutant
                     ::http/host   "0.0.0.0"
                     ::http/join?  false
                     ::http/port   (Integer. (or (env :port) 5000))}
                http/default-interceptors
                (update ::http/interceptors into [http/json-body])))

Here we:

Define the service with the routes and other keys
Pass the map to the http/default-interceptors function that will return a new service map with the ::http/interceptors key
Add the json-body interceptor to the chain to serialize the data as JSON when appropriate.

If we try to request the same data again using curl we can see now that the response is JSON encoded:

curl http://localhost:5000/heroes/Superman
{"name":"Clark","surname":"Kent"}

You could apply the exact same approach to incoming body parameters if you had a route that used them. I won't dive deeply here, but the installation method for the body-params interceptor is the same as other interceptors.

We've completed a significant milestone with us returning data in the format most people expect when interacting with a Web API! But we're not done. Any Web API worth its salt should be well protected. You'll use Auth0 and some custom interceptors to accomplish that in my next article.