Clojure Start Time in 2019

From 2011 to 2015, I wrote an annual Clojure Year in Review post, attempting to summarize all the interesting things that happened in Clojure in the last year. After 2015, I gave up. There was just too much happening, and I couldn’t keep track of it all.

A couple of years ago, I got tired of the “Clojure start-up is slow” meme so I decided to measure it. I found that, yes, Clojure does take a measurable amount of time to boot, but the actual start time is dominated by tooling and libraries.

Since then, new ways of running Clojure have been popping up all over the net. I decided to repeat the experiment with all the ones I could find. Think of this as the “Clojure Runtime Platforms Year in Review” for 2019.

This is not meant to be a comprehensive benchmark. This is one set of anecdotal experiences on a 4-year-old laptop. I merely wanted to survey the options, and I wouldn’t be surprised if I missed a few.

Unlike my previous post, where I only measured time to start and immediately exit, this time I decided to benchmark a simple “Hello, world” program. I figure this is the smallest program that can still be considered “real” in that it “does something.”

Baseline: Java

I started with Java. Remember Java?

public class HelloWorld {
    public static void main(String args[]) {
        System.out.println("Hello, world!");
    }
}

Oh public static void, how I’ve missed you. Once a mere bit of technical jargon, you’ve become an evocative description of the world we live in.

Ahem.

Compiled with javac HelloWorld.java and invoked as java -cp . HelloWorld, this ran in a couple hundred milliseconds.

Clojure (JVM) Launchers

Leiningen

Still the most widely-used Clojure build tool, Leiningen is stable and has lots of plugins and integrations.

Leiningen launches two JVM processes, one for itself and one for your project. This gives it a lot of power and flexibility at the cost of a longer start-up time.

I made a tiny Clojure project with one source file:

(ns example.core)

(defn -main [& args]
  (println "Hello, world!"))

A trivial project.clj file:

(defproject example "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.10.1"]]
  :main example.core)

and ran it with lein run, which took about 3 seconds.

Clojure CLI / tools.deps

The Clojure Command Line Tools, a.k.a. tools.deps, also launch two JVMs on the first run, but after that the classpath is cached in a file so that subsequent runs only launch one JVM.

With the same source file and a minimal deps.edn file:

{:deps {org.clojure/clojure {:mvn/version "1.10.1"}}}

I ran clj -m example.core. The first run took about 5 seconds, but subsequent runs only 1.5 seconds.

Embedded ClojureScript

This is where things start to get interesting. ClojureScript has been capable of self-hosting, i.e., compiling itself, for some time now. With the rapid growth of JavaScript-based tooling, there are now multiple options for running ClojureScript as a standalone process without a JVM.

Planck

Planck is written in C and ClojureScript, packaged as a standalone binary with an embedded JavaScriptCore engine running self-hosted ClojureScript.

Planck comes with its own I/O and shell libraries ported from Clojure(JVM).

I ran it as planck -c src -m example.core, which averaged just under one second.

Lumo

Lumo is written in JavaScript and ClojureScript, packaged as a standalone binary with an embedded V8 JavaScript engine running self-hosted ClojureScript.

Lumo supports Node.js libraries and can compile ClojureScript into JavaScript without a JVM.

I ran it as lumo -c src -m example.core, which averaged just under half a second.

Alternative Implementations of Clojure

Here things get even more interesting. These projects are not built on either Clojure or ClojureScript, but aim to be source-compatible with some subset of “Clojure” the language.

Joker

Joker is an interpreter, written in Go, for a subset of Clojure. It comes with its own libraries, some ported from Clojure(JVM), others adapted from the Go standard library. It can also be used as a Clojure(Script) linter.

I wrote a script:

(ns hello)

(defn -main []
  (println "Hello, world!"))

(-main)

And ran it with joker hello.joke, which averaged around 50 milliseconds.

Small Clojure Interpreter / Babashka

This one took me a little while to wrap my head around, but it’s pretty cool. Sci is an interpreter for a subset of Clojure written from scratch in Clojure (cljc). It can be used as a library from Clojure, JavaScript, or Java.

Babashka packages Sci as a standalone binary with GraalVM. What you end up with is a small, fast interpreter suitable for scripts and one-line shell invocations.

I ran it as bb -cp src -m example.core, which finished in an impressive 13 milliseconds.

Sci is the basis for several other utility programs including the clj-kondo linter [correction: not clj-kondo, but others such as bootleg and dad].

I only wish it were named scittle, pronounced “skittle.” Or science.

Clojure-like languages

Going even further afield, these projects do not necessarily aim for compatibility with “Clojure” but are inspired by it to varying degrees.

Pixie

Pixie is an interpreter, written in RPython, for a language “heavily inspired by Clojure.” Using the RPython/PyPy tool chain, Pixie is compiled into a single native binary with its own GC and JIT.

Unfortunately, I couldn’t get Pixie to build successfully. I think was running into issue #535. The project has a few different committers, but development seems to have petered out in 2017.

Ferret

Ferret is a language “heavily inspired by Clojure both syntactically and semantically.” It is written in and compiles to C++, designed for use in embedded systems.

I compiled a trivial Clojure program:

(defn -main []
  (println "Hello, world!"))

(-main)

Which produced about 2800 lines of C++ and compiled into an 88 KB binary.

Not surprisingly, it started up very quickly, finishing in under 10 milliseconds.

Hy

Hy compiles a Clojure-like syntax into Python. It’s more Python than Clojure, with Python-native data structures and functions, but it adds Clojure-like syntax, macros, and let-scoped locals.

I wrote a script like:

#!/usr/bin/env hy
(print "Hello, world!")

Which ran in a bit under 100 milliseconds.

Native-compiled Clojure

But wait, there’s more! With GraalVM it is now possible to compile Clojure(JVM) all the way down to a native binary. Babashka uses this for its own mini-Clojure interpreter, but we can theoretically compile any Clojure program into native machine code.

I made another trivial “Hello world” program and compiled it via the lein-native-image plugin. It took a couple of tries, tinkering with the native-image configuration, and I had to downgrade to Clojure 1.9.0 because of a known compatibility issue with 1.10. But it worked, and the result was comparable to other native-compiled code: under 10 milliseconds.

My final project.clj looked like this:

(defproject example "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.9.0"]]
  :plugins [[io.taylorwood/lein-native-image "0.3.1"]]
  :native-image {:name "example"
                 :graal-bin "graalvm-ce-java11-19.3.0/Contents/Home/bin"
                 :opts ["--verbose"
                        "--report-unsupported-elements-at-runtime"
                        "--initialize-at-build-time"]}
  :main example.core)

Baseline: C

Just for kicks, I decided to compare the native-compiled options (Ferret, GraalVM-compiled Clojure) with actual native code written in C.

I dusted off enough memories to crank out a “Hello world” in C:

#include <stdio.h>

int main() {
  printf("Hello, world!");
}

Compiled via GCC with no special options, it ran in about 4 milliseconds.

Given the imprecision of these benchmarks — I’m just using time in a shell — I’d guess that anything under 10 milliseconds is lost in the noise. That puts Ferret and GraalVM-compiled Clojure in the same category as C, with Babashka only a few milliseconds late to the party.

Think about that for a minute: these are Clojure programs that start as fast as C.

Results

Once again, this is not a realistic benchmark, but here are my results.

For each runtime, I ran it 3 times as a warm-up, followed by 3 measurements. I took the real time measured by Bash time. The 3 measurements were consistent in every case; this table gives the averages:

Runtime Time(ms)
Java 195
Leiningen 3072
Clojure CLI 1503
Planck 924
Lumo 474
Joker 53
Babashka 13
Ferret 3
Hy 95
GraalVM 5
C 4

Java, Ferret, GraalVM-compiled Clojure, and C are compiled ahead-of-time, all others are interpreted or JIT-compiled from Clojure(Script) source.

clojure-startup-2019.png

Obviously, this is not in any way a fair comparison. Please don’t tweet some stupid, clickbait headline like “Joker 30x faster than Clojure.” “Hello world” is a silly benchmark for a programming language in general. But for certain specific use cases, like command-line scripts integrated into a larger process, the total execution time of a tiny program is something you might actually care about.

The point is, there’s a version of Clojure that fits almost any use-case you can think of.

If you want to write command-line utilities in a language that feels like Clojure, Joker and Babashka are both good options.

If you want the full power of ClojureScript but not the start-up cost of the JVM, Planck and Lumo are there for you.

If you need to work with Python but can’t bear life without S-expressions, check out Hy.

And if for some reason you are determined to write Lisp for an embedded microcontroller, Ferret has got you covered.

Finally, while GraalVM is still new and somewhat experimental, it’s going to be a game-changer. My compiled “Hello world” image is only 7.6 MB, and let’s not forget that’s with “real” Clojure and (some of) a real JVM!

I’m even more interested to see what happens when people combine these technologies in novel ways. Imagine a serverless application that starts a GraalVM-compiled binary for quick responses during cold-starts, then switches to a regular JVM for higher throughput once it’s warmed up. Better yet, imagine a deployment platform that could hide that difference from your application code.

Back in 1995, Java was pitched as “write once, run anywhere.” For the past twenty years or so, that mostly meant “write once, run on any machine with a fast CPU and several GB of memory where you don’t care about start-up time.” But that’s changing. We’ll probably never achieve “write once, run anywhere” for any arbitrary program. But these projects prove that we can write some program that runs almost anywhere using the same language we enjoy on the server.

3 Replies to “Clojure Start Time in 2019”

  1. Thanks a lot for doing this.

    One thing: for a fair comparison between say lein run and the compiled options, I’d have liked to see the time for compilation + startup.

    I’m interested in this type of speed for the edit/compile/run cycle mostly (where the seconds that lein takes really add up and create frustration).

  2. You forgot to mention using ClojureScript with Node, you can then use nexe to package the compiled JS as a self-contained executable as per the guide: https://clojurescript.org/guides/native-executables

    This works either with cljs.main compilation, or compiling with lumo as well.

    On my machine, hello-world is around 400ms. Where as running the hello world script directly with lumo is around 635ms, and 1.5s with planck. Which makes me believe my machine is bout 33% slower than the one used in the article. And thus, compiling the ClojureScript and than running it with Node or packaging it as an executable binary with nexe does result is faster startup times.

Comments are closed.