Elixir HTTP Benchmark - Ace vs Cowboy

This is a minimal happy-path benchmark of the ACE HTTP 2 server. It is a measurement of simple HTTP HEAD and GET requests with varying payload size - no other HTTP headers are used or tested.

Percentiles

The timings are made with Apache Benchmark using concurrency level of 16 - the keep alive flag -k is not used. The server is a 16 vCPU AMD EPYC 7000.

ab [-i] -n 100000 -c 16 -e /tmp/results.csv http://localhost:xxxx/1000

For controls we test against NGINX serving static files on the same server, and Cowboy running in both static mode and with HTTP handlers.

The requests are HEAD and GET with response payloads of 1, 10, 100, 1.000, 100.000 Bytes. The HTTP handlers are supplied by Raxx.

defmodule Tmp.AceRaxxServer do

  use Ace.HTTP.Service, port: 8065, cleartext: true
  use Raxx.SimpleServer

  @impl Raxx.SimpleServer
  def handle_request(%{method: :HEAD, path: _}, _state) do
    response(:ok)
  end

  def handle_request(%{method: :GET, path: ["1000"]}, _state) do
    response(:ok)
    |> set_header("content-type", "text/plain")
    |> set_body(@b1000)
  end
  ...  

Results

The 99th percentile timings show that all the servers handle the request within 1-2ms of each other:

HTTP Benchmark

The 100th percentile (longest) timing can be quite high for Ace - it can be 1s and this is normally the very first request to a cold freshly started BEAM.

The raw charts:

NGINX Static

Cowboy Static

Cowboy Raxx

Ace Raxx

The server was running at about 50% CPU during the BEAM tests, and nearly idle during the NGINX test:

Benchmark on m5a.4xlarge

Charts are made with the gnuplot library.