Elixir Macros Demystified, part 1: what are macros for anyway?

Phoenix on Rails is a comprehensive guide to Elixir, Phoenix and LiveView for developers who already know Ruby on Rails. If you’re a Rails developer who wants to learn Phoenix fast, click here to learn more!

Elixir macros are one of the language’s more advanced features. They’re the essential ingredient of Elixir metaprogramming - the art of writing code that generates other code at compile time. Macros aren’t the first thing a beginner should study, but they’re still a powerful tool that any serious Elixir developer needs to understand eventually.

But when I was getting started, I found the official Elixir metaprogramming documentation to be a little confusing. It explains things from the bottom up, starting with the lowest-level concepts like quote, unquote and quoted expressions, before it’s necessarily clear what metaprogramming is or what these concepts are actually useful for.

I don’t think this is the best way to learn. It’s like trying to explain how to build a plane by first teaching you about the different types of material used in a plane’s construction, before you’ve even seen a plane or understand how planes fly.

In this series of posts, I’ll take the opposite approach. I’ll teach you how to metaprogram in Elixir and write Elixir macros, but I’ll start from the top down. First we’ll understand what Elixir macros are at a high level and what they achieve. Then once you’ve got the overall picture, we’ll study the nuts and bolts.

Why we need macros in Elixir

You might not realise it, but you already use Elixir macros every day. Some of the language’s most fundamental constructs like def, if and case, are actually implemented as macros under the hood. And many popular Elixir tools and libraries make extensive use of macros - Phoenix, for example, uses them everywhere, such as in this router code:

defmodule MyAppWeb.Router do
  use MyAppWeb, :router

  scope "/", MyAppWeb do
    pipe_through :browser
    
    get "/users/:id", UserController, :show
  end

  …

scope/2, pipe_through/1 and get/3 are all macros, not functions. The above code makes macro calls that will be evaluated at compile time.

Or take a look at this Ecto query:

from(u in User, where: u.name == "George", order_by: [desc: u.inserted_at])

This syntax definitely confused me when I was a beginner. I didn’t understand where that u comes from, since we’re not declaring it or using it anywhere outside the call to from/1. It doesn’t look like something that would work in a regular Elixir function call - but it works, because from/1 is actually a macro.

Another library that makes heavy use of macros is ExUnit, Elixir’s in-built test library. Consider this simple test suite that deliberately fails:

defmodule Tests do
  use ExUnit.Case

  test "fail on purpose" do
    assert 2 + 2 == 5
  end
end

test/2 and assert/1 make a nice domain-specific language (DSL) for writing succinct, readable tests. And because they’re macros, they’re more powerful than if they were implemented as functions.

The power of macros

Let’s stick with ExUnit, as it provides a really great example of what macros make possible. Look at what happens if we run that failing test:

$ mix test tests.exs 

  1) test fail on purpose (Tests)
     tests.exs:4
     Assertion with == failed
     code:  assert 2 + 2 == 5
     left:  4
     right: 5
     stacktrace:
       tests.exs:6: (test)


Finished in 0.00 seconds (0.00s async, 0.00s sync)
1 test, 1 failure

Randomized with seed 511047

Something interesting is happening here. ExUnit doesn’t only tell us that the test failed: it prints a bunch of extra information about why it failed, including the precise expression (2 + 2 == 5) that was asserted.

This wouldn’t be possible if assert was a function defined with def.To understand why, let’s try writing our own simple testing library.

We’ll start with this basic test harness:

defmodule Tests do
  import Assertions

  def run_test() do
    IO.puts("run test:")
    assert 2 + 2 == 5
  end
end

Tests.run_test()

To make it work, we need an Assertions module with an assert/1. Let’s try defining this as a function with def:

defmodule Assertions do
  def assert(assertion) do
    if assertion do
      IO.puts("passed")
    else
      IO.puts("failed")
    end
  end
end

Run the test^[1] and it prints our failure message, but we don’t get any additional information:

$ elixir tests.exs
run test:
failed

What more can Assertions.assert/1 do? When we call assert 2 + 2 == 5, the expression 2 + 2 == 5 gets evaluated to false before it’s passed to the function. In other words, when we write this:

assert 2 + 2 == 5

… then we’re effectively just writing this:

assert false

So assert/1 knows the test has failed, but it’s lost all information about why the test failed and what was asserted.

Without metaprogramming, we’d be stuck. The only way to print detailed information about test failures would be to write everything out explicitly, e.g. like this:

defmodule Tests do
  def run_test() do
    IO.puts("run test:")

    if 2 + 2 == 5 do
      IO.puts("passed")
    else
      IO.puts("""
      Assertion with == failed
      code: assert 2 + 2 == 5
      left: 4
      right: 5
      """)
    end
  end
end

Tests.run_test()

$ elixir tests.exs
run test:
Assertion with == failed
code: assert 2 + 2 == 5
left: 4
right: 5

But this is hideous - and imagine how repetitive it would be if we needed to write every test like this.

What we really need is some way to work with a line of code like 2 + 2 == 5 as code, not as its evaluated result. Our ideal assert/1 would take raw expressions like 2 + 2 == 5 and transform them into something like the code above. That is, the line:

assert 2 + 2 == 5

Would be converted at compile time into the code:

if 2 + 2 == 5 do
  IO.puts("passed")
else
  IO.puts("""
  Assertion with == failed
  code: assert 2 + 2 == 5
  left: 4
  right: 5
  """)
end

In other words, we need a macro.

Elixir macros do exactly what we’re looking for - they take one piece of code and transform it into another piece of code at compile time. Used effectively, they can make your code much more productive and maintainable, and can be used to extend Elixir into beautiful domain-specific languages for specialised tasks such as testing.

This concludes the high-level explanation of what macros are. In the next post, we’ll learn more about how macros work and how to write them.