On a recent project, working with Elixir, Ecto, and Phoenix, we were encountering difficulty modeling and managing a complex business workflow. This is a fairly typical problem to have on any project—there have been many times I've known that I'm building an implicit finite state machine but don't take the time to make it explicit—but here it was becoming particularly unmanageable. It became clear that we needed to implement a more formal state machine to encode this logic.
Eventually, we implemented a solution that allows us to easily model these workflows in code, in a simple enough way that it also facilitates talking through these workflows and statecharts with non-technical project stakeholders. Read on to learn more about ExState, or try it out to see if it's useful to you.
The Problem
Tracking any complex process or workflow inevitably requires conditional logic based on the state of entities involved in the workflow. This can easily lead to a proliferation of fields and states used to track this process. Take an example Order process, for instance:
defmodule Order do
use Ecto.Schema
schema "orders" do
field :created_at, :utc_datetime
field :confirmed, :boolean
field :confirmed_at, :utc_datetime
field :shipped, :boolean
field :shipped_at, :utc_datetime
field :shipment_status, :string
field :cancelled, :boolean
field :cancelled_at, :utc_datetime
has_many :items, Item
end
end
In the above schema, we have four fields related to tracking the state of the order. These fields can change independently and it's unclear how they should relate to one another. Is %{confirmed: false, shipped: true, cancelled: true} a valid state? Probably not, but it would require a case by case validation to enforce that.
The order workflow and state becomes much easier to understand and validate if we collapse those fields into one state field:
defmodule Order do
use Ecto.Schema
schema "orders" do
field :state, :string
field :created_at, :utc_datetime
field :confirmed_at, :utc_datetime
field :shipped_at, :utc_datetime
field :cancelled_at, :utc_datetime
has_many :items, Item
end
end
defmodule Orders do
alias Ecto.Multi
def cancel(order) do
if order.state in ["pending", "confirmed"] do
order
|> Order.changeset(%{status: "cancelled", cancelled_at: DateTime.utc_now()})
|> Repo.update()
else
{:error, "can't cancel this order"}
end
end
def ship(order) do
if order.state == "confirmed" do
Multi.new()
|> Multi.update(:order, Order.changeset(order, %{status: "shipped", shipped_at: DateTime.utc_now()}))
|> Multi.run(:ship, fn _repo, %{order: order} -> Shipments.ship(order) end)
|> Repo.transaction()
else
{:error, "order must be in the confirmed state"}
end
end
end
We can then use the states ["created", "confirmed", "shipped", "delivered", "cancelled"] to represent our order workflow. But we still need to replace the shipment_state field and to express sub-states of the "shipped" state. We're also still required to do a lot of explicit checking of the current state before taking any action.
Statecharts
A lot of the workflows and processes we build in our software can be modeled as finite state machines. The system, or an entity in the system, is in one given state at any time. In that state, the system handles events and transitions to a next state with logic that is dependent upon the current state.
Elixir and Erlang already provide tools for implementing state machine behavior through gen_statem and its predecessor gen_fsm. These are great for implementing stateful processes, but aren't well-suited for state that's stored in a database with a request and response lifecycle.
Our order process resembles a finite state machine, but with a few additional details. In the "shipped" state, we want to track multiple shipment_status states. So the "shipped" state itself represents multiple states:
[
{"shipped", "pending"},
{"shipped", "in_progress"},
{"shipped", "arriving_soon"},
{"shipped", "out_for_delivery"}
]
Rather than a simple finite state machine, we have a hierarchical state machine, with certain states like "shipped" representing a lower level of states and transitions.
ExState
ExState was designed to simplify the definition of states, events, and transitions for simple or hierarchical state machines in Elixir. ExState can also persist these states and metadata to the database through Ecto for use in web apps or other database-backed applications.
Using ExState begins with a definition module. ExState.Definition defines the workflow macro, which builds a data representation of the state chart and binds it to the module and the associated functions for transitioning the workflow with a subject and other context data.
defmodule OrderWorkflow do
use ExState.Definition
workflow "order" do
subject :order, Order
initial_state :pending
state :preparing do
initial_state :pending
on :cancel, :cancelled
state :pending do
step :confirm
on_completed :confirm, :confirmed
end
state :confirmed do
step :ship
on_completed :ship, {:<, :shipped}
end
end
state :shipped do
initial_state :in_transit
on_entry :update_shipped_at
state :in_transit do
on :arriving_soon, :arriving_soon
end
state :arriving_soon do
on :out_for_delivery, :out_for_delivery
end
state :out_for_delivery do
on :delivered, {:<, :delivered}
end
end
state :delivered do
final
end
state :cancelled do
on_entry :update_cancelled_at
final
end
end
def update_shipped_at(%{order: order}) do
order
|> Order.changeset(%{timestamp => DateTime.utc_now()})
|> Repo.update()
end
def update_cancelled_at(%{order: order}) do
order
|> Order.changeset(%{timestamp => DateTime.utc_now()})
|> Repo.update()
end
defp update_timestamp(order, timestamp) do
order
|> Order.changeset(%{timestamp => DateTime.utc_now()})
|> Repo.update()
end
end
The subject of the workflow is an Ecto model that defines a workflow association using the has_workflow macro.
defmodule Order do
use Ecto.Schema
use ExState.Ecto.Subject
schema "orders" do
field :created_at, :utc_datetime
field :confirmed_at, :utc_datetime
field :shipped_at, :utc_datetime
field :cancelled_at, :utc_datetime
has_workflow OrderWorkflow
has_many :items, Item
end
end
The associated context module can then update the model and its workflow through ExState:
defmodule Orders do
alias Ecto.Multi
@doc """
Create an order and workflow in a Ecto.Multi transaction.
"""
def create_order(attrs) do
{:ok, %{order: order, workflow: workflow}} =
Multi.new()
|> Multi.create(:order, Order.new(attrs))
|> ExState.Multi.create(:order)
|> Repo.transaction()
end
@doc """
Load, complete step, and persist.
"""
def confirm_order(order) do
execution = ExState.load(order)
{:ok, execution} = ExState.Execution.complete(execution, :confirm)
{:ok, order} = ExState.persist(execution)
end
@doc """
Use `ExState.event/3` convenience function to load, transition, and persist.
"""
def cancel_order(order) do
{:ok, order} = ExState.event(order, :cancelled)
end
@doc """
Load, complete, ship, and persist in a transaction.
"""
def ship(order) do
{:ok, %{ship: shipped_order, shipment: shipment}} =
Multi.new()
|> ExState.Ecto.Multi.complete(order, :ship)
|> Multi.run(:shipment, fn _repo, %{order: order} ->
Shipments.ship(order)
end)
|> Repo.transaction()
end
def arriving_soon(order) do
{:ok, order} = ExState.transition(order, :arriving_soon)
end
end
States
States are defined in four main forms.
Atomic States
Atomic states have no child states. The following three states are atomic states:
workflow "example" do
initial_state :atomic_a
state :atomic_a do
on :next, :atomic_b
on :done, :atomic_done
end
state :atomic_b do
on :back, :atomic_a
end
state :atomic_done
end
Compound States
Compound states contain child states and specify an initial state (one of the child states). The :preparing and :shipped states in the order workflow are compound states.
Final States
A final state represents a state of either a child state or the entire workflow where the state should be considered "complete."
workflow "example" do
initial_state :a
state :a do
on_final :b
state :one do
on :did_one, :two
end
state :two do
final
end
end
state :b
end
Transient States
Transient states are used for dynamically resolving the next state based on conditions defined in the definition module. The transient state immediately handles a "null" event, :_, and transitions to the first state in the list that a guard allows.
defmodule SetupWorkflow do
use ExState.Definition
workflow "setup" do
initial_state :unknown
state :unknown do
on :_, [:accept_terms, :working]
end
state :accept_terms
state :working
end
def guard_transition(:unknown, :accept_terms, context) do
if context.user.has_accepted_terms? do
:ok
else
{:error, :accepted}
end
end
end
Actions
Actions are useful for triggering side effects on certain events. Actions are called as functions on the definition module, and should return :error | {:error, reason} if transactional execution should be halted when an action cannot be completed. Actions can also return an {:updated, subject} tuple to replace the updated subject in the execution state.
defmodule ExampleWorkflow do
use ExState.Definition
workflow "example" do
state :working do
on_entry :send_entry_email
on :cancel, :canceled, actions: [:send_cancelled_email]
on_final :done, actions: [:send_final_email]
end
end
def send_entry_email(_context), do: :ok
def send_final_email(_context), do: :ok
def send_cancelled_email(context) do
{:updated, Map.put(context, :document, %{context.document | cancellation_email_sent_at: DateTime.utc_now()})}
end
end
Guards
Guards can ensure that transitions are only made when certain conditions are met. The guard_transition/3 function on the definition module will be called during workflow execution with the current state, next state, and the context as arguments. A guard returns :ok to allow the transition, or {:error, reason to prevent the transition.
ExState doesn't rescue exceptions in guards or actions, so exception handling behavior is dependent upon the current database transaction context, if any.
Steps
Steps are a convenient way to collapse an implicitly linear set of states and events into an explicitly ordered list of events. This is useful for exposing required steps to UI components or API consumers. Steps can also be ignored through the use_step/2 callback.
state :working do
initial_state :adding_name
on_final :reviewing
state :adding_name do
on :name_added, :adding_email
end
state :adding_email do
on :email_added, :adding_phone_number
end
state :adding_phone_number do
on :phone_number_added, :confirming
end
state :confirming do
on :confirmed, :done
end
state :done do
final
end
end
state :reviewing
The above workflow could be rewritten using four explicit steps:
defmodule SetupWorkflow do
use ExState.Definition
workflow "setup" do
subject :account, Account
initial_state :working
state :working do
step :add_name
step :add_email
step :add_phone_number
step :confirm
on_completed :confirm, :reviewing
end
state :reviewing
end
def use_step?(:add_phone_number, context) do
context.account.phone_number_required?
end
def use_step?(_, _), do: true
end
{:ok, account} = ExState.create(account)
{:ok, account} = ExState.complete(account, :add_name)
{:ok, account} = ExState.complete(account, :add_email)
{:error, _reason} = ExState.complete(account, :confirm)
{:ok, account} = ExState.complete(account, :add_phone_number)
{:ok, %{workflow: %{state: "reviewing"}}} = ExState.complete(account, :confirm)
Decisions
Similar to steps, decisions are required events that transition based on the value provided in the decision.
defmodule ReviewWorkflow do
use ExState.Definition
workflow "review" do
initial_state :rating
state :rating do
step :rate
on_decision :rate, :good, :done
on_decision :rate, :bad, :feedback
end
state :feedback do
step :provide_feedback
on_completed :provide_feedback, :done
end
state :done do
final
end
end
end
{:ok, review} = ExState.create(review)
{:ok, review} = ExState.decision(review, :rate, :good)
Querying State
You'll likely want to use the workflow state in queries as well. ExState has some builtin queries to help with this:
import ExState.Ecto.Query
shipped_orders =
Order
|> where_any_state(:shipped)
|> Repo.all()
in_transit_orders =
Order
|> where_state([:shipped, :in_transit])
|> Repo.all()
confirmed_orders =
Order
|> where_step_complete(:confirm)
|> Repo.all()
Without Ecto
ExState can be used without persisting the workflow to the database, either for testing or in memory use cases. Just use the functions defined on the workflow definition module itself:
%{state: %{name: "delivered"}, context: %{order: order}} = execution =
OrderWorkflow.new(%{order: order})
|> OrderWorkflow.transition_maybe({:completed, :confirm})
|> OrderWorkflow.transition_maybe({:completed, :ship})
|> OrderWorkflow.transition_maybe(:out_for_delivery)
|> OrderWorkflow.transition_maybe(:arriving_soon)
|> OrderWorkflow.transition_maybe(:delivered)
{:error, _reason, execution} = OrderWorkflow.transition(execution, :cancel)
Notes
Try ExState on hex, read the docs, or check out the code on GitHub for additional documentation and examples.
Credit to David Khourshid and xstate for excellent documentation, examples, and API inspiration on this topic.