Deploying an Elixir app to Digital Ocean with mix_deploy
By DevOps on Mon 13 January 2020
inThis is a gentle introduction to getting your Elixir / Phoenix app up and running on a server at Digital Ocean (affiliate link). It starts from zero, assuming minimal experience with servers.
It builds the app on the server, then uses Erlang releases to run the code under systemd. It uses the mix_deploy library to handle deployment tasks.
Digital Ocean's smallest $5/month plan runs Elixir great. This guide uses their managed databases service so you don't need to manage the database.
We will be using a boilerplate Phoenix project with PostgreSQL database. It assumes you are running macOS on your dev machine and Ubuntu 18.04 on the server.
These instructions are based on this working example application and the principles described in the blog post "Best practices for deploying Elixir apps".
This post includes basic instructions to prepare your existing Elixir/Phoenix application for deployment using mix_deploy. See preparing an existing project for deployment for more details.
If you have any questions, contact me on the Elixir Slack at jakemorrison
or
open an issue on GitHub.
Overall approach
- Create the server
- Configure ssh
- Configure the build / deploy user
- Check out code on the server from git and build a release
- Deploy the release
You can first get the template running, then prepare your own project for deployment.
NOTE: This guide works with Ubuntu 18.04, CentOS 7, Ubuntu 16.04, and Debian 9.4. If you are not sure which distro to use, choose Ubuntu 18.04. The approach here works for dedicated servers and cloud instances as well.
The actual work of building and deploying releases is handled by simple shell scripts which you run on the build server or from your dev machine via ssh, e.g.:
ssh -A deploy@web-server
cd build/mix-deploy-example
git pull
# Build release
bin/build
# Extract release to target directory on local machine, creating current symlink
bin/deploy-release
# Run database migrations
bin/deploy-migrate
# Restart the systemd unit
sudo bin/deploy-restart
Create the server
Go to Digital Ocean (affiliate link) and create a Droplet (virtual server).
- Choose an image: Choose Ubuntu 18.04
- Choose a plan: Standard is fine
- Choose a size: The smallest, $5/month Droplet is fine
- Choose a datacenter region: Select a data center near you
- Add your SSH keys: Select the "New SSH Key" button, and paste the
contents of your
~/.ssh/id_rsa.pub
file. Create an ssh key, if you don't have one already. On Mac OS, you can copy your SSH key to clipboard by runningcat ~/.ssh/id_rsa.pub | pbcopy
. - Choose a hostname: The default name is fine, but awkward to remember and type. Use "web-server" or whatever you like
The defaults for everything else are fine. Click the "Create" button.
Configure ssh to talk to your server
Note the IP address of your new droplet in the Digital Ocean UI.
Configure ~/.ssh/config
on your local dev machine so you can
connect to the server.
# Change the IP address below to the actual IP address of your Droplet
Host web-server
HostName 123.45.67.89
Create a deploy user on the web server
For security, we use two operating system user accounts, the deploy
user to
build and deploy the app, and the app
user to run the app.
Connect to the web server as root:
ssh root@web-server
Create the deploy
user:
useradd -m -s /bin/bash deploy
Configure sudo
to allow the deploy
user run commands as root without a password:
echo "deploy ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/10-app-deploy
There are more sophisticated ways to manage users. We normally manage user accounts with Ansible.
Configure ssh access to the deploy
user.
Create the .ssh
directory and set permissions:
mkdir -p ~deploy/.ssh
chown deploy:deploy ~deploy/.ssh
chmod 700 ~deploy/.ssh
Allow the ssh key you set for the droplet root user to log into the deploy
user account:
cp ~root/.ssh/authorized_keys ~deploy/.ssh
chown deploy:deploy ~deploy/.ssh/authorized_keys
chmod 600 ~deploy/.ssh/authorized_keys
Exit the ssh session and connect again using the deploy
user.
ssh -A deploy@web-server
If it doesn't work, check that the ssh key on your dev machine
(.ssh/id_rsa.pub
) is in the ~/.ssh/authorized_keys
file for the deploy
user and
check the file permissions. Try with -vv
or look at /var/log/auth.log
on
the server.
The -A
flag on the ssh command gives the session on the server access to your
local ssh keys without copying them to the server. If your local user can
access a GitHub repo, then you can do it on the server.
Make sure that the deploy
user can run commands with sudo:
sudo -s
exit
Check out the app source
As the deploy
user on the build machine, create the build dir:
mkdir -p ~/build
Check out the app source:
cd ~/build
git clone https://github.com/cogini/mix-deploy-example # or your app repo
cd mix-deploy-example
Install build dependencies
In order to build the app, we need to install Erlang, Elixir and Node.js on the build server. Run the following script to install Erlang, Elixir and Node.js from OS packages:
sudo bin/build-install-deps-ubuntu
See the instructions on the Elixir website for more details on installing Elixir and dependencies.
We generally use ASDF to manage build tools. That allows us to precisely specify versions and install multiple versions at once.
Create the database
Most apps use a database. You can install the database on the same Droplet as you run your app. It works fine and is cheaper, but then you have to manage the db. This guide uses Digital Ocean's managed databases service.
In the Digital Ocean UI, select Create → Databases.
- Choose a database engine: Select PostgreSQL 11
- Choose a cluster configuration: $15/month is fine
- Choose a datacenter: Use the same data center as your droplet
- Choose a unique database cluster name: The default name is fine
While the database is being created, in the “Getting started” section of the page, click the bullet point that says “Secure this database cluster.” Under “Restrict inbound connections” select your droplet and click “Allow these inbound sources only.” This ensures that only your application server can connect to the database.
Create app databases and users
If you are creating a database per app, you can use the defaultdb
database and doadmin
user that the setup wizard created for you. However, it
is better to create a separate database and database user
for each app environment.
Configuration
There are four kinds of things that we may want to configure:
-
Static data, e.g. file paths. This is the same for all machines.
-
Settings specific to the environment, e.g. the hostname of the db server.
-
Secrets such as db passwords, API keys or TLS certificates.
-
Dynamic attributes such as the IP addresses of the server or other machines in the cluster.
In a simple deployment, you can put all the configuration in config files like
config/prod.exs
. It will then be compiled into the release package.
When building on a different machine, e.g. a CI/CD system, it's more secure to keep secrets separate from the release and load them at runtime. Similarly, we may want to build a release and run it in a test environment, then deploy it in production.
In these cases, we keep the config outside the release file and load it at runtime. We might read it from environment variables or external config files. Or we might read it from a system like AWS Systems Manager Parameter Store. See "Best practices for deploying Elixir apps" for more details.
Building
On your build machine, build the app by running the build script:
bin/build
In addition to the normal Phoenix build steps, this command sets up the deploy scripts by
running the following mix_systemd
and mix_deploy
commands:
# NOTE: run by bin/build
mix systemd.init
mix systemd.generate
mix deploy.init
mix deploy.generate
The configuration is minimal. We just change the name of the OS user that the
app runs under to app
in config/prod.exs
:
config :mix_systemd,
# Run db migrations before starting the app
# exec_start_pre: [
# [:deploy_dir, "/bin/deploy-migrate"]
# ],
app_user: "app",
app_group: "app"
config :mix_deploy,
# Generate runtime scripts from templates
templates: [
# Systemd wrappers
"start",
"stop",
"restart",
"enable",
# System setup
"create-users",
"create-dirs",
"set-perms",
# Local deploy
"init-local",
"copy-files",
"release",
"rollback",
# DB migrations
"migrate"
],
app_user: "app",
app_group: "app"
Prepare runtime configuration
Edit config/prod.exs
with the runtime settings:
import Config
config :mix_deploy_example, MixDeployExample.Repo,
username: "doadmin",
password: "CHANGEME",
database: "defaultdb",
hostname: "db-postgresql-sfo2-xxx-do-user-yyy-0.db.ondigitalocean.com",
port: 25060,
ssl: true,
pool_size: 15
config :mix_deploy_example, MixDeployExampleWeb.Endpoint,
secret_key_base: "CHANGEME2"
You can generate a unique value for secret_key_base
using this command:
mix phx.gen.secret
Build
Build the app and make a release:
bin/build
Initialize local system
Run this once to set up the system for the app, creating users and directories for releases, runtime configuration, etc.:
sudo bin/deploy-init-local
As this script changes group membership, you should log out and in again to reload user privileges.
Deploy the app
Deploy the release to the local machine:
# Extract release to target directory, creating current symlink
bin/deploy-release
# Run database migrations
bin/deploy-migrate
# Restart the systemd unit
sudo bin/deploy-restart
Check that it works
Make a request to the server:
curl -v http://localhost:4000/
You can get a console on the running release:
sudo -i -u app /srv/mix-deploy-example/bin/deploy-remote-console
You can also have a look at the logs:
sudo systemctl status mix-deploy-example
sudo journalctl -r -u mix-deploy-example
You can roll back the release with the following:
bin/deploy-rollback
sudo bin/deploy-restart
Configure the server to listen on port 80
Listening on port 4000 might be fine if it's behind a load balancer, otherwise we need to make the app available on port 80. There are two ways to do this:
-
Port forwarding using
iptables
(see Port forwarding with iptables) -
Reverse proxy using a web server, (see Serving your Phoenix app with Nginx)
After you complete this step, you should be able to access your website in the browser by navigating to your droplet's public IP address.
SSL
The steps necessary to get SSL set up with your Phoenix application depend on
the approach that you took in the previous step. If you are forwarding ports
using iptables
, then you should set up SSL in your application's endpoint,
as described in Phoenix docs.
You can get an SSL certificate for free from Let's Encrypt.
If you are running behind an Nginx reverse proxy, you should instead set up SSL in Nginx. The necessary steps are described in Digital Ocean's tutorials.
How to prepare your Phoenix app for deployment
Following are the steps used to set up this repo. You can do the same to add it to your own project. This repo is built as a series of git commits, so you can see how it works step by step.
Generate Phoenix project
mix phx.new your_app
mix deps.get
cd assets && npm install && node node_modules/webpack/bin/webpack.js --mode development
- Add
mix.lock
to git - Add
package-lock.json
to git
Configure releases
Elixir 1.9 has built in support for creating releases. For earlier versions, use the Distillery library.
Generate initial config files in the rel
dir:
mix release.init
Check the rel
directory into git.
In mix.exs
, tell Mix not to include Windows executables in releases. In the main project
configuration, add the option :releases
:
def project do
[
app: :mix_deploy_example,
version: "0.1.0",
elixir: "~> 1.9",
elixirc_paths: elixirc_paths(Mix.env()),
compilers: [:phoenix, :gettext] ++ Mix.compilers(),
start_permanent: Mix.env() == :prod,
aliases: aliases(),
deps: deps(),
# add this line:
releases: releases()
]
end
Then, in the same file, add a private function that returns your project's release configuration:
defp releases do
[
# change this to your application name
mix_deploy_example: [
include_executables_for: [:unix]
]
]
end
Tune the OS for performance
For optimal performance, it is recommended that you increase the default limits for open TCP ports and file handles. Follow the instructions in the post Tuning TCP ports for your Elixir app.
Add runtime config files
Loading runtime configuration from Elixir source files in Elixir 1.9 is very straightforward.
The file config/releases.exs
is copied into your release and evaluated at startup, and you
can load your Elixir configuration file from within that file. Edit config/releases.exs
:
import Config
import_config "/etc/mix-deploy-example/config.exs"
Create a config/prod.secret.exs.sample
that you can use to generate production
configuration files on the build server:
import Config
# Change these identifiers to ones specific to your application
config :mix_deploy_example, MixDeployExample.Repo,
username: "CHANGEME",
password: "CHANGEME",
database: "CHANGEME",
hostname: "CHANGEME",
port: 5432,
ssl: true,
pool_size: 15
config :mix_deploy_example, MixDeployExampleWeb.Endpoint,
secret_key_base: "CHANGEME2"
Add migrator module (optional)
In order for the bin/deploy-migrate
script to work properly, you need to add a migrator
module to your project. The instructions on how to do so are described in the post
Running Ecto migrations in a release.
If your build server and production server are the same machine, you can also skip this
step and just run your migrations with MIX_ENV=prod mix ecto.migrate
.
Set up ASDF
Create a .tool-versions
file in the root of your project, describing the versions
of OTP, Elixir, and Node that you will be building with:
erlang 21.3
elixir 1.9.0
nodejs 10.16.0
Install mix_deploy and mix_systemd
Add libraries to deps from Hex:
{:mix_systemd, "~> 0.5.0"},
{:mix_deploy, "~> 0.5.0"}
Or from GitHub:
{:mix_systemd, github: "cogini/mix_systemd", override: true},
{:mix_deploy, github: "cogini/mix_deploy"},
end
Add rel/templates
and bin/deploy-*
to .gitignore
:
echo '/rel/templates' >> .gitignore
echo '/bin/deploy-*' >> .gitignore
Copy build and utility scripts into your repo
Copy shell scripts from the bin/
directory of the mix-deploy-example
repo to the bin/
directory of your project.
These scripts build your release or install the required dependencies:
build
build-install-asdf
build-install-asdf-deps-centos
build-install-asdf-deps-ubuntu
build-install-asdf-init
build-install-asdf-macos
build-install-deps-centos
build-install-deps-ubuntu
This script verifies that your application is running correctly:
validate-service
Check these scripts into git.
Configure for running in a release
In config/prod.exs
, uncomment or add this line so that Phoenix can run correctly in a release:
config :phoenix, :serve_endpoints, true
In the same file, configure mix_deploy
and mix_systemd
to run your application
as the app
user. This step is mandatory:
config :mix_deploy,
app_user: "app",
app_group: "app"
# Minimal
config :mix_systemd,
app_user: "app",
app_group: "app"
Still in prod.exs
, configure your application's endpoint to fetch port number from environment
variables. The corresponding variable will be set by systemd
:
config :your_app_name, YourAppNameWeb.Endpoint,
http: [:inet6, port: System.get_env("PORT") || 4000],
# ...
Confirm that everything compiles by building the app:
mix deps.get
mix deps.compile
mix compile
You should be able to run the app locally with:
# Create development database
mix ecto.create
# Compile assets with production settings
(cd assets && npm install && npm run deploy)
mix phx.server
open http://localhost:4000/
If everything seems to work, you can proceed with deployment just like you did
with the mix-deploy-example
sample application.