How to automatically set up a development machine with Ansible

Have you ever wanted to automate the tedious installation and configuration of programs that you use every day? Would you like to automate it so that you can do it again and again on multiple computers without much effort, e.g. when you get a new laptop? Would you like to update your configuration once and update all laptops and servers you use at once? Then read on as we set up a development machine together, using the automation tool Ansible.

What we will achieve

Setting up a new computer involves two main activities: installing software and configuring it. That's why I will focus on this two activities in the article. I will assume that:

• We want to install some software packages from different sources.
• We have some dot files in a remote git repository that we want to download and copy to their respective locations.
• We want to do it all automatically, so that we can get the same environment when setting up or updating a machine, be it our new laptop or a server.

Specifically, we will install command line tools bat for viewing files, exa for listing directories and Vim. All of them provide much nicer experience than standard tools cat, ls and vi.

We will also install Podman as (more than) an alternative Docker client and because we like Python, we will make sure the package manager Poetry is installed on the system too.

We will store and source our custom dot files (config files starting with .) from a remote Git repository. We will work with two such files: .vimrc for configuring Vim and .aliases for storing shell aliases.

In our Vim config we allow the movement of the cursor besides the last character on the line:

set virtualedit=all

Our aliases will replace ls and docker commands with our newly installed alternatives:

alias ls='exa -la'
alias docker=podman

With that we will be able to still list directories with ls, but get a nicer output:

To make it work we will need to move the config files from the remote repository to the correct locations like ~/.vimrc and ~/.bashrc.

Using Ansible

To make all of that happen, we will use Ansible. It is an agent-less software automation tool that executes tasks on typically remote systems using SSH connections. It is often used to provision and configure servers, but as we will see - there are other use cases. We could do the same with plain shell scripting, however there are some advantages when using Ansible. It makes it easier to get a cross-platform solution if needed, can run on multiple different machines at once without much effort and provides automatic idempotency so that actions are not performed unnecessarily.

The article assumes that we are on Fedora, but we can easily tweak it for a different system by replacing dnf package manager and dnf Ansible module with a package manager for the respective system.

Let's start by installing Ansible:

sudo dnf install ansible -y 

Ansible playbook

The basic Ansible artefact is an Ansible playbook. It is a YAML file defining a set of tasks that we want to perform. Besides the tasks themselves, a playbook can contain some additional configuration.

We will start with the following template:

- hosts: localhost
  become: yes
  vars:
    - user: 'pstribny'
    - dotfiles_repo: 'git@github.com:stribny/dotfiles.git'
    - ssh_key: '.ssh/id_rsa'

  tasks:
    - name: "Name of the task"
      ansible_module: 
        module_parameter:
          - value
          - value2
      task_parameter: value

hosts: defines the set of machines where the tasks will be performed. As we are primarily interested to configure our own machine, we use localhost as the value. If we want to configure a remote machine, we can easily replace it with an IP address. There are also other options how to define what hosts to target, e.g. by specifying it on the command line when we run the Ansible playbook, or using an inventory file.

become: yes will execute the defined tasks under the root user. We could also specify become_user: user if we want to execute the tasks under a specific user on the target machine.

vars: lists our own variables that we can reuse throughout the playbook. user will be used later on to construct the path to the user's directory, dotfiles_repo points to a remote Git repository from where we will source our dot files and ssh_key is the path to an SSH key relative to the user's home directory that will grant the access to that repository. If you want to run the final playbook from the article, these are the values that you will need to replace.

The tasks: section of the playbook defines tasks. This is only an example for now, showing that every task will have a name, a set of parameters, and specify which Ansible module should perform the task. We will be using only build-in Ansible modules, so there is no need to install anything else. We can also see two types of values: a list of parameters or a single value can be provided depending on the parameter.

Installing packages

Ansible supports many package managers out of the box. We will now install the mentioned system packages for Fedora using dnf module and Python packages using pip module. The basic usage of these modules is the same, and it is also easy to change it to a different package manager. Ansible also features a package module which would be handy when creating cross-platform configuration. Let's create the tasks:

  tasks:
    - name: "Install system packages"
      dnf: 
        name:
          - vim 
          - bat
          - exa
          - podman
        state: latest

    - name: "Install Python packages"
      pip:
        name:
          - poetry
        state: latest

We just list the packages under name: and tell Ansible what should be the resulting state:. The basic state is present which will ensure that the particular package is installed on the system, but here we specify latest that will also upgrade a package if there is a new version.

Downloading the git repository

Checking out a remote git repository can be done with git module. We need to set repo: to our remote repository, e.g. on Github, and dest: to be the place where to put the downloaded files:

  tasks:
    - name: "Check out dotfiles from repository"
      git:
        repo: "{{ dotfiles_repo }}"
        dest: ./tmp-dotfiles
        accept_hostkey: yes
        key_file: "/home/{{ user }}/{{ ssh_key }}"
        force: yes
      delegate_to: localhost
      run_once: true
      become: no

We can see the use of our variables in action. Ansible uses jinja2 syntax that will be familiar to Python programmers. All we need is to put our string in quotes and surround the variable itself with double curly braces, e.g. "{{ variable }}".

accept_hostkey: yes and key_file: will make sure that our local SSH key is accepted by the remote host and vice versa.

force: yes will discard any changes in the already cloned repository. This is because we only want to keep the temp directory as a copy and nothing else.

delegate_to: makes the git module to clone the git repository locally instead of cloning it on the target machine. In our case when hosts: localhost point to the same machine it might feel weird, but if we target a remote machine it would suddenly make sense. In our case we don't want to transfer SSH keys to the remote machines and clone it there, we want to use our computer to clone the repository and copy the files to remote hosts afterwards.

run_once: is here so that we could easily extend the number of machines we want to run it on. It will ensure that even if we are configuring 5 machines, the repository will be cloned just once.

Configuration

We need to place our configuration files in the correct locations. We want to place .vimrc in ~/.vimrc and include .aliases in the Bash config file ~/.bashrc. Let's start by copying both files to the home directory. This can be done via copy module. We need to specify src and dest paths and optionally can also set file ownership and permissions.

  tasks:
    - name: "Copy .vimrc"
      copy:
        src: ./tmp-dotfiles/.vimrc
        dest: "/home/{{ user }}/.vimrc"
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: '0644'

    - name: "Copy .aliases"
      copy:
        src: ./tmp-dotfiles/.aliases
        dest: "/home/{{ user }}/.aliases"
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: '0644'

The mode: '0644' will make sure that the owner {{ user }} can both read and write to the file. We could forbid ourselves writing to the file if we would like to emphasize that the configuration should be changed in the dot files repository first. I like to keep it writable since sometimes we might want to just test something before commiting it.

Now we need to solve the ~/.aliases file that doesn't do anything on its own, unlike ~/vimrc that will be picked up by Vim automatically. Instead of mixing our aliases file with .bashrc, we will include it with the following snippet:

# in ~/.bashrc
if [ -f ~/.aliases ]; then
  source ~/.aliases
fi

This simple Bash script will check whether a file ~/.aliases exists and if so, executes it. To achieve that with Ansible we are going to use blockinfile module for replacing text blocks in files. If we would want to replace only one line, there is also lineinfile module.

  tasks:
    - name: "Load aliases in .bashrc file"
      blockinfile:
        path: "/home/{{ user }}/.bashrc"
        block: |
          if [ -f ~/.aliases ]; then
            source ~/.aliases
          fi

The usage is simple, we just specify the file in question and a block of text that should be present inside it. The pipe | is called Literal Block Scalar and can be used in Ansible to specify a block of text where new lines and trailing spaces should be preserved.

The modified file won't look like we expect. Ansible will insert its own markers around the text:

# BEGIN ANSIBLE MANAGED BLOCK
if [ -f ~/.aliases ]; then
  source ~/.aliases
fi
# END ANSIBLE MANAGED BLOCK

The final playbook

The complete devenv.yml:

---
- hosts: localhost
  become: yes
  vars:
    - user: 'pstribny'
    - dotfiles_repo: 'git@github.com:stribny/dotfiles.git'
    - ssh_key: '.ssh/id_rsa'

  tasks:
    - name: "Install system packages"
      dnf: 
        name:
          - vim 
          - bat
          - exa
          - podman
        state: latest

    - name: "Install Python packages"
      pip:
        name:
          - poetry
        state: latest

    - name: "Check out dotfiles from repository"
      git:
        repo: "{{ dotfiles_repo }}"
        dest: ./tmp-dotfiles
        accept_hostkey: yes
        force: yes
        recursive: no
        key_file: "/home/{{ user }}/{{ ssh_key }}"
      delegate_to: localhost
      become: no
      run_once: true

    - name: "Copy .vimrc"
      copy:
        src: ./tmp-dotfiles/.vimrc
        dest: "/home/{{ user }}/.vimrc"
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: '0644'

    - name: "Copy .aliases"
      copy:
        src: ./tmp-dotfiles/.aliases
        dest: "/home/{{ user }}/.aliases"
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: '0644'

    - name: "Load aliases in .bashrc file"
      blockinfile:
        path: "/home/{{ user }}/.bashrc"
        block: |
          if [ -f ~/.aliases ]; then
            source ~/.aliases
          fi

We can run it with ansible-playbook devenv.yml --ask-become-pass. The --ask-become-pass option will prompt us for the password of the become: yes user. This is typically necessary when we want to set up our own computer (hosts: localhost) and might not be required if we connect to a remote machine with an SSH key, if the SSH key is for that user.

After the playbook runs we can test that the software is installed and configured. For instance, let's see the devenv.yml file in bat:

And that's all! We can now reuse and extend the Ansible playbook and save plenty of time. For instance, we could install PostgreSQL with Ansible and set up all our accounts. Let me know what you think at @stribny!

Last updated on 5.2.2021.