Building and managing applications in development or local environments can be a daily struggle for devs. Like anything, wouldn’t it be great if it were more straightforward? Luckily for us, there’s a tool called Dagger!
In this article, we will be demonstrating how you can use Dagger with Docker to improve your CI/CD pipelines.
Jump ahead:
CI/CD pipelines have become an essential part of our software development lifecycles. While they make it a lot easier to release software reliably and consistently, they also add a degree of complexity and have a bit of a learning curve for developers to bear in mind.
Most CI/CD tools are hosted on the cloud, meaning you will have to push your changes to a remote repository and typically wait a long time for the CI process to execute.
You may be thinking why we can’t just run it locally. Well, most CI/CD tools don’t work locally, and the ones that do are very difficult to set up. What we need is the ability to run our CI/CD pipelines locally on our machine, while also in a cloud-based CI/CD tool of our choice.
And that’s where Dagger comes in.
Dagger is an open source dev kit for CI/CD. It works using Cue, a powerful configuration language made by Google that helps to validate and define text-based and dynamic configurations. We will also be using BuildKit, in which Dagger’s configuration is executed.
Additionally, the client-server architecture has the same concept as that of Docker. The Dagger daemon engine, meanwhile, runs on any container runtime.
Dagger aims to solve many issues by standardizing the way we write CI/CD pipelines. In this tutorial, we’ll explore how to install and set up Dagger and build a container image.
Before we proceed, it is necessary to be familiar with the following technologies:
N.B., Dagger’s configuration is executed in BuildKit — this is one of the most-used features of Docker Engine
Before you begin, you’ll need to have Docker Engine installed, as this is required for Dagger to work. If you don’t have Docker installed, you can do so by using a Docker installation script.
Run the following to get started:
curl -fsSL https://get.docker.com -o get-docker.sh sh get-docker.sh
Once this is completed, you can run docker version to be sure Docker was installed correctly.
The next step is to install Dagger, and this can be done using the following installation script:
cd /usr/local curl -L https://dl.dagger.io/dagger/install.sh | sh
If you are not a super user, you may need to prepend the last command with sudo.
Once complete, run the command dagger version to confirm Dagger’s installation.
N.B., if you’re not running Linux or macOS, or you face any issues during installation, you can check out the official installation guide here.
In any CI system, you need to define steps and actions in a certain format (YAML, most likely) and run it in your CI system.
Dagger makes it possible to:
As noted in the official docs, actions are the basic building blocks of Dagger  —  they abstract complex automation into simple components that can be reused.
Actions can be executed using the following command:
dagger do <action_name>
A cool feature of Dagger is that it doesn’t need to distinguish between pipelines and steps; everything is simply an action.
There are two types of Dagger actions:
Composite actions have four lifecycle stages:
It all starts with a plan! The keyword dagger.#Plan is used to identify a plan in our Dagger configuration files.
A plan is an execution context for actions. It specifies:
Now, let’s move on to using Dagger to build container images.
Dagger can execute regular Docker files to build container images.
To begin, you need to create a main.py file with the following contents (this would be a Python application):
Learn more →# filename: main.py
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello_world():
return "<p>Hello, Dagger!</p>"
The next step is to create a Dagger configuration file; dagger.cue.
// filename: dagger.cue
package main
import (
"dagger.io/dagger"
"universe.dagger.io/docker"
)
#PythonBuild: docker.#Dockerfile & {
dockerfile: contents: """
FROM python:3.7-stretch
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential
COPY . /app
WORKDIR /app
RUN pip install flask
ENTRYPOINT ["python"]
CMD ["main.py"]
"""
}
dagger.#Plan & {
client: filesystem: "./": read: contents: dagger.#FS
actions: build: #PythonBuild & {
source: client.filesystem."./".read.contents
}
}
You can see from the Dagger configuration that the action builds a Docker image from the Python application we created earlier.
In addition, the build steps are defined using an inline approach that allows us to write the Dockerfile contents inside Dagger’s configuration.
It is required to initialize and update the project, as this installs these required Dagger packages. You can do this using the following commands:
dagger project init dagger project update
Now, we can proceed to executing the build plan using Docker. To trigger the build action, run the following command:
dagger do build
You can also confirm that Dagger has successfully built the container image using the Dockerfile contents by checking the list of Docker images to verify that the image exists, as shown here:
Now, you know how to build container images with Dagger!
Dagger is making an effort to replace the complicated Bash scripts that many existing CI/CD systems require in place of more modular, quicker, and portable components.
You may create reusable modules that are adapted to your needs and dependent on the tools you’re using to produce them, and other people can use your modules as a starting point to create new modules that are suited to their unique use cases.
As someone that works for a consulting firm, we have a number of use cases where we must frequently swap between projects that use different CI/CD stacks. Therefore, the promise to build an abstraction layer on top of a current CI runner seems intriguing to me — independent of CI runners, we could set up our own processes and apply them to our projects.
To finish up our article today, there are several use cases for why you may want to use Dagger with Docker for your CI/CD pipelines.
If you frequently write intricate build or deployment scripts, share large sections of scripts between teams, spend a lot of time attempting to debug problems in the CI, or prefer a local build to be an easy script, then Dagger may well be the tool for you.
Likewise, it might also be right for you if your CI pipeline is taking too long to do tasks serially when it might be parallelized. Finally, the project we’ve worked on today shows how simple it is to create container images using Dagger!
Here are some additional helpful resources you can use for your projects:
Let me know about your experiences using Dagger with Docker in the comments section below!
Install LogRocket via npm or script tag. LogRocket.init() must be called client-side, not
server-side
$ npm i --save logrocket
// Code:
import LogRocket from 'logrocket';
LogRocket.init('app/id');
// Add to your HTML:
<script src="https://cdn.lr-ingest.com/LogRocket.min.js"></script>
<script>window.LogRocket && window.LogRocket.init('app/id');</script>
This tutorial will explore the application of heatmaps in JavaScript projects, focusing on how to use the Heat.js library to generate them.
Eleventy (11ty) is a compelling solution for developers seeking a straightforward, performance-oriented approach to static site generation.
Explore some CSS tools that offer the perfect blend of efficiency and flexibility when handling CSS, such as styled-components and Emotion.
RedwoodJS announced support for server-side rendering and RSCs in its Bighorn release. Explore this feature for when it’s production-ready.
One Reply to "Using Dagger with Docker for better CI/CD"
Interesting article, I’ll look into dagger.
Btw, when you say “append in front of” you mean “prepend”