Vault: How to integrate with Jenkins?

Table of contents

Reading Time: 5 minutes

What is Vault?

Hashicorp Vault is a tool for managing our our secrets. It has support for multiple secrets and we can enable access to both humans and machines separately. Secrets can be stored, dynamically generated, and in the case of encryption, keys can be consumed as a service without the need to expose the underlying key materials.

In terms of its architecture, it has a lot of components which all require brief explanation of their own but will go outside the scope of this article. To move forward we just have to keep in mind that there exists a set of components which combine to form what we call the cryptographic barrier. No secret will ever leave this barrier as long as functioning of Vault is concerned.

Why use Vault with Jenkins?

One very common problem with keeping our secret credentials in Jenkins is how pipeline handles credentials. Whenever a pipeline obtains a secret that is scoped, there are no limitations to it on how much and in what ways the pipeline can use them. This poses a potential security threat considering the pipeline may be interacting with uncontrolled external.

The next problem with Jenkins is the use of its encryption methods. There is a static, single common master key and static, single common hudson.util.Secret object. Both of these combine to encrypt our credentials. If access to our Jenkins host is compromised, we risk leaking these two keys and thus are credentials will be exposed.

This is where Vault comes to our rescue. The first thing that it did was adding another layer of authentication over Jenkins. The next advantage for it was its access control policies. We can add multiple policies to access the same secret. Then we can call it in our pipelines keeping in mind how much access we want to give. We can even do as much as defining permission of access for machines and programs. This gives us better isolation and a patterned access rules.

Accessing Vault from Jenkins

Enabling access to Vault requires the installation of hashicorp-vault-plugin and a running Vault instance. Next, we need to decide the method of authentication we will be using for accessing it. Hashicorp provides us with multiple authentication methods which are lister below:

AppRole
AliCLoud
AWS
Azure
Cloud Foundry
GitHub
Google Cloud
JWT/OIDC
Kerberos
Kubernetes
LDAP
Oracle Cloud Infrastructure
Okta
RADIUS
TLS Certificates
Tokens
Username and Password
App ID (Deprecated)
MFA (Legacy/Unsupported)

Our aim is to connect and application (Jenkins) to Vault. For this purpose the best way is to use AppRole.

AppRole Authentication

This authentication method allows machines or “apps” to authenticate with Vault-defined roles. An “AppRole” represents a set of Vault policies and login constraints that must be met to receive a token with those policies. The scope can be as narrow or broad as desired. An AppRole can be created for a particular machine, or even a particular user on that machine, or a service spread across machines.

Before logging in, we need to configure our login parameters.

Step 1: Enable AppRole Authentication

vault auth enable approle

Step 2: Create a role

vault write auth/approle/role/role_name \ 
secret_id_ttl=10m \ 
token_num_uses=10 \ 
token_ttl=20m \ 
token_max_ttl=30m \ 
secret_id_num_uses=40

Note: If you want your token and secret_it_ttl to live forever (a use case which come more often than not), set token_ttl and secret_id_ttl to 0.

Other useful parameters can be found here: https://www.vaultproject.io/api/auth/approle

Now that we have created our AppRole, we will need the role-id and secret-id. These two values will be used in combination to login successfully.

Step 3: Get role-id

vault read auth/approle/role/my-role/role_name
role_id     db02de05-fa39-4855-059b-67221c5c2f63

Step 4: Get secret-id

vault write -f auth/approle/role/my-role/secret-id
secret_id               6a174c20-f6de-a53c-74d2-6018fcceff64
secret_id_accessor      c454f7e5-996e-7230-6074-6ef26b7bcf86
secret_id_ttl           10m

Note: Take a note of secret-id as this is the only time we can see it.

Creating Jenkins Credentials

Now we have created and taken note of both role-id and secret-id. The next step is to add this in Jenkins and generally we keep only this credential in Jenkins and all others in Vault.

We will select Vault App Role Credential type in Jenkins and fill out the information. Role-id, secret-id, and path are the only requirements.

Configuring Global Setting for Vault

In the Configure System page on our Jenkins server, go to the Vault Plugin section. There we need to give our Vault URL and select the credential that we have just created in the precious step.

Now we are all set to run our pipeline and access secrets present in Vault.

Simple Pipeline to Access Vault Secrets

def secrets = [
  [path: 'secret/jenkins/github', engineVersion: 2, secretValues: [
    [envVar: 'PRIVATE_TOKEN', vaultKey: 'private-token'],
    [envVar: 'PUBLIC_TOKEN', vaultKey: 'public-token'],
    [envVar: 'API_KEY', vaultKey: 'api-key']]],
]
def configuration = [vaultUrl: 'http://my-vault.com:8200',  vaultCredentialId: 'vault-approle', engineVersion: 2]
                      
pipeline {
    agent any
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
        disableConcurrentBuilds()
    }
    stages{   
      stage('Vault') {
        steps {
          withVault([configuration: configuration, vaultSecrets: secrets]) {
            sh "echo ${env.PRIVATE_TOKEN}"
            sh "echo ${env.PUBLIC_TOKEN}"
            sh "echo ${env.API_KEY}"
          }
        }  
      }
    }
}

With everything configured properly, we should be seeing an output similar to this one.

[Pipeline] {
[Pipeline] stage
[Pipeline] { (Vault)
[Pipeline] wrap
[Pipeline] {
[Pipeline] sh
+ echo ****
****
[Pipeline] sh
+ echo ****
****
[Pipeline] sh
+ echo ****
****
[Pipeline] }
[Pipeline] // wrap
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

If there is an error at any step, we will see a message for access denied, and the environment variable will print null.

[Pipeline] {
[Pipeline] stage
[Pipeline] { (Vault)
[Pipeline] wrap
Access denied to Vault Secrets at 'secret/jenkins/github'
[Pipeline] {
[Pipeline] sh
+ echo null
null
[Pipeline] sh
+ echo null
null
[Pipeline] sh
+ echo null
null
[Pipeline] }
[Pipeline] // wrap
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Conclusion

In this blog we ran over why there is a need of a more secure credential storage for Jenkins and how Vault is an answer to that. As visible, integrating Vault with Jenkins is not too complex and the access strategies in it provide a lot of options to us as its users. I hope this post helps you to integrate your Jenkins instances with the secure Vault.