Using Ansible in Jenkins – Ansible Plugin

Jenkins + Ansible Options

I have been doing a lot of Jenkins, Ansible, Terraform, and similar automations lately and I have seen multiple ways of running Ansible in Jenkins.  These include:

  1. Install Ansible on your Jenkins node and call it with “sh” in a pipeline.
  2. Install Ansible on your Jenkins node and call it with the ansible plugin for Jenkins (not a default plugin).
  3. Run a docker image with Jenkins and configure ansible there.

I think the 3rd option is the most powerful as you can separately configure and version Ansible for your various Jenkins jobs.  But the second option is pretty sleek as well and is what we’ll talk about here.

Jenkins Ansible Plugin

If you go to the Jenkins plugins management page, you can install the ansible plugin pretty easily.  There are some good documents on it right here too.  After installing that plugin, I also suggest you install the AnsiColor plugin.

With both of those in place, you can get ansible running your playbooks with its private key credentials and it will print beautiful colored output to your logs.

Here is an example of how to call it from a pipeline.  Note that this lets you specify your target hosts in a CSV, assuming your playbook uses the “target_hosts” group.  It also disables host key checking (which took me a while to work out as the docs are wrong).  For some reason my ssh wouldn’t work without that setting with this plugin even though it is not ideal.

pipeline {
    agent any

    parameters {
        string(name: 'GIT_BRANCH', defaultValue: 'master', description: 'git branch to work from.')
        string(name: 'TARGET_HOSTS_CSV', defaultValue: 'none', description: 'target deployment hosts.')
    }

    stages {
        stage('Deploy application.') {
            steps {
                sh 'echo [target_hosts] > /tmp/inventory.ini'
                sh 'echo "${TARGET_HOSTS_CSV}" | tr "," "\n" >> /tmp/inventory.ini'

                ansiColor('xterm') {
                    ansiblePlaybook(
                        playbook: './ansible/playbook.yml',
                        inventory: '/tmp/inventory.ini',
                        credentialsId: 'your-jenkins-pk-credential',
                        disableHostKeyChecking: true,
                        colorized: true)
                }
            }

        }
    }
}

Ansible – Install AWS CLI and Log in To Amazon ECR Docker Registry via Ansible

There is probably a much cleaner way of doing this using off-the-shelf automations.  But I was just following along with the AWS installation instructions and got this working.

- name: Download AWS CLI bundle.
shell: "cd /tmp && rm -rf /tmp/awscli* && curl 'https://s3.amazonaws.com/aws-cli/awscli-bundle.zip' -o 'awscli-bundle.zip'"

- name: Update repositories cache and install "unzip" package
apt:
name: unzip
update_cache: yes

- name: Unzip AWS CLI bundle.
shell: "cd /tmp && unzip awscli-bundle.zip"

- name: Run AWS CLI installer.
shell: "/tmp/awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws"

- name: Log into aws ecr docker registry
when: jupyterhub__notebook_registry != ''
shell: "$(/usr/local/bin/aws ecr get-login --no-include-email --region us-east-1)"

In order to do the actual login, you need to ensure your EC2 instance has an IAM role assigned to it that has reader privileges. Then you should be all good!

Resolving the ContextualVersionConflict: (cryptography 1.7.1 (/usr/lib/python2.7/dist-packages) error.

I was trying to run ansible over a debian slim docker image and I got this error, which was rather cryptic.  There wasn’t any particularly good information on google for fixing it either.

ContextualVersionConflict: (cryptography 1.7.1 (/usr/lib/python2.7/dist-packages), Requirement.parse(‘cryptography>=2.5’), set([‘paramiko’]))
ERROR! Unexpected Exception, this is probably a bug: (cryptography 1.7.1 (/usr/lib/python2.7/dist-packages), Requirement.parse(‘cryptography>=2.5’), set([‘paramiko’]))
the full traceback was:

Traceback (most recent call last):
File “/usr/local/bin/ansible-playbook”, line 97, in <module>
mycli = getattr(__import__(“ansible.cli.%s” % sub, fromlist=[myclass])…

In my case, running this before my ansible install resolved the issue.  It just forces an upgrade to a newer version which ansible is okay with.  Of course, if you were dependent on the lesser version this may not be an option.

pip install –force-reinstall cryptography==2.7

 

Ansible Vault Secrets – Encrypt Single Strings in Config Files (Not Whole File)

Why Use Ansible Vault?

Ansible vault is a very useful tool in the world of devops.  It provides you an easy way to check your entire configuration into version control (like git) without actually revealing your passwords.

This means that when you apply config changes, it gets recorded and you can roll-back or upgrade just like you do with normal code.  Nothing is lost, and nothing is insecure.

Encrypting a Single Property

Ansible can be used in countless ways; it is a very flexible tool.  Take variables for example; you can put them in your inventory file, in separate variable files, at the top of your playbook, in the CLI command, and I’m guessing even more places.

In this case, lets say you have a nice existing variables file that just happens to not be encrypted.  You don’t want to split up your file and you don’t want to encrypt the whole file either because then it is hard for others to review (e.g. why encrypt the DNS name for a service you’re hitting?).

Example vars-dev.yml:

mysql_password: password123
dns_name: some-service@company.com

So, let’s encrypt just the password in that file and leave everything else as plain text. We’ll use this command to encrypt the password (password123).

$> ansible-vault encrypt_string --vault-id vault_password 'password123' --name 'mysql_password'
mysql_password: !vault |
$ANSIBLE_VAULT;1.1;AES256
31336265626432653930373332313963666565336333303731653133613732616234383634613134
6463393739616232613837363065376336346265646432360a383464326231333035633762663738
65396562386166613537346165646539303334383039396235636331343830633761303239643434
6335343035646665640a666338346130636161663561376662666566316565646133653162663065
3534
Encryption successful

Note that “vault_password” in this line is actually a text file in the current directory that just has the password to my vault in it. Keeping your vault password in a file lets you use it in automation like Jenkins without it showing up in log entries/prompts.

$> cat vault_password
some_good_password

The entire part from ‘mysql_password’ onward in the command output (prior to Encryption Successful) is the line you should now use for the password in your variables file. So, the modified variables file looks like this:

Updated Example vars-dev.yml:

mysql_password: !vault |
$ANSIBLE_VAULT;1.1;AES256
31336265626432653930373332313963666565336333303731653133613732616234383634613134
6463393739616232613837363065376336346265646432360a383464326231333035633762663738
65396562386166613537346165646539303334383039396235636331343830633761303239643434
6335343035646665640a666338346130636161663561376662666566316565646133653162663065
3534

dns_name: some-service@company.com

Testing It Out

At this point, let’s test that we can use this partially encrypted variable file for something real.

Let’s start by creating some new files, all of which we’ll be lazy with and just keep in our current directory:

We’ll create a config.properties template file that uses our variables.

mysql.password={{mysql_password}}
dns.name={{dns_name}}

Well create a playbook.yml file that is our actual playbook to run:

---

 - name: Ansible Vault Example
   hosts: my_hosts
   tasks:
     - name: Copy config file to servers with decrypted password.
       template: src="config.properties" dest="/tmp/config.properties"

And we’ll create inventory.yml to tell ourselves which hosts to target. We’ll just target localhost, but whatever. Make sure localhost can ssh to itself for this :).

[my_hosts]
localhost

Now we can run our test! Use this command to run our playbook with our inventory file and our variables file with our vault password. Based on the playbook, it should copy config.properties from our current directory to /tmp/config.properties with the variables replaced and decrypted where relevant.

ansible-playbook -i inventory.yml --vault-password-file vault_password --extra-vars @vars-dev.yml playbook.yml

Now we can cat /tmp/config.properties and see that it worked!

$> cat /tmp/config.properties
mysql.password=password123
dns.name=some-service@company.com

I hope this helps you in the future! I’ll be doing more blogs on Ansible, Vault, etc in the near future. So, come back for more!

Ansible – Refer to Host in Group by Index

Occasionally it is very useful to refer to a host in a group by an index.  For example, if you are setting up Apache or HAProxy, you may need to push a configuration file out to each host that can redirect to all other hosts.

It is actually quite easy to refer to the hosts in a group by index, but its not necessarily easy to google it unfortunately.  Here is the syntax for the first 3 hosts in a group:

{{groups['coordinators'][0]}}
{{groups['coordinators'][1]}}
{{groups['coordinators'][2]}}