Artifactory + Maven Deploy +/- Jenkins – 502 Failure

This is just a quick post…

If you are running a maven (mvn) deploy to artifactory, be it locally or through Jenkins/etc, and you get an error like this:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy (default-deploy) on project sample-ws: Failed to deploy artifacts: Could not transfer artifact com.company.cs.sample:sample-ws:jar:3.2.2 from/to central (https://adlm.company.com/artifactory/sample-maven): Failed to transfer file https://adlm.company.com/artifactory/sample-maven/com/company/cs/sample/sample-ws/3.2.2/sample-ws-3.2.2.jar with status code 502 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

You need to edit your ~/.settings.xml and add in your artifactory-user/artifactory-token (preferably for you on your own desktop and for a service account on your server).  You can generate your token in your user settings in the Aritfactory API.

<settings>
  <servers>
    <server>
      <id>central</id>
      <username>your-user</username>
      <password>your-token</password>
    </server>
  <server>
    <id>snapshots</id>
      <username>your-user</username>
      <password>your-token</password>
    </server>
  </servers>
</settings>

Jenkins Pipeline – Maven + Artifactory Example With Secure Credentials

There are always a million ways to do things in Jenkins, but often using the appropriate plugins for common tools pays off a lot.  Here is an example of using multiple maven commands to execute a build, the result of which (a zip file) is then uploaded to Artifactory.

Sample Pipeline

Honestly, there will almost definitely be an artifactory plugin as well; but in this case, I used these 2 plugins:

  • Pipeline Maven Integration Plugin – Allows you to target various maven installations set up in your Jenkins “Global Tool Configuration”.
  • Credentials Binding Plugin – Allows you to pull user names and passwords from credentials into environment variables to use in commands easily.

This was enough to let me run the maven commands I needed and to let me then call the Artifactory REST API safely to upload my one artifact, which is a zip in this case.

I hope this example helps you! 🙂

pipeline {
    agent any
    parameters {
        string(name: 'SOURCE_BRANCH', defaultValue: 'master', description: '...')
        string(name: 'RELEASE_TAG', defaultValue: '', description: '...')
    }
    stages {
        stage('Build and deploy presto zip to artifactory.') {
            steps {

                sh 'echo "Building to version = $RELEASE_TAG"'

                withMaven(maven: 'maven-3') {
                  sh "mvn versions:set -DnewVersion=$RELEASE_TAG"
                  sh "mvn clean install -DskipTests"
                }

                withCredentials([usernamePassword(credentialsId: 'artifactory-token', usernameVariable: 'AUSR',
                    passwordVariable: 'APWD')]) {
                  sh '''curl -X PUT -u $AUSR:$APWD -T presto-server/target/presto-server-$RELEASE_TAG.tar.gz "https://company.com/artifactory/repo/io/presto/$RELEASE_TAG/presto-server-$RELEASE_TAG.tar.gz" '''
                }
                
                sh 'echo "Done building and tagging to name = $RELEASE_TAG"'
            }
        }
    }
}

Using Ansible in Jenkins – Ansible Plugin

Jenkins + Ansible Options

I have been doing a lot of Jenkins, Ansible, Terraform, and similar automations lately and I have seen multiple ways of running Ansible in Jenkins.  These include:

  1. Install Ansible on your Jenkins node and call it with “sh” in a pipeline.
  2. Install Ansible on your Jenkins node and call it with the ansible plugin for Jenkins (not a default plugin).
  3. Run a docker image with Jenkins and configure ansible there.

I think the 3rd option is the most powerful as you can separately configure and version Ansible for your various Jenkins jobs.  But the second option is pretty sleek as well and is what we’ll talk about here.

Jenkins Ansible Plugin

If you go to the Jenkins plugins management page, you can install the ansible plugin pretty easily.  There are some good documents on it right here too.  After installing that plugin, I also suggest you install the AnsiColor plugin.

With both of those in place, you can get ansible running your playbooks with its private key credentials and it will print beautiful colored output to your logs.

Here is an example of how to call it from a pipeline.  Note that this lets you specify your target hosts in a CSV, assuming your playbook uses the “target_hosts” group.  It also disables host key checking (which took me a while to work out as the docs are wrong).  For some reason my ssh wouldn’t work without that setting with this plugin even though it is not ideal.

pipeline {
    agent any

    parameters {
        string(name: 'GIT_BRANCH', defaultValue: 'master', description: 'git branch to work from.')
        string(name: 'TARGET_HOSTS_CSV', defaultValue: 'none', description: 'target deployment hosts.')
    }

    stages {
        stage('Deploy application.') {
            steps {
                sh 'echo [target_hosts] > /tmp/inventory.ini'
                sh 'echo "${TARGET_HOSTS_CSV}" | tr "," "\n" >> /tmp/inventory.ini'

                ansiColor('xterm') {
                    ansiblePlaybook(
                        playbook: './ansible/playbook.yml',
                        inventory: '/tmp/inventory.ini',
                        credentialsId: 'your-jenkins-pk-credential',
                        disableHostKeyChecking: true,
                        colorized: true)
                }
            }

        }
    }
}

Jenkins – su to Jenkins User

Today I was setting up a new Jenkins server to run docker image builds and pushes to Amazon ECR.

Jenkins installed fine, as did the AWS CLI, and docker.  Unfortunately, when I went to use docker against AWS from Jenkins, I had some integration issues at first (which is less than surprising).

Anyway! this required me to become the “jenkins” user which Jenkins runs as by default when installed with its normal installers.  Unfortunately, when you try to “su – jenkins”, you will find that not much happens.

I found in this stack-overflow post that this is because the jenkins user is a service account not made for interactive terminals.  Here is the quote:

Jenkins is a service account, it doesn’t have a shell by design. It is generally accepted that service accounts shouldn’t be able to log in interactively

If for some reason you want to login as jenkins, you can do so with: sudo su -s /bin/bash jenkins

So, just do the following and you’ll be fine.

sudo su -s /bin/bash jenkins

Installing Jenkins on Centos 7.x for Docker Image Builds

First, just start by getting a root shell so we can drop the sudo command from everything (e.g. type sudo bash).  This isn’t best practice, but just make sure you exit out of it when you are done :).

Centos 7.x Complete Installation Steps

Note that this installs wget, OpenJDK Java 1.8, Jenkins, enables Jenkins to be on with system start, and then it installs docker, and finally, it adds the Jenkins user to the docker user group so that it can run commands from it effectively (using elevated privileges).  Lastly, it restarts Jenkins so that the user has the new docker group permissions in the running process.

yum update -y
yum install wget -y
yum install java-1.8.0-openjdk-devel -y
wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
rpm –import https://jenkins-ci.org/redhat/jenkins-ci.org.key
yum install jenkins -y
service jenkins start
chkconfig jenkins on
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager –add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install -y docker-ce docker-ce-cli containerd.io
systemctl start docker
usermod -a -G docker jenkins
service jenkins restart
 

Validation

Assuming this all worked, you should see Jenkins running on your localhost:8080 port for that server, and you can follow its on-screen instructions.

For docker, you can run the hello world container to see if it is properly set up (docker run hello-world).

Resources

I got this information from the Jenkins wiki, the docker site itself, and one stack overflow entry as shown below:

Jenkins Pipeline Maven /Build + Deploy to Nexus/Artifactory in Docker

Overview

If you want to use a Jenkins pipeline to build your maven project in a clean docker container, you can do so as shown below.

You can do any maven commands you like.  In this particular case I am:

  • Setting the version in the POM to the one provided by the person running the Jenkins job.
  • Building and deploying the maven project to Artifactory.

This is the “build one” of “build once deploy everywhere”.

To test this though, you can just swap my commands out for maven –version or something simple like that.

Configuration

Create  a new pipeline job and:

  1. Add a string parameter called VERSION_TO_TAG.
  2. Set pipeline definition to “Pipeline script from SCM”.
  3. Give your repo checkout URL – e.g. ssh://git@url.mycompany.com:7999/PROJECT/repo-name.git.
  4. Specify your git credentials to use.
  5. Specify your Jenkinsfile path (It is probably literally just “Jenkinsfile” which is the default if you have the Jenkinsfile in the root of your repo).

Make sure your project has a “Jenkinsfile” at its root with a definition like this:

pipeline {
    agent {
        docker { image 'maven:3-alpine' }
    }

    stages {
        stage('Set version, build, and deploy to artifactory.') {
            steps {
                sh 'mvn versions:set -DnewVersion=$VERSION_TO_TAG'
                sh 'mvn deploy'
            }
        }
    }
}

Now, when you build your project, you should see it docker-pull a maven:3-alpine image, start the container, and run our maven commands and upload artifacts to the Artifactory repository you have set up in your maven POM (in your distributionManagement section in case you haven’t done that yet).