( MLOps + DevOPs Task 2 )
Problem Statement:-
1. Create container image that’s has Jenkins installed using Dockerfile
2. When we launch this image, it should automatically start Jenkins service in the container.
3. Create a job chain of job1, job2, job3, and job4 using the build pipeline plugin in Jenkins
4. Job1: Pull the Github repo automatically when some developers push the repo to Github.
5. Job2: By looking at the code or program file, Jenkins should automatically start the respective language interpreter install image container to deploy code ( eg. If the code is of PHP, then Jenkins should start the container that has PHP already installed ).
6. Job3: Test your app if it is working or not.
7. Job4: if the app is not working, then send an email to the developer with error messages.
8. Create One extra job job5 for monitor: If container where the app is running. fails due to any reason then this job should automatically start the container again.
Step 1: Create a Jenkins Container Image
Below is the Dockerfile I am using to create a Jenkins Container
Let's go through Dockerfile line by line
Line 1: I am telling it to take Base OS from centos for creating thecontainer
Line 2: RUN is the Keyword used to run the command in the container. Here I am installing wget because will need it to get some files(Jenkins repo) over the internet
Line 3: Downloading Jenkins Repository and adding it to repository lists
Line 4: Adding Jenkins repository key
Line 5: Installing Jenkins using yum
Line 6: Installing open-jdk(For running Jenkins we need open-jdk 11 or 8)
Line 7: Installing yum-utils for managing/configuring yum
Line 8: Adding docker repository
Line 9: Installing Docker
Line 10: Installing Python and Git(will be required for this task)
Line 11: Exposing Port 8080 which is the default port for Jenkins
Line 12: Running Jenkins using java
To build this image type the following command:
$ docker build -t <tagYourImage> .
// docker build -t anon3214/jenkins:latest .
To verify your image has been created, type the following command and check with the tag which you used while creating is present or not :
$ docker image ls
To verify container has Jenkins installed successfully, we need to run it :
$ docker run -it -p 8081:8080 — name jenkins anon3214/jenkins:latest
After running the command, you can see that Jenkins started and is running
you can also confirm it by going to http://localhost:8081 because we exposed the Jenkins to port 8081 of the local machine.
- But if you don’t want to lose data after terminating then you can add/mount a persistent folder to the container at /root/.jenkins/ (this location may be different depending upon which base installation configuration and os you used)
- If you want to run the container and also give files from Jenkins, it might be not possible because we are running docker on a local machine and not in the container itself, so to solve this I added an extra folder /root/task2data to Jenkins container in which will store all important data which can be passed to other docker containers also
- Also if you want to run docker in Jenkins you will either need to set up distributed Jenkins setup
- Or you can just use your local machine docker by providing/exposing docker socket to Jenkins container
In the following, I have also used a folder and mounted it at /root/.jenkins/ and I am also using the local machine docker socket to run containers.
Command :
$ docker run -it -p 8081:8080 -v /root/docker_jenkins/:/root/.jenkins/
-v /var/run/docker.sock:/var/run/docker.sock
-v /root/task2data:/root/task2data
— name jenkins anon3214/jenkins:latest
Now when you will go to the Jenkins page at first it will ask for the initial Admin Password. You can find it while running the docker container, I have highlighted it in the above images. Or you can go into the docker container and find it in the file located at “/root/.jenkins/secrets/initialAdminPassword”
You will need it to enter this password when first opening Jenkins webpage.
Now after continuing you will be taken a page to install plugins. You can download suggestions.
After installing plugins you will be taken to create the first admin user page. Just fill in the necessary information and continue.
After this you will be taken to instance configuration, just continue and then the Jenkins will restart if you installed plugins that need a restart. It will take 3- 6 minutes to restart Jenkins.
After this, you will be taken to the login page of Jenkins and you will need to provide the username and password which you set up during the creation of the admin user in previous steps.
Now our Jenkins container has been configured for use.
Now let's create a Job chain and pipeline.
Step 2: Job1 →=Pull the GitHub repository automatically when some developers push code to Github.
Once you have successfully started and Configured Jenkins, you will see the Welcome page of Jenkins in which you have an option to create a new job/item, click it to create a job and then name your job (job1) and choose freestyle project.
The first job will be to → Copy Code from GitHub to /root/task2data/ as soon as the developer pushes the code to GitHub.
For this, we will need to set up GitHub WebHook which will need to provide Jenkins URL to GitHub. So we need to expose our Jenkins URL for this we will be using ngrok.
You will need to download ngrok and run the following command :
$ ./ngrok http 8081 (to expose port 8081)
But since we also need to expose port 81 for testing our webpages, we can configure it in yml file of ngrok.
I have already created a free account and added auth to ngrok configuration files and also added some more information to the configuration file(/root/.ngrok2/ngrok.yml) to expose ports 80 and 8081.
after this run ngrok using the following command :
$ ./ngrok start — all (to run from yaml configuration)
For Creating WebHook:
→Go to your GitHub repository Setting →Webhooks → Add Webhook
Now its time to configure our Job1
Output of Job 1
Step 3: Job2 → By looking at the code or program file, Jenkins should automatically start the respective language interpreter install image container to deploy code ( eg. If the code is of PHP, then Jenkins should start the container that has PHP already installed ).
This Job will run automatically after Job 1.
It will check the code and start the respective docker container
Step 4: Job3 →Test your app if it is working or not.
We will use the curl command to check whether our website is running or not.
If it gives status 200 then it is good.
But if the status is not 200 then the build will fail as we have set the exit status to 1.
With the build fail next job 4 will send a message/email to the developer.
But in Build Trigger there is no option for a build if fails, so for this, we need to install an extra plugin called Downstream-Ext.
After downloading restart Jenkins.
As you can see I have added Post-build trigger to run job4 if the job3 build fails and that is when our website won’t give a 200 response.
The output of job 3:-
Step 5: Job4 →if the app is not working, then send email to a developer with error messages.
This job will run if the job3 build fails and it will mail the developer.
There are two ways to send mail:-
1) Write your own script to send mail
2) Use Jenkins in-build Email Notification or Extended Email Notification which we get after installing the Downstream-Ext plugin
So I am using Jenkins Extended Email Notification
To configure Email notification you need to Go to Manage Jenkins → Configure System → Extended Email Notification and Enter the Username and Password of the SMTP you are going to use to send the email.
In my case, I am using Gmail, if you are also doing the same then you need to enable Less Secure Apps in your google account settings.
As you can see I have set up the post-build as Extended Email Notification.
Now our pipeline is complete. We need one more job to keep checking the container status
Step 6: Create One extra job job5 to monitor → If container where the app is running. fails due to any reason then this job should automatically start the container again.
It will check every 5 minutes whether container is running or not.
Hooray, we have completed our Jenkins pipeline.
To get a better visualizing of our pipeline we have a Build Pipeline Plugin.
You can download it and set it up by following steps :
Go to Manage Jenkins →Manage Plugins
→ Click on Available →Search for Build Pipeline Plugin
→ Download/install and restart Jenkins
To Build a pipeline → Click on “+”
Enter the name for pipeline and select Build Pipeline
Configure → Select Initial Job → Job1
And apply, You will get the following Build Pipeline
As you can see if the build is successful and after job3 test the webpage using curl if it gives 200 status then it won’t trigger job4.
But if gives a status code other than 200 then job4 will be triggered(i.e when job3 build fails).
Hence our work has been completed :).
