CI / CD / DevOps

Jenkins – A quick introduction

By | February 21st, 2018|CI / CD / DevOps|

Most testers must have heard about Jenkins, and an increasing number of people are using or looking to use it in their automation and continuous integration efforts. The popularity of Jenkins is unquestionable and still is the most widely used CI tool, due to being open source and an eco-system building around it providing support for integrations with lots of platforms and libraries.

In this post, I thought to create a quick read on what is Jenkins and its main feature. I’m not using any technical terms from the tool and keeping it simple to make it easier for someone new to Jenkins.

Why Jenkins?

But before going into it, why we needed Jenkins? The process of deploying software (on multiple environments) can be tedious. If it can be done automatically will reduce a lot of time and effort which can be used elsewhere. Therefore, the main problem solved is building and deploying software on multiple environments throughout the different SDLC stages and run tests where needed.

History

I naively thought CI was a more recent phenomenon. Jenkins however, was first developed in 2004 by sun microsystems under the name ‘Hudson’. Over the years and legal battles, the name changed to Jenkins along with many changes in ownership till the community became an independent project. Surely this has been one a vibrant and flourishing community resulting in the widely accepted CI tool for many.

What is Jenkins

It’s a CI tool. This sentence provides no value into explaining what the tool does. So, let’s try again. It’s a tool which can build and deploy software and run post deployment tasks. A picture speaks a thousand words, so let me ‘draw’ a thousand words to elaborate that:

This process can start from different type of triggers provided. The AUT’s code is fetched, built and deployed, tests are run and moved on to the next stage to repeat the cycle or wait for a decision maker to confirm the next transition. With time there is a lot more you can do with Jenkins through the many plugins it has, however the main functionality is around these few main stages.

Jenkins does not bound the user to concrete stages (at least in Jenkins 2), these headings are therefore not representing exact phases you must use in Jenkins. Rather to provide an easy understanding about how it’s generally done.

Triggers

Back in the day someone would confirm from stakeholders once we are ready to move to the next stage of SDLC, then the release manager / team would deploy the next version from source control on the test environment. One this stage was completed; the next stage would require another correspondence and confirmation from a bunch of people followed up by hours long activities to patch the new environment.

Jenkins provides few basic triggers used to start the process of rolling our ‘Continuous Integration’ snowball down the hill. Triggers can be defined against a new code changes / new version in our source control. As soon as a new version is found by Jenkins on the SCM (Source Control Management), it triggers the process. Other common triggers are time / date specific or other configurations the tool can poll for. Mostly it’s based on new versions in the SCM.

Build Process

The first thing once the process initiates is to build the code. For larger applications this can mean a lot of small steps. To keep it simple, you will check out the latest queries for building the database and latest code repo for your code project and build (compile and package) the code. This would vary from team to team on how the processes are designed so there no single best practice IMHO (there certainly are bad practices which are beyond the scope of this article).

In many cases this would not be a complex activity, a few configurations and you’d be good to go. Specially if you are using build automation / package manager tools like maven or npm. Checking out the code would be one config line, and another to compile and build that code and your done for this stage.

Deploy

This is going to vary a lot, perhaps might not even be considered as a separate stage and you’d just add another step after building the code. Or if you are creating a new environment on the runtime for your tests (perhaps using docker) this would be a separate stage. The purpose here is to just deploy your AUT in the environment you want it to be tested in.

An important point to mention, if you have automation scripts to run (which should be the case) that would also be a part of the checking out and ‘deploying’. These tests might not be installed on a server, but certainly before you run those tests they would be fetched from a source control. Therefore, if you are building your environment, any pre-requisite tool deployments and test code / tools needed would be a part of the process here as well.

Test and Package

Once you have your environment ready and tests in hand, it’s time to run them and find out if we have any problems. Jenkins has its own reporting process which can be customized using plugins. This allows to see the results of all the tests executed, and can be consumed in different forms including sent as notifications in emails. You can also archive your code base if needed for revisioning purposes at the end of this stage.

Agents (Slave nodes)

I too dislike the slave notation but it makes things easier to understand. Jenkins can interact with a lot of tools sitting on different machines / environments. Typically, a decent Jenkins setup might be interacting with quite a few environments at the same time giving instructions and getting feedback. This is how you can have parallel test runs using Jenkins.

Environments can be created using different methods ranging from a physical machine to VMs to containerized environments (e.g. docker) or use SaaS tools like Sauce Labs / Browser Stack or Selenium grid. Out of these pros and cons for each and method to decide what to use would be a separate topic to discuss.

This example might be non-practical but illustrates the different ways these environments can be created / tests executed.

Plugins and integrations

The best thing I like about Jenkins is the number of plugins available to support many tools and libraries. All commonly used platforms provide plugins for Jenkins including test libraries and tools. Other plugins can be used to further configure Jenkins and allow to perform other complex tasks including customization and exporting of test results.

I must mention here, it’s a good idea to make sure whatever tool you are using around CI stages, it must support Jenkins. Most tools do, but if you get stuck with one that does not, it is not far-fetched if you’d have to look for a replacement which does.

Jenkins 2.0

There has been a lot of advancement in Jenkins since its earlier versions, especially when CI is becoming a more common practice across the industry. One major difference is the ‘pipeline’ method of creating ‘jobs’ in Jenkins which was not available in the earlier version. The traditional ‘freestyle jobs’ of Jenkins has specified stages for a job in a specific sequence, however the pipeline type job available in Jenkins 2.0 is script based written in Groovy. This allows for a lot more flexibility and customization to the process.

Other considerable changes in version 2.0 include a greater set of default plugins installed (making the first time use of the tool easier) and many supported plugins including the blue ocean plugin (creates a different GUI for the tool). It’s recommended to use version 2.0 for someone starting off now to take advantage of all the features.

The Magic of Continuous Testing

By | September 11th, 2017|CI / CD / DevOps|

 

Have you ever reached release time for your product only to find out major bugs coming out of testing? Does this sound like what you face on a routine basis? Trying to drill into finding the root cause for this many times ended up inconclusive, there was no one practice or person to be held responsible at the end. This is where continuous testing comes in to save the day.

This is a huge topic and one can easily write a book on this. This post along with this video attempts to highlight probably the very basics I feel are most important.

The bug bottle neck

I call the last minute barrage of bugs ‘the bug bottleneck’. To some it seems the only obstacle coming in between releasing a new product version are these testers coming up with last minute bugs. Why were these issues not found earlier and had to be reported a day before release.

The bug bottleneck develops not over a few days, it’s a culmination of issues which could have been prevented from the very first sprint cycle. More complex and huge the product is, the more difficult it becomes to test it adequately causing more undetected bugs. Here is a typical sprint cycle with manual testing at the end:

Dev team implements the new feature for the sprint with some fixes. This introduces new bugs some being directly related to implementation of the new feature and some ripple effects due to the new feature / fixes. The testers should now ideally test all the new features with regression around each one of them in detail. However usually there is not enough time for them to test the feature AND do regression around them during a sprint, especially if this is a complex application. Bugs do slip from this stage and keep piling up. A few sprints down the line nearing release dates, there is an army of undetected bugs which start getting caught as soon as the regression cycle hits which is then too late.

What is continuous testing?

In our example, testers were not able to cover enough ground from the very first sprint cycle causing the melt down at the end. Adding more testers is not going to be a sustainable solution either, especially for large applications. The solution is automating as many tests possible and running them as part of the sprint process.

Continuous testing is defined in many ways. I understand it to be a process with testing ingrained into every step of the sprint including development, and keep evolving the tests based on feedback of what works and what does not. Testing is therefore a ‘continuous’ process and not just an activity done when code is pushed to QA.

It would start from running automation during development and testing. As soon as a developer checks in his / her code, tests for at least the changed areas of the application are executed giving the developer an immediate feedback if anything fundamental is not working. Similarly when testing kicks in, automation should run a regression on the application while manual testers can focus purely on understanding and testing the new features.

Why Continuous Testing

This is the next step in the software development processes evolution. The last big change was Agile which replaced waterfall. There were some forms of agile practices already in place like iterative models, however agile methodology was classified as a different species.

We have reached the next step of evolution here, and it’s for the same purpose agile came into being, reduce time to market and increase quality. Like the iterative model, the fundamentals to this next step of evolution were hidden in agile but not prominent and were being neglected, necessitating the next step. This next step is known by many names including continuous integration, continuous delivery, DevOps, continuous testing. All of them have different meaning and implementation, but work towards the same purpose, reduce time to market and increase product quality. To truly have a DevOps model, one would need continuous delivery, needing continuous testing needing continuous integration at the end creating one system interconnected for the same purpose.

Back to our example, with continuous testing using automation in place, our sprint would look something like this:

Issues being captured during development coupled with a higher ratio during testing cycle lead to a smaller number of bugs getting slipped passed our net. Come regression week (s), we will find far less issues during that time and hopefully no major ones.

 Requirements for Continuous Testing

There can be a very long list here, the few I found important for one of the products I worked on are discussed briefly.

 Increase test coverage

Unless we have a decent coverage of the application, the purpose of capturing bugs from code changes will not be accomplished. Today’s applications are far more complex which cannot be automated just from the UI. Careful planning and ingenuity is needed to have a decent coverage.

 Make results meaningful

Tests a developer would like to see on adding a feature / updating code should be there. Yes, unit tests would be reflecting that, but also API and some UI level would be helpful too. Similarly for the tester, an automation batch run should be augmenting a tester’s wish list of areas to test. Therefore feedback and contribution from both these groups is vital.

Secondly the results should come in time. If we send the developer a report for 2 major bugs in a feature he / she implemented a week later, that’s of no use. The result should be there in hours while the implementation details are fresh in mind. Also for the tester, they should base their tests looking also at the automation report. For instance, “Areas A, B and C passed, I can test a slight variation there and that would be enough. D and E failed, probably there are more issues around this. “. Unless the report is delivered BEFORE the manual tester starts planning his test, it’s going to be late.

 Test before check in

Before a piece of code is checked in, it should be tested through the automated scripts and added to the branch only when passing. This would be easier than done since it would require a lot of pre-requisites. Continuous integration portions implemented, automated tests running efficiently and processes institutionalizing the branching process. however once done, the benefit would be far beyond the cost.

 Test quickly

That was the whole idea right? To test quickly with quality. If a tester is able to utilize the automation results as mentioned, it can help reduce the testing time per issue, allowing to test more areas and test quickly. This freed up time should be used to the advantage of shrinking the test cycle and increase time to market.

Word of caution here, by no measure this means to cut regression in half (or the team for that matter), rather increase the number of releases per year and have a quick turn around.

 Collaboration

To get the desired results, all departments need to coordinate with one another sharing what worked and what can be improved. This would involve not only development and QA, but support, sales and other departments giving information on how the updates went with the customers and what next to be prioritized, what areas need more automated tests around, what should be included in manual tests and so on.

Continuous testing is going to be an ongoing journey, not a destination. For it to be continuous, it necessitates evolution and continuous improvement. Being an emerging concept and mostly early adopters in the arena having it fully implemented, I feel a lot of evolution is in store around implementing this idea and how to do it well. One thing is certain, teams, products and companies to adopt this early on will be the most successful.