Welcome to this Cerberus Testing tutorial for enabling automated tests execution in your Bitbucket pipelines.

We will first clarify the value of using such quality gates in your software development lifecycle. Then, we will provide you with step-by-step instructions for deploying them.

At the end of this article, you will be able to configure and scale your quality gates in Bitbucket according to your requirements.

This article is using the Cerberus Testing demo instance, so you can also reproduce it on your end easily. You can use it on your instance if you desire to.

If you are new to Cerberus Testing, we recommend that you start by having a look at our GitHub page, website and overview video. This tutorial is also available as a video.

Prerequisites

This tutorial requires the following elements:

• Access to a Cerberus Testing instance at least in version 4, self-hosted or SaaS.
• An available Cerberus Testing campaign, more information here.
• Authorization to call the Cerberus Testing APIs, network and gateway grants.

If you have any questions on those prerequisites, have a look at our documentation, videos or contact your administrator. Feel free to interact with the community on our Slack channel or submit a GitHub issue.

Companies have to accelerate for survival

Successful quality assurance patterns emerged from the industry in the last decades. We all heard about CI/CD, Pipelines, Quality Gates. But why?

In fact, cycle of changes have to accelerate within organizations . While becoming digital,  software delivery must be faster and safer.

Consequently, automation has been largely adopted to cope with the speed requirement. For quality assurance, automating tests early in the process became a best practice.

The Value of Quality Gates in CI/CD Pipelines

Performing tests while software changes are delivered is a challenging task. Different roles, positions and processes must work effectively together.

Indeed, this is why CI/CD pipelines evolved to incorporate testing as a mandatory step in the delivery process. When done right, it can provide significant value.

First, it forces us to leverage automation for both deployment and testing, removing costly and error-prone manual steps.

Secondly, automation leads us to explicit the knowledge we can find in the automated tests, thresholds for GO/NOGO for instance.

How Cerberus Testing integrates Automated Tests in CI/CD

Deploying automated tests in a CI/CD pipeline also requires the integration of different systems: The code repository, the pipeline, and the automated tests.

In our case, Bitbucket is providing the code repository and CI/CD pipelines, while Cerberus Testing is enabling the automated tests (Web, Mobile App, APIs among others use-cases).

The integration between the two solutions is managed through APIs leveraging the Cerberus Testing Runner plugin for ease of deployment.

Various integrations are possible depending on the requirements and context. A simple trigger can be configured, or more advanced scenarios can be chosen.

As we use the Cerberus Testing Runner plugin, the integration is “Bi-directional”. 

Concretely, the automated test campaign is launched and the pipeline will wait for its completion until a defined timeout.

That way, you save precious time by only configuring your pipeline, freeing you from coding specific scripts.

Run Tests in Bitbucket Pipelines with Cerberus Testing 

In this section, we detail the necessary steps for configuring the Cerberus Testing automated tests within your Bitbucket pipeline.

1.1 Add a Cerberus Testing Pipeline File

The first step is to add a specific step in our pipeline to launch the automated tests.

Navigate to the “Deployments” menu of your Bitbucket project. If you already have a pipeline file, you can choose to edit that one. 

You can search for a Pipeline template on the right. Look for “Cerberus Testing Runner”.

The documentation will appear once you select the correct pipeline template. Several examples are available for inspiration and simpler deployment.

For our example, we will define a simple step to launch the available campaign in the demo instance.

We just need to specify those two parameters in the corresponding variables.

1.2 Commit, Test and Confirm the Integration

We are now ready to use this step in our pipeline.

To confirm our changes, we will commit the file to the project. Bitbucket will automatically trigger a pipeline following the commit.

Navigate to the “Pipeline” menu and select the last pipeline that should be running. The new pipeline step will be executing, displaying the logs.

As you can see, the variables and campaign link are available for logging and observability purposes. 

If you access the link, you will see your campaign result directly in Cerberus Testing.

The automated test execution end s when either the campaign is complete or the timeout reach its limit.

1.3 Externalize Cerberus Testing variables for reuse

A good practice is to externalize your pipeline configuration using the Bitbucket variables.

That way, you will decouple your components and allow better reuse, scalability and maintainability of your pipelines.

You can do this by going to the “Repository settings” and defining “Repository variables” with a corresponding name and value.

Once defined, you can use the variable in your pipeline file with a “$” sign.

For example, a repository variable “CERBERUS_CAMPAIGN” will be referenced as “$CERBERUS_CAMPAIGN” in your pipeline.

Scaling your CI/CD with Quality Gates & Automated Tests

The previous section enabled you to define a single quality gate in your pipeline. But typical software projects are more complex.

In fact, a traditional setup will have various test environments. So, the first step is to correctly define your environments in Bitbucket.

Then, you can define a specific pipeline step for the corresponding test environments.

A good practice is to have automated tests for each environment. This will force your tests to be reliable, fast and useful at detecting defects early.

Each quality gate can be configured using the Cerberus Testing Runner pipeline template. 

Expand your CI CD automated test

We hope this guide will help you to improve your digital product quality with CI/CD quality gates that you can scale.

Other typical integrations, such as Jenkins and Azure DevOps Pipeline are available, feel free to have a look at the various content available. 

If you want to implement your own case fast, you can access the available Plans. We will be happy to support you in your test automation journey.