Welcome to this Cerberus Testing tutorial for enabling automated tests execution in your Bitbucket pipelines.

We will first clarify the value of using such quality gates in your software development lifecycle. Then, we will provide you with step-by-step instructions for deploying them.

At the end of this article, you will be able to configure and scale your quality gates in Bitbucket according to your requirements.

This article is using the Cerberus Testing demo instance, so you can also reproduce it on your end easily. You can use it on your instance if you desire to.

If you are new to Cerberus Testing, we recommend that you start by having a look at our GitHub page, website and overview video. This tutorial is also available as a video.

Prerequisites

This tutorial requires the following elements:

  • Access to a Cerberus Testing instance at least in version 4, self-hosted or SaaS.
  • An available Cerberus Testing campaign, more information here.
  • Authorization to call the Cerberus Testing APIs, network and gateway grants.

If you have any questions on those prerequisites, have a look at our documentation, videos or contact your administrator. Feel free to interact with the community on our Slack channel or submit a GitHub issue.

Companies have to accelerate for survival

Successful quality assurance patterns emerged from the industry in the last decades. We all heard about CI/CD, Pipelines, Quality Gates. But why?

In fact, cycle of changes have to accelerate within organizations . While becoming digital,  software delivery must be faster and safer.

Consequently, automation has been largely adopted to cope with the speed requirement. For quality assurance, automating tests early in the process became a best practice.

The Value of Quality Gates in CI/CD Pipelines

Performing tests while software changes are delivered is a challenging task. Different roles, positions and processes must work effectively together.

Indeed, this is why CI/CD pipelines evolved to incorporate testing as a mandatory step in the delivery process. When done right, it can provide significant value.

First, it forces us to leverage automation for both deployment and testing, removing costly and error-prone manual steps.

Secondly, automation leads us to explicit the knowledge we can find in the automated tests, thresholds for GO/NOGO for instance.

How Cerberus Testing integrates Automated Tests in CI/CD

Deploying automated tests in a CI/CD pipeline also requires the integration of different systems: The code repository, the pipeline, and the automated tests.

In our case, Bitbucket is providing the code repository and CI/CD pipelines, while Cerberus Testing is enabling the automated tests (Web, Mobile App, APIs among others use-cases).

The integration between the two solutions is managed through APIs leveraging the Cerberus Testing Runner plugin for ease of deployment.

Various integrations are possible depending on the requirements and context. A simple trigger can be configured, or more advanced scenarios can be chosen.

As we use the Cerberus Testing Runner plugin, the integration is “Bi-directional”

Concretely, the automated test campaign is launched and the pipeline will wait for its completion until a defined timeout.

That way, you save precious time by only configuring your pipeline, freeing you from coding specific scripts.

Run Tests in Bitbucket Pipelines with Cerberus Testing 

In this section, we detail the necessary steps for configuring the Cerberus Testing automated tests within your Bitbucket pipeline.

1.1 Add a Cerberus Testing Pipeline File

The first step is to add a specific step in our pipeline to launch the automated tests.

Navigate to the “Deployments” menu of your Bitbucket project. If you already have a pipeline file, you can choose to edit that one. 

You can search for a Pipeline template on the right. Look for “Cerberus Testing Runner”.

The documentation will appear once you select the correct pipeline template. Several examples are available for inspiration and simpler deployment.

For our example, we will define a simple step to launch the available campaign in the demo instance.

We just need to specify those two parameters in the corresponding variables.

1.2 Commit, Test and Confirm the Integration

We are now ready to use this step in our pipeline.

To confirm our changes, we will commit the file to the project. Bitbucket will automatically trigger a pipeline following the commit.

Navigate to the “Pipeline” menu and select the last pipeline that should be running. The new pipeline step will be executing, displaying the logs.

As you can see, the variables and campaign link are available for logging and observability purposes. 

If you access the link, you will see your campaign result directly in Cerberus Testing.

The automated test execution end s when either the campaign is complete or the timeout reach its limit.

1.3 Externalize Cerberus Testing variables for reuse

A good practice is to externalize your pipeline configuration using the Bitbucket variables.

That way, you will decouple your components and allow better reuse, scalability and maintainability of your pipelines.

You can do this by going to the “Repository settings” and defining “Repository variables” with a corresponding name and value.

Once defined, you can use the variable in your pipeline file with a “$” sign.

For example, a repository variable “CERBERUS_CAMPAIGN” will be referenced as “$CERBERUS_CAMPAIGN” in your pipeline.

Scaling your CI/CD with Quality Gates & Automated Tests

The previous section enabled you to define a single quality gate in your pipeline. But typical software projects are more complex.

In fact, a traditional setup will have various test environments. So, the first step is to correctly define your environments in Bitbucket.

Then, you can define a specific pipeline step for the corresponding test environments.

A good practice is to have automated tests for each environment. This will force your tests to be reliable, fast and useful at detecting defects early.

Each quality gate can be configured using the Cerberus Testing Runner pipeline template

Expand your CI CD automated test

We hope this guide will help you to improve your digital product quality with CI/CD quality gates that you can scale.

Other typical integrations, such as Jenkins and Azure DevOps Pipeline are available, feel free to have a look at the various content available. 

If you want to implement your own case fast, you can access the available Plans. We will be happy to support you in your test automation journey.

Leave a comment

Your email address will not be published. Required fields are marked *

Cerberus-Testing Terms Of Service


Website Terms and Conditions of Use

1. Terms

By accessing this Website, accessible from http://vgeouyk.cluster030.hosting.ovh.net, you are agreeing to be bound by these Website Terms and Conditions of Use and agree that you are responsible for the agreement with any applicable local laws. If you disagree with any of these terms, you are prohibited from accessing this site. The materials contained in this Website are protected by copyright and trade mark law. These Terms of Service has been created with the help of the Terms of Service Generator and the Privacy Policy Template.

2. Use License

Permission is granted to temporarily download one copy of the materials on Cerberus Testing's Website for personal, non-commercial transitory viewing only. This is the grant of a license, not a transfer of title, and under this license you may not:

  • modify or copy the materials;
  • use the materials for any commercial purpose or for any public display;
  • attempt to reverse engineer any software contained on Cerberus Testing's Website;
  • remove any copyright or other proprietary notations from the materials; or
  • transferring the materials to another person or "mirror" the materials on any other server.

This will let Cerberus Testing to terminate upon violations of any of these restrictions. Upon termination, your viewing right will also be terminated and you should destroy any downloaded materials in your possession whether it is printed or electronic format.

3. Disclaimer

All the materials on Cerberus Testing’s Website are provided "as is". Cerberus Testing makes no warranties, may it be expressed or implied, therefore negates all other warranties. Furthermore, Cerberus Testing does not make any representations concerning the accuracy or reliability of the use of the materials on its Website or otherwise relating to such materials or any sites linked to this Website.

4. Limitations

Cerberus Testing or its suppliers will not be hold accountable for any damages that will arise with the use or inability to use the materials on Cerberus Testing’s Website, even if Cerberus Testing or an authorize representative of this Website has been notified, orally or written, of the possibility of such damage. Some jurisdiction does not allow limitations on implied warranties or limitations of liability for incidental damages, these limitations may not apply to you.

5. Revisions and Errata

The materials appearing on Cerberus Testing’s Website may include technical, typographical, or photographic errors. Cerberus Testing will not promise that any of the materials in this Website are accurate, complete, or current. Cerberus Testing may change the materials contained on its Website at any time without notice. Cerberus Testing does not make any commitment to update the materials.

6. Links

Cerberus Testing has not reviewed all of the sites linked to its Website and is not responsible for the contents of any such linked site. The presence of any link does not imply endorsement by Cerberus Testing of the site. The use of any linked website is at the user’s own risk.

7. Site Terms of Use Modifications

Cerberus Testing may revise these Terms of Use for its Website at any time without prior notice. By using this Website, you are agreeing to be bound by the current version of these Terms and Conditions of Use.

8. Your Privacy

Please read our Privacy Policy.

9. Governing Law

Any claim related to Cerberus Testing's Website shall be governed by the laws of fr without regards to its conflict of law provisions.

Thank you for your interest in cerberus,

In order to proceed with your SaaS we need some in order to confirm your instances accesses.

Please provide accurate contacts to secure the process.

You can cancel any time the subscription, please read Terms of Use.