The Inexpensive Way To Quickly Monitor Your Automated Tests

cerberus-testing-the-inexpensive-way-to-monitor-your-automated-tests

Quality at Speed requires reactivity.

Automated tests allow you to verify the state of software, but their value decreases with the time to react. Hence, the capability to receive live notifications based on tests results is a must-have.

Your test automation platform must contain these features out of the box to let you focus on improving the business and the product. You have no time to lose in coding these features.

Cerberus Testing is the open-source test automation platform providing you instant notifications on your test campaigns. You can quickly receive updates by email, Slack, Google Teams, or Google Chat.

This article is a tutorial about setting up your test automation notifications. At the end of this guide, you will be able to configure automated test monitoring with flexibility.

You can implement it on your own using a Free plan.

Start by defining your test campaign content

Your number of automated tests will rapidly grow. You will reach a point where you need to run a set of tests under defined conditions and be notified only once. Test campaigns available in Cerberus Testing are the solution.

You can create a test campaign in Cerberus Testing by accessing the “Run > Campaign” menu. From there, you can set a fixed (tab “TestCase List”) or dynamic list (“Environments, Countries & Robots list”) of test cases to be part of your campaign.

Figure 1: The dynamic way to define a test campaign in Cerberus Testing.

The dynamic way is usually recommended for a decoupled and scalable test approach. You just have to define the test automation criteria to be part of your campaigns, such as system, application, environment, country, or robots. The tests will run in the QA environment for two countries on Chrome in the above example.

You can then define additional execution parameters based on your needs.

Fine-tune the campaign execution parameters

The monitoring of your automated tests helps you to react to unmet requirements. A failed execution usually means that a problem is happening for your users and should be fixed quickly. Cerberus Testing gives you advanced traceability to diagnose issues quickly.

You natively access to a variety of campaign execution parameters to improve your automated test’s reliability. You can act at three levels:

Traceability of execution, page source, robot, and console logs.
Visibility through screenshot or video (Zalenium only) recording.
Execution for timeout, retries, or priority.

Figure 2: The test campaign execution parameters of Cerberus Testing.

Good practices are to increase the traceability only for failed cases to optimize both the execution and debugging. Retries should be used carefully to avoid hiding real instability issues that your customers are actually experiencing.

The next step is to select when to execute the campaigns.

Setup the execution with scheduling, CI/CD, or APIs

Reacting to your test automation campaigns requires them to run at the right time. You will essentially need to trigger your tests in two ways: via scheduling or an external trigger. Cerberus Testing natively supports both options.

Scheduling enables you to execute your test campaign in a defined context regularly. For example, you can perform regular sanity checks of a test environment or run them in production environments for customer journey monitoring. Cerberus Testing allows you to define multiple scheduling for the same campaign for maximum flexibility.

Figure 3: The native scheduling of test campaigns available in Cerberus Testing.

The second option is to trigger your test campaign from an external source. You can leverage the native CI/CD connectors for Jenkins, Gitlab, among others. Additionally, you can script your own integration via using the public APIs of Cerberus Testing.

Now comes the time to set the notifications.

Define the notifications triggers and channels

Notifications are essential for reactivity. You want to receive them in a common place to see them on time and treat them right when they appear. Cerberus Testing provides you with a variety of built-in connectors.

The need for an accelerated collaboration in a remote world increases significantly the use of instant messaging. Slack, Google Chat, and Microsoft teams are the most widely used business communication solutions. You can natively receive notifications on these platforms.

Figure 4: The native notifications connectors available in Cerberus Testing.

You can define one or more notifications channels per campaign. Each connector lets you define in which cases you want to be notified (campaign start, end, or end with failure(s) only). You only need to set the channels here. Consult this article if you need guidance for creating a notification channel.

You are now ready to react quickly to your test automation.

Deliver Quality at Speed with improved reactivity

You will receive notifications based on your tests results and monitoring criteria. Your team can now collaborate better on test automation: when a test fails, your team can take immediate actions to fix the issue.

Figure 5: An automated slack notifications for a Cerberus Testing campaign.

This improved capability makes a difference in achieving Quality at Speed. Companies must accelerate to remain of value for their users. Notifications are true enablers to perform software changes in shorter cycles.

Cerberus Testing lets you focus on your business priorities by natively providing the necessary set of test automation features. Your goal is to leverage test automation to improve the product quality, now.

Access to Cerberus Testing for free here.

The Inexpensive Way To Quickly Monitor Your Automated Tests