On Building a Unified Test Automation Framework

Test automation is hard.

Historically, at the end of the software value chain, we have to implement automated tests when automation requirements were missed upstream.

And usually, there is no test automation framework available, resulting in a project within the project.

We then start to build a framework to at least configure and execute the minimal automated tests needed.

But then we realize we need even more, eventually reaching the conclusion that a unified test framework was probably a better choice.

Let’s see what happens during that journey to better anticipate it.

Follow Cerberus Testing for more open-source test automation.

Why we need a test automation framework

Companies require to accelerate their rate of software delivery to increase their value proposition faster than competitors.

The software delivery lifecycle must be streamlined for fast iterations.

But speed alone is not enough; stability is a must-have for the user experience.

Test automation comes to give more confidence to the team to deliver software changes at an accelerated rate, automating a set of verifications.

The need for fast implementation also cascades to test automation, where a framework brings speed avoiding reinventing the wheel each time.

We then need more than one framework

From a first experience, we build one typology of framework, for example for web testing in keyword-driven testing.

But with a few more tests and applications to test, we realize requiring:

• Parallelization of tests to keep execution time short;
• Execute different types of tests, for example on mobile and APIs;
• Add the capability to define test in Behavior-Driven-Development (BDD);
• Use variable data sets for a particular test for more qualitative tests;
• Add the support to multiple application and environments;
• Build a library of reusable actions and controls for modular testing.

This list starts to be quite extensive. With a few hours available, the best option is to select the most important items and implement them fast in a local framework.

There is no time to reflect on the entire platform, bringing too much complexity and implementation overhead in this first step.

This approach works for some time, until the developer coding the framework leaves or that we just have no time to update libraries or implement new features.

We realize the need for a hybrid framework

The big learning from the previous step became: “If we had planned and designed better, we would have built a hybrid framework”.

It’s true that this approach enables us to reuse part of the functionalities of one framework for another one.

But one assumption was to have sufficient time to implement such reuse.

Teams are usually under pressure to deliver on a daily basis, maximum at the sprint level improvements for the final product; not for the test framework.

After all that time passed, we realize building a test automation framework is a long journey, much like a marathon than a sprint.

We start to consider ready-to-use solutions

After all the investment made, someone will ask “there was no ready-to-use solution on the market fulfilling similar needs?”

Either someone has a good answer, or there will be silence.

A lot of teams do not consider available solutions for the following reasons:

• We just need a few features, there’s no need for such complete product;
• We have special processes and needs, building our own is the only way;
• We don’t have a budget approved, so we can only build internally;
• etc.

The thing is that only a few companies have really specific needs or need minimal features.

The vast majority will require the same type of features over time, hence making sense for actors to emerge with reusable products for common problems.

We start using a unified test automation framework

We pass through this journey building Cerberus Testing. At that time, there was no real offer on the market, wherever paid or open-source.

It started for web testing, then mobile, then APIs testing, being the reason for naming it “Cerberus”, the 3-headed dog guard of hell in Greek mythology.

We started with keyword-driven testing, and rapidly added support for BDD, modular, data-driven testing part of the framework.

In the end we built more than a hybrid test automation framework: we made a unified test automation framework available on a single interface.

Stop coding, start testing today.