Types of automated testing

Test automation can deliver quick results. Provided that you plan it and adhere to the best testing practices and guidelines. Apart from the test automation strategy, you need to know the guidelines which – although they vary as per application type, tools and technology used – can facilitate the automation process. Knowing things in advance and planning the testing ahead can reduce the maintenance effort and make the scripts more usable.

What are the most commonly used types of automation testing? We are discussing it in this article. Some types of testing are more suited for automation than other so it is handy to know them before you decide on your test automation strategy and the choice of the right testing tool.

Black box vs. white box testing

Two main approaches to testing are:

Black box testing means that testing the application does not require knowledge of its internal working or implementation. The phrase “black box” means that the application is like a black box for the tester supplying input and analyzing the output.
White box testing (otherwise called “glass box testing”): is the opposite of black box testing. It means that the tester knows the internal workings and the implementation details, such as code, configs, etc. The tester carries on testing paths relying on this knowledge.

Testing pyramid

This is a different way of organizing tests. The testing pyramid has been designed to give an overview of different levels of testing processes – from the smallest units to big, overall processes.

Level 1:

This is where the white box testing takes place. Unit testing means individual units of code or groups of software units. Unit tests are written in code and tied to software developments; hence they need to be done by developers.

Level 2:

This testing verifies how different units of code interact with one another. Testing involves checking a chain of components (also external) that handle a process or an operation. Integration testing also includes testing interaction between hardware and software but also other components of infrastructure.

Level 3:

This testing covers a process from end to end. The scope of this type of testing depends obviously on the process but these tests often span across different technologies. The purpose of end-to-end testing is to ensure that a workflow works as required from the perspective of an end user. That is why this testing often applies the black box method.

Test automation frameworks

When we talk about types of automation testing, it is worth mentioning the frameworks commonly applied to testing. A test automation framework is a grouped set of rules and corresponding tools used for building test cases. The general rules include coding standards that help testers avoid manual data entry as well as different test data techniques and benefits, object repositories and any additional information that might come in handy to run the tests efficiently and in a suitable manner.

Which testing framework is the most suitable for your organization? The answer depends on the company-specific requirements and sometime internal (department) needs. You can also utilize one or more of them so that your test yields optimal results and are comprehensive.

However, no matter which one you choose, they all give you the same benefits with regards to test automation. Here they are:

Maximum coverage
Code reusability
Low-cost maintenance
Minimal manual intervention
Recovery scenario
Easy reporting

Now, let’s take a look at each test automation framework in more detail while also discussing some of their pros and cons:

Linear Framework (Record and Playback)

This is the most basic testing framework. For each test case, testers write and run a test script. Most of the time, they record and play it back on screen (hence the name). As Linear Framework is very simple, it is mostly recommended for small teams and where test automation has only been introduced recently.

Module (Modular) Based Testing

This approach proposes that the “application under test” (a test case) is divided into smaller parts called modules that are logical, isolated and independent of each other. Then, independent test scripts are created for each of these modules. As the modules are separated by an abstraction layer, any changes that are introduced in the various sections of the application don’t impact the module itself.

Modules of a given scenario are handled by a “master script” which saves testers a lot of time spent on workflow optimization and leads to more cost-efficient maintenance. This framework is also quite scalable. Plus, if the changes are implemented in only one part of the application, only the test script that represents this part of the app needs fixing. However, to successfully implement the Modular Based Framework, testers need to have test automation knowledge and plan the testing ahead. With implementing test scripts separately for each module, data is embedded into the test scrips which means that whenever you need to test with a different set of test data, this requires a lot of manipulations to be made in the test script.

Library Architecture Testing: this framework divides the application intro functions by grouping similar tasks within the test scripts instead of breaking up the test cases. Then, the functions are stored in a reusable library. As the functions include tasks with common goals, whenever functionality is needed, they can be called upon by the test script.

Library Architecture Testing allows greater reusability and offers flexibility in testing. However, writing the test script is more time consuming. Similarly to Module Based Framework, test data is lodged in the test scripts so whenever changes are needed, test scripts need to be altered, too.

Data-Driven Testing: this approach suggests that data and test scripts are kept separately. This allows the teams to test functionality again and again, simply changing the data sets for testing. The Data-Driven Testing framework allows you to store the test data in an external system and access it at any time you need to add new scripts to the test case.

This considerably reduced the total number of scripts and a lesser amount of code is required to test a complete set of scenarios. Data-Driven Testing also means increased flexibility and maintainability. However, the process is quite complex and requires proficiency in programming language for developing tests scripts.

Keyword-Driven Testing: This framework extends the concept behind the Data-Driven Testing framework. It only allows the tests to extract the test data to an external source but also organize the data by specific codes (which are also referred to as “keywords”). The codes can be used to alter the test scripts and conclude different possible outcomes from the test framework. The keywords ultimately dictate the functions performed by every application.

The big advantage of this framework is that each keyword can be used multiple times across different test scripts. However, as the framework gradually grows as new keywords are introduced, it becomes more and more complicated.

Hybrid Testing: This approach is a combination of some of the others outlined above, mainly data-drive and keyword-driven frameworks. It is sometimes called the ideal automation framework as it leverages the benefits of all associated frameworks. Hybrid Testing allows for more scope of testing, increased productivity and higher efficiency.

Behavior-Driven Development framework

It allows automation of functional validations in a clear, understandable format to business analysts, developers, testers, etc. This framework does not require programming language. For BDD, there are also many different tools available for testing.

Types of testing that can be automated

Types of automation testing vary, depending on the needs and requirements of a given organization or department. Most common types of automation testing are used by testers for:

regression tests: they combine functional and non-functional tests to check if the system has not regressed after a change
smoke tests: tests to cover the most critical functionalities of a given software solution.
integration tests: cover all individual pieces and functionalities of a software solution and test them all together to verify if they co-operate correctly.
security tests: these tests cover both functional and non-functional tests and look for any vulnerabilities within the software and reveal any potential exploits in the system
performance tests: these are non-functional tests helping testers evaluate certain system criteria (e.g. stability, responsiveness, etc.) and check how the system handles stress and load.
acceptance tests: there determine if the software if “acceptable” to the end-users. They are run as a final phase before the software is released.

Types of automated testing – a quick guide