Regression tests for ERP cloud

During the implementation of a cloud-based ERP or an update project, companies more often combine ERP with the transformation of business to the cloud. Thus, they initiate two extremely interesting processes, resulting in a profound change in the scale and method of regression testing.
According to its assumption, an ERP system should be flexible enough to function, change and adapt according to business requirements, while minimizing business risks. This means a multitude of tasks related to customization, updates and integration with different platforms. As a result, it translates into many updates created, tested and integrated into one consistent, standard code.

Journey to the cloud

What's important is that the smallest change in one business process entails changes in the others. Therefore, each one must be consistent with the logic of the operations - verified in regression processes. The specificity of ERP update management is a serious test of effectiveness in IT-business relations. It's an interesting challenge for specialists who understand their complexity.

However, when a company decides to move to the cloud and to purchase an ERP based on the cloud - a completely new experience takes place. It has many aspects and perspectives. One of them is the number of updates that require efficient management. For this reason, an important source of change will be the contribution of the company itself.

ERP in the cloud. The new standard

The key element is the cloud provider. The new ERP, e.g. Microsoft Dynamics 365, will be updated on a bimonthly basis, so the work schedule for this is fixed and strictly defined. This is a demanding undertaking that can be very challenging, especially when it comes to manually testing hundreds of processes for regression.

Of course, it is up to the company how it responds to new trends in enterprise resource planning. The investment in Microsoft Dynamics 365 means a transition to the most modern corporate engine, effectively and widely supporting business activities.

Decision making involves considering and calculating risks and costs. In this context, the company may opt for less frequent updates every 3 or 6 months. However, regardless of the decision, the problem of regression tests remains valid - in the second scenario it's only postponed.

Decisions can go much further and focus on even rarer updates or even stopping them. However, this is a strategy doomed to failure in advance. When deciding on a cloud solution, Test Managers, ERP Project Managers or CIOs will have to deal with the issue of update management and determine the number of man-days for regression tests.


In case of a three-month update and with a relatively simple ERP implementation, it's possible to estimate 3 to 4 man-days per regression test every quarter. In other, much more frequent cases it will be 10 to 20 man-days per update. This applies to the standard range of test requirements for a typical customer.

Depending on the frequency of such procedures, this means an average of 40 to 80 man-days per year, on a quarterly basis. If the company decides to update at two-monthly intervals, it will be 120 to 240 man-days per year. This includes the processes of customization, management of new functionalities, etc. This is often a real challenge. This is why it's worth turning to automatic testing.

Automated tools almost everywhere

Automated regression testing tools have long been used in many business areas. However, for cloud-ERP solutions this is a relatively new element. In the case of Microsoft Dynamics 365, the continuous development of the application, including two-month update cycles, requires automatic testing for regression.

Which tool will be the most suitable? Microsoft provides its customers with its own free package - RSAT (Regression Suit Automation Tool). Looking for better usability, efficiency as well as functionality and testing range, it is recommended to choose an independent tool, designed for automatic regression testing in Dynamics 365 environment - Executive Automats, developed by XPLUS.

Tests. When to start?

There are currently 3 scenarios for the integration of automatic implementation or update regression testing:
  • early adoption of the test tool
  • in the post-launch phase (opposite to the previous one)
  • implementation phase (compromised).

When the project of ERP implementation in the cloud is finished, most of the processes are completed, standards have already been selected and approved, the code is frozen and user acceptance tests (UAT) are prepared.

This is a critical moment for the entire ERP implementation process. Before moving to the UAT phase it's worthwhile to initiate an automated testing tool. The scale and number of tests will reach its peak in this phase. In order to minimize the risk associated with the lack of user acceptance, it will be necessary to conduct hundreds of tests, both functional and performance. UAT will also reveal many errors, and the automated testing tool will be the basic element of consistency adjustment after the errors are fixed.

Automated tests designed during this phase will be useful in further development of the system. ROI from the automatic regression test tool will be delivered faster. This is the first possible scenario.

Of course, you can delay the adoption of the automation tool to the post launch phase. It seems reasonable and natural to do so because of the knowledge of the vendor and the cycle of internally generated updates. This is the second scenario.

This can be an advantageous solution because Test Managers or CIOs usually decide to switch to a hybrid and balance automatic and manual regression tests.

Once the system is operational, the portfolio of processes involved in the new solution will be easily accessible. This makes it much easier to calculate risks, estimate costs, evaluate the importance of processes and predict the need for changes and updates in the future.

Then comes the moment to design a new testing strategy tailored to future needs. However, not all entrepreneurs plan to wait that long. In fact, they have to start automatic testing as soon as possible. This is of course possible - depending on the process and standard adopted.

You can choose automatic tests, starting with simpler, less important ones. The set of tested and proven processes, i.e. the safe and proven area of the new ERP environment, will grow systematically.

However, it's important to choose early and complex adoption of automatic regression tests. Equally important is the self-awareness of the organization processes and a deep, experience-based knowledge of the implementation integrating the company. Moreover, it is necessary to correctly predict which processes may be undesirable and how.

Which scenario to choose?

It all depends on the level of advancement of a company with strong IT-business relations. It will be equally important to choose an appropriate integration company, willing to share its experience and best practices - will be equally important.

Many companies are still primarily focused on business rather than technology, but without underestimating their efforts to strengthen their operations, they may choose to automatically test for regression in the start-up phase.

Those who need more time and want to make the right calculations and adapt the process to business and financial opportunities are likely to move on to automatic testing after the launch phase. In any case, it will be a new path, in a new standard - cloud-based ERP.