A survey performed by Panaya in 2012 of 100 SAP-run enterprises revealed that even the most mature testing organizations perform only 15% to 25% of their functional regression testing using test automation tools.
The benefits are attractive to most enterprises - the flexibility of SAP ERP - also creates the biggest deterrent. The hundreds of millions of lines of code represent a virtually infinite number of test scenarios. Several factors increase the complexity of, and limit the viability of test plan automation:
Software code in general, and SAP ERP in particular, is highly data and environment driven,
The cost of creating and maintaining automated test scripts and test data,
Highly complex SAP ERP systems continue to evolve after implementation.
Enterprises rely on manual testing to ensure required functionality. They assign experts from IT and from key business users to complete the necessary testing.
A 2002 study prepared for National Institute of Standards and Technology reported that 50 percent of software development budgets go to testing. Yet, testing often utilizes unstructured and non-standard methods.
Busy Project Managers are tasked with overseeing testing, in addition to the other phases of a development cycle. A certain body of test cases may or may not exist to be executed automatically. The Project Manager determines what needs to be manually tested, and assigns specific test scenarios to the experts.
The experts from IT and business units then evaluate the priority of this testing against all other normal work. For most people, testing is a low importance and non-urgent task. Compounding the testers lack of enthusiasm, manual test scripts tend to be vague documents containing little or incomplete instructions.
The Project Manager will now ask for results. The tester, feeling rush, haphazardly completes assigned tests. They report vague test results, like the scripts.
Test management also tends to to be manual, often using general office products including test documents, spread sheets, e-mail, or instant messages.
Streamlining Manual Testing
Enterprises demand manual testing by experienced business users to ensure the SAP ERP system responds as needed, to support those users.
The manual test process requires better tools to create more effective and efficient planning, execution and reporting.
Panaya offers solutions for enterprises to greatly increase the productivity of SAP ERP manual testing. Their cloud based offerings focus on analyzing the changes to the SAP system and customized code.
First, information is collected at the Customer's SAP production system, including their customized code and the production usage. Analysis of this data yields a risk based approach to reduce manual testing by up to 70%.
Panaya's software captures the business user experience while she navigates the SAP ERP application, and generates detailed and accurate scripts for manual testing. In subsequent executions of the test scenario, the same data can be inserted and results evaluated against the business user's experience.
With this approach, standard script formats are saved in a common repository. Results can be be verified to ensure completeness of testing, and tracking for repairing exposed defects.
Enterprises struggle to manage the system changes of the evolution of their SAP ERP systems, and the rising cost of ensuring those changes function properly. Financial controls and business pressures force short test cycles upon IT departments.
Panaya.com identifies potential errors in customizations and prioritizes required tests. The enterprise can estimate project budgets and timeframes accurately by eliminating the need for a lengthy manual test cycle.
This creates up to 35% savings of the total project budget, while reducing the project duration. The Enterprise can be certain that all high risk scenarios are thoroughly tested