Acceptance Test

Description

• A determination made by the client (prior to roll-out) as to whether or not the new IT system functions as promised.  The Acceptance Test follows the more technical Information Technology Testing (e.g. unit, integration, stress/volume tests), which is generally conducted by the organisation or an independent contractor.

Business Value

• Acceptance testing is the final step before delivering the system.  It proves to the client that the IT system operates in accordance with the functional requirements and officially marks the acceptance of the system as satisfactory for installation and use.  Unless the customer officially examines the finished product and deems it satisfactory, the contractor has nothing to support the assertion that the system is ready for roll-out and that payment is due.

Approach

The Acceptance Test consists of a series of steps (pre-approved by the client) to “try out” every feature described in the functional requirements, using real data, and real or simulated interfaces.  When all testing activities have been completed successfully, the appropriate client decision-maker signs off the official test plan, signifying that the system is ready to be installed and used.

1. Develop the Acceptance Test plans.
1. Determine the conditions under which the system will be tested
1. The team conducting the Acceptance Test is responsible for assessing how the IT package has been set up and whether the requested modifications have been made.  The team limits its focus to whether or not the system functionality meets the defined needs of client organisation.  The team should not attempt to test all of the program code underlying the package software, as this is primarily the vendor’s responsibility (and typically represents a large portion of the cost of the package software).
2. Identify a series of “actions” and “expected results” to prove that the system performs correctly.  Where possible, divide the overall testing into a number of cycles, each building on the results of the previous.  In this way, one of several typical life cycles can be followed:
1. The life cycle of business transactions, e.g. new client, corrections to standing data, sales orders, corrections, deliveries, invoice, overdue payment processing, credit stop, part payment, account close, etc.
2. Time sequences, e.g. new month, normal day, month end, year end
3. Sequences of interfaces in and out of the system,
4. Day in the life of the various users concerned.
3. Obtain concurrence from the “User Manager” that the proposed test steps are adequate to determine system acceptance.
2. Assign testing responsibilities.
3. Set up a “clean environment” for conducting the tests.  (Tests should not suffer from unpredictable results due to other uses being made of the same computer environment—e.g. multiple data or program versions.)
1. Build in the flexibility to change versions and scenarios freely, particularly when parts of a test must be repeated once a problem is resolved.  Given the high number of system changes that can occur during Acceptance Tests, clearly define and rigidly enforce backup and recovery procedures.
4. Conduct the Acceptance Test.  Obtain sign-off from the User Manager, (involving external and internal auditors, if appropriate).
1. Testing normally leads to a high volume of “incident” reports, typically caused by a misunderstanding or miscalculation of the expected results (e.g. due to unanticipated data inputs resulting in a different test).
2. Perform Acceptance Tests in a controlled manner—i.e. report, log and investigate all such “incidents”.  If corrections are applied, repeat that test (along with any other test that may have been affected by the modifications).  If, on occasion, modification to the system itself are required, make these changes without          undue delay, for they can hold up other system testing activities.
5. Obtain customer acceptance of Acceptance Test results and approval to proceed to implementation/roll-out. (Note that acceptance must come from the  Business Organisation and not from an external consultant)

Guidelines

Tactics/Helpful Hints

• Since it is probably not possible (and certainly not reasonable) to test every single set of circumstances that can arise, ensure that the testing strikes a reasonable balance between comprehensive coverage and risk.
• A well-designed testing environment (plus a defined testing baseline) may also prove useful in future years, when validating future system releases and ongoing enhancements.

Resources/Timing

• The actual Acceptance Test can be conducted by contractor personnel under the close supervision of actual users.  In such cases, the contractor personnel serve both as scribe—noting successfully-completed steps, unsuccessful steps (and the conditions under which they failed)—and as on-the-spot “troubleshooter”.  Be aware, however, that Acceptance Tests are often best conducted by skilled users (i.e. trained on the new system), who are highly familiar with the required functionality of the system.
• Clearly specify the roles and responsibilities of the User Manager and Test Leader during Acceptance Test activities:
• User Manager—review and sign off each specific aspect of the testing.
• The User Manager works with the project team to see that the tests are comprehensive and satisfactory and accepts the results on behalf of the client organization, by:
• Ensuring that the definition of the tests provides comprehensive and effective coverage of all reasonable aspects of functionality
• Checking that the expected results have been properly calculated
• Reviewing the impact of any incidents, problems and issues raised and assisting in their resolution.
• Ensuring that the final outcomes of the tests are satisfactory.
• Test Leader (often a core project team member)—the person responsible for monitoring detailed definition and execution of each part of the testing .
• Consulting with the User Manager and all interested parties to define the objectives, and the test details.
• Managing the preparation of test scripts.
• Managing the conduct of the tests, including detailed planning, booking resources (human/environmental), as well as interdependencies with development or other testing activities.
• Monitoring the usage and follow-up of any test incident control forms.
• Reviewing and obtaining agreement on the results of the tests with the User Manager user and other interested parties.
• Ensuring the overall delivery of the successfully completed test.
• Conduct tests as rapidly and efficiently as possible.  Many aspects of testing inevitably run together, flagging many issues for resolution.  These aspects require strong management and control, as well as pre-defined standards and procedures for the testing activities.