Context

All the steps of the planning phase should be gone through. The results are usually established in a separate test plan, if the test level is organised as a standalone activity. In some cases, particularly with iterative or agile development, the test level is integrated into the total process and the test plan is part of the project plan. The effort required to create the plan depends on what is already available. The presence of a master test plan, of Generic Test Agreements, or a Testing line organisation with instructions, templates and standards can make creating the test plan significantly easier, as it is easy to refer to them. In creating the test plan, the test manager should allow for the possible and the impossible. An important factor here is the existing “testing maturity”, or the quality of the test process. If there is familiarity with test phasing, if test tools are available and the testers are using test design techniques, how are the management and reporting normally managed? If the testing is not very mature, the test manager cannot expect too much from the test process or the testers involved in it. This applies to a lesser extent to the maturity of the development or maintenance process that surrounds testing. If this is chaotic and unmanageable, it is probably inadvisable to invest in the “perfect” test process; a “reasonable” test process will suffice.

• The client for the test level
• Aim and importance of the system or package to the organisation
• General requirements
• The organisation of the development, maintenance or implementation process
• The (delivery) plan for the system to be developed or maintained, or package to be implemented
• The method of developing or maintaining the system or implementing the package
• If there is a master test plan, it should be fixed and approved
• Insight into the development and production environment, so that the test environment can be defined.

If this information is not yet available, for example because the development approach is still unknown, it will have a negative effect on the lead time, the effort required for the creation of the plan, or on the quality and required degree of detail. Also required is the willingness and opportunity to agree on all kinds of aspects of the test process.

Method of operation

The test manager, as a rule, is the originator of the test plan. Ideally, a master test plan will be available. On this basis and in consultation with the client, he will formulate the assignment, making an allowance for the four BDTM (Business Driven Test Management) aspects of Result, Risks, Time and Costs. Subsequently, the test manager will prepare himself for the forthcoming phase by holding various discussions with stakeholders and consulting other sources of information, such as documentation. At the same time, he defines the assignment further in close co-operation with the client, and determines the scope of the test level.

In the event that, for the master test plan, a product risk analysis has not been executed, or if it is too general, a detailed analysis is carried out with the client and other stakeholders. This is done in order to establish the required results of the testing for the client (the test goals) and evaluate the risk level of the parts (object parts) and characteristics of the system or package to be tested. This analysis forms the basis of the test strategy and the process advances to an iterative stage. As part of the strategy based on the product risk analysis the tester determines the characteristics/object parts that should be tested, and with which test type and to which depth (the greater the risk, the greater the depth). Then the test costs are estimated in outline form and the test activities are planned (covering the biggest risks as early as possible). This is to be agreed on by the client and other stakeholders and, depending on their views, possibly revised. In that case, the steps are then gone through again. In accordance with BDTM, the client therefore has a clear understanding of the test process and can manage the balance of Time and Money versus Result and Risk. Subsequent to this, the test manager refines the strategy further by determining test units and translating the decisions about depth of testing into firm statements on which coverage is being aimed for. He then allocates test techniques to the characteristic/object part combinations, making allowance for the available test basis, resources and infrastructural provisions. Using these techniques, the test cases (and, for example, the checklists) are designed and executed at a later stage.

The figure below illustrates this.

Figure 1: From assignment and test goals to test cases

Further steps in the plan formulation are that the test manager establishes the test basis, defi nes the test products and builds up the test organisation. The test manager also defi nes the required infrastructure. Test management is furnished with procedures and standards, supported as far as possible with tools. As a rule the elements available in the master test plan, Generic Test Agreements, the test policy or the Testing line organisation are used.

The most important risks that threaten the test process are cited, and possible measures are proposed for managing these risks. As a last step, the test manager has the test plan approved by the client. While the activities in this subprocess are described in sequence, in practice, certain activities will be done several times and/or in a different order. If, for example, certain infrastructure parts are required for a test and cannot be supplied, then the strategy may have to be adjusted.

Roles/responsibilities

The primary responsible role in the creation of the test plan is taken by the test manager, sometimes known as the test coordinator.

Test manager or test coordinator?

While in this section the term of test manager is consistently used to refer to the individual responsible for the test process, in practice it is also often a test coordinator who heads the system or acceptance test. The differences are more emotional and circumstantial than objective, but generally, the following is the case:

• The more authorisations involved, the more the term of test manager is preferred
• The greater the scope of the test, ditto
• The greater the size of the test, ditto
• If an overall test manager is managing the overall test process, test coordinator is preferred
• If a test coordinator is coordinating the overall test process, test manager is preferred

Activities

The creation of the test plan involves the following activities:

1. Establishing the assignment
2. Understanding the assignment
3. Determining the test basis
4. Analysing the product risks
5. Determining the test strategy
6. Estimating the effort
7. Determining the planning
8. Allocating test units and test techniques
9. Defining the test products
10. Defining the organisation
11. Defining the infrastructure
12. Organising the management
13. Determining the test project risks and countermeasures
14. Feedback and consolidation of the plan.

The scheme below shows the sequence and the dependencies between the various activities. Every one of the activities may be gone through several times, as the result of an activity may mean a previous activity needs to be revised. As earlier indicated in the method of operation, the steps 5, 6 and 7 have an explicitly iterative character:

Figure 2: Creating the test plan