Setting up the total test process by:

• Aligning the test levels
• Minimising overlaps or gaps in the test coverage
• Optimal distribution of available resources, e.g.
  • Testers
  • Infrastructure and tools
  • Technical or domain expertise
• Detecting the most important defects at the earliest possible stage
• Testing as early as possible on the critical path of the overall project
• Achieving uniformity in the test processes
• Laying down agreements with stakeholders
• Informing the client of the approach, planning, estimated effort, activities and deliverables in relation to the total test process.

The plan provides insight into the various test and evaluation levels to be used, in such a way that the total test process is optimised. All other test plans must be based on the master test plan. As such, the master test plan constitutes the management basis for the underlying test levels.

Context

A master test plan is necessary when multiple test levels are used. This is usually the case, both for new builds and maintenance or package implementation – either waterfall or iteratively.

After determining which test levels there are or should be, it must be determined which of these test levels are to be included in the  master test plan. In theory, all test levels and evaluation types (reviews, inspections) are eligible, but in practice usually the test levels for system and acceptance testing (see figure below) are aligned in the master test plan.


Figure 1 - Examples of the fields of coverage of a master test plan

To align test and evaluation types in a master test plan, each test or evaluation level must provide insight into what it tests and doesn’t test, and with what intensity. However, alignment becomes difficult or even impossible if such insight is not provided and, for instance, it is known for the development test that a programmer will test, but now how. If such insight can or will not be provided for a specific test level, it becomes very difficult to include it in the master test plan. The project and test management must change this. This goes even more if the master test plan is meant to prescribe what and how the individual test levels should test.

The master test plan must be started early on in the system development process, so that as many test levels as possible can be included. ‘Early’ in this context may mean when creating the requirements, or even on initiation of the project. The later on in the process the master test plan is started, the fewer opportunities there are to optimise and align the test levels.

A master test plan can be created and executed regardless of the ‘test maturity’ of the relevant test levels, i.e. the quality of their test processes. If test maturity is low, however, the test manager must keep this in mind in drawing up the master test plan. He cannot set excessive demands for setup and management of the separate test levels.

• The estimated test effort in the master test plan serves as input for the project proposal in which a budget is requested.
• The test products are also included in the configuration management plan. The test manager must have influence on this to ensure that the test products can be managed according to his wishes.
• The testability review of the test basis can be done in the regular reviews. The quality plan must then provide for participation of the testers in the reviews.

• Aim and importance of the system or package for the organisation
• Global requirements
• The organisation of the development process
• Overall (delivery) planning
• Global system size (in function points or hours)
• The method for the system to be developed or maintained or the package to be implemented
• The client for the total test process

The unavailability of this information, e.g. because the development approach is not yet known, has negative consequences. In other words, the lead time or required effort to create the plan increase, or the quality and desired detail level of the plan are reduced. Furthermore, the organisation must be willing and able to make general arrangements in the fi eld of testing.

Method of operation

As the author of the master test plan, the test manager supports the client in formulating a clear assignment taking account of the four BDTM aspects Result, Risks, Time and Costs, and records these in the plan. The test manager then works on the upcoming process by having discussions with stakeholders and consulting information sources, such as documentation. In parallel, the test manager further elaborates the assignment and determines its scope in consultation with the client. Together with the stakeholders a product risk analysis is performed to determine the required results of testing, and which parts and aspects of the system or package to be tested present higher or lower risks. This analysis represents the basis for the test strategy.

A process with iterative properties emerges. In the test strategy, it is determined which test levels must be set up, the aspects to be tested by the various test levels and with what thoroughness are established (the greater the risk, the more thorough testing). A rough estimate of the required effort is then created for the test levels and the overall test process, and a planning is established (the greater the risk, the earlier). This is aligned with the client and other stakeholders and, depending on their views, adjusted if necessary. In this case, the steps are executed once again. By conforming with BDTM, the client has an explicit level of control in relation to the test process.

Further steps in the creation of the plan are: the test manager defines the products that must be delivered by the test levels and makes a proposal as to the form of the test organisation, centrally and overall per test level. He aligns the infrastructure requirements of the test levels and defines the necessary test infrastructure. Test management can be set up in part at the master test plan level. This can be achieved by both defining central procedures and standards for management and the central management of e.g. defects and test products. Both options aim to prevent reinventing the wheel in the separate test levels. The main (project) risks threatening the test process are listed, and possible measures are proposed to manage these risks. As his last step, the test manager submits the master test plan to the client and any other stakeholders for approval.

While the activities in this subprocess are described successively, certain activities will be executed several times and/or in a different order in actual practice. When, for instance, the infrastructure required for a test cannot be delivered or is judged too costly, the test strategy may have to be adjusted. 

Roles/responsibilities

The test manager, sometimes also known as overall test coordinator, is the role primarily responsible for creating the master test plan. In some cases, the role is combined to include quality assurance (QA) responsibilities, to integral test and QA manager.

Activities

Creating the master test plan consists of the following activities:

1. Establishing the assignment
2. Understanding the assignment
3. Analysing the product risks
4. Determining the test strategy
5. Estimating the effort
6. Determining the planning
7. Defining the test products
8. Defining the organisation
9. Defining the infrastructure
10. Organising the management
11. Determining the test project risks and countermeasures
12. Feedback and consolidation of the plan.

The diagram below shows the overall order of and dependencies between the various activities. All activities may be executed several times, because the results of one activity may require a review of the previous one. As mentioned in the method, steps 4, 5 and 6 have an explicitly iterative nature:


Figure 1: Creating the master test plan