Dennis Janssen

Reporting and Advice

“How’s testing going? Is the information system ready to go into production?” As soon as the system is delivered to the test project, all the parties involved are interested in the first test results. The test manager will provide regular insight into the progress, the quality of the information system and any problems surrounding the test project. Through regular reporting, the test project holds the attention of the client and stakeholders. They know then which product risks and related requirements are covered and which risks still have to be tested. With this information they can make a decision on whether to go on to a subsequent test level or to put the information system into production.

Evaluation and Transfer

When the information system has successfully gone into production and the test project has been discharged, the test manager disbands the team, takes his leave of the project and moves on to the next one. His responsibilities seem to end here. But is that right? If he never looks back at what went well and what did not during the test project, and why, there will be no improvement in the following project. Only by making an evaluation can he, as well as the organisation, learn from and improve on the test process. And if he fails to organise the maintenance of the testware for the information system, he does himself or a colleague in a subsequent project a grave disservice. In short, the test manager must never forget evaluation of the test process and transfer of the testware, even though the information system is operating successfully in the production environment.

The Test Organisation

The scope and depth of the test are set out in the test strategy. The test manager has based the planning and estimating on that information. The cluster matrix sets out which test levels will be carried out. A system test will largely be performed within the IT project, while an acceptance test will be carried out by the business side (a users’ organisation or a central test department). The test organisation should be described in the test plans. Sometimes a reference to documentation already present in the organisation is sufficient.

Risk & Requirement Based Testing

Risk management plays an important role in management practice. In test projects, two types of risk are involved: project risks and product risks. Test project risks are related to problems that may endanger the test activities in a project. At the same time, insight into the product risks is important if decisions have to be made on whether to put an information system into production. In consultation with the parties directly involved, the stakeholders, the test manager surveys these risks and their impact.

The Quick Scan

It is advisable for the test manager to first carry out a brief feasibility study of the test project. In this “quick scan” phase, the test manager establishes the status of all the important aspects of the test. To do this he employs various components of the Test Management Model. Among other things there are the presence and usability of standards, organisational and budgetary instruments, management involvement and availability of resources. This inventory process should not be too lengthy. Its duration depends on the scale and complexity of the project, but should be no longer than 5 days. The test manager can use the information to start the necessary preparations at an early stage. The results of the study are discussed with the client and other stakeholders and are used to finalise the assignment of the test manager. The quick scan is only used to make an inventory and is not part of the test project preparation.

The Test Management File

In order to maintain a good overview of the test project, the test manager must establish and manage all information relating to it. This means not only the documents created by the test manager himself but also, for example, the testware developed by the testers and all other information of importance to the test project. Having a clear structure helps the test manager to provide information to the stakeholders in a clear and efficient manner giving them a better insight into the project. The test management file discussed here equips the test manager with a model for distributing information.

The Dynamics of Testing

Following the preparation, it is time to go on to design the testware and eventually to perform tests on the information system. The test strategy and the test plan have been developed and signed off by the stakeholders. Nevertheless, in practice, the analysis and test execution do not always proceed according to the predictions made in the test strategy and test plan and based on the information that was available at the time. The project and its context are always in motion. Within the dynamics of the test execution, the test manager’s attention shifts away from creating conditions to controlling the situation. Everything must be geared towards achieving a successful result, for testing is not something you do “on the side”. It is a specific profession. The management required and the potential problems demand specific knowledge. During this phase, the test manager has the opportunity to demonstrate his added value.

Testing and Test Management

Automated teller machines that issue the wrong banknotes, Space Shuttles that crash on landing, telephone networks that go down and websites that are vulnerable to hackers are all familiar examples of ICT systems that fail, and there are new examples almost every day.