This IV&V Handbook provides guidance to The Agency on pragmatic best practice methods, models and techniques for the successful IV&V engagement to complement The Agency’s ICT Project.
It has been developed through:
(a) the synthesis and fusion of the latest good practices in the public sector worldwide,
(b) pragmatic lessons learned from pilot implementations of the IV&V engagements (using various ICT project lifecycles) in Malaysia as part of the Pengujian Perisian Sektor Awam (PPSA) Pilot 2013-2015 initiative,
(c) incorporating latest government circular(s) related to IV&V, as listed in the Pre-Requisites of Section 3 Pre-Engagement Phase,
(d) consistent adoption of IEEE Std 1012 i.e. standards for V&V, and ISO/IEC/IEEE 29119 i.e. standards for software testing. The standards also address IV&V (and/or third party testing) for different systems, hardware, and software products including a full system, sub-system, or part of a system, including:
- Quality levels,
- Minimum acceptable tasks to be performed,
- Intensity and rigor needed for IV&V services, and
- Detailed criteria for IV&V tasks.
This IV&V Handbook guides the IV&V implementation by focusing on understanding and establishing the context for IV&V which covers:
(a) four (4) foundational areas of definition of IV&V,
(b) guiding principles of IV&V,
(c) stakeholders and scope of MyTCoE’s support for IV&V, and
(d) the IV&V engagement process.
Figure 1.1 below summarizes the five (5) core areas that must be understood, adopted and implemented by The Agency to yield a successful IV&V engagement.
Note: These core areas are explained in detail in subsequent sections of this IV&V Handbook.
Figure 1.1: Executive Summary
Stakeholders
Four (4) stakeholder groups must be recognized in any IV&V engagement process. They are:
(a) The Agency,
(b) IV&V Provider,
(c) Development Team, and
(d) MyTCoE.
Engagement Phases
The IV&V engagements focus on three (3) distinct engagement phases, which are:
(a) Pre-Engagement phase :- It specifies distinct activities that must be completed before an IV&V engagement can proceed,
(b) Engagement phase :- It specifies:
(i) distinctive components of testing and related IV&V services to be secured,
(ii) alternatives for PMO-TMO relationships,
(c) Post-Engagement phase :- It specifies distinct activities to be completed for on-going success of the ICT software system acquired and implemented.
PMO-TMO relationship
This IV&V Handbook details key principles for designing successful IV&V partnerships and benefits that can be expected from such arrangements. The PMO-TMO relationship is a critical one in the IV&V client-vendor relationship where the Test Management Office (TMO) has a major role to play. This IV&V Handbook lays out the TMO’s roles and responsibilities in Section 4.2.1 TMO Functional Overview on page 68.
Three (3) options are provided for consideration by The Agency, as follows:
(a) A standard relationship, where The Agency and IV&V Provider are independent parties that function on a peer level,
(b) A collaborative relationship, where the IV&V Provider’s TMO augments The Agency’s PMO in the project’s approach to quality and IV&V implementation,
(c) An integrated relationship where they are merged to operate as Project Test Management Office (PTMO).
ICT Project Lifecycles
Four specific ICT project lifecycles are defined to guide The Agency in developing an appropriate IV&V client-vendor relationship and to select specific testing services on a value-add capability augmentation basis.
The four (4) ICT project lifecycles are:
(a) Ground-Up,
(b) Legacy Disposed,
(c) Legacy Enhanced, and
(d) Production System.
Additional IV&V Services
This IV&V Handbook provides guidance as to specific test levels and test types that should be selected as part of an IV&V engagement. These combinations of test levels and test types should augment the testing performed by the Development Team. Some examples are depicted in Table 1.1 below:
Table 1.1: Examples of test levels and test types
Additionally, advanced additional service that can complement and add further value to the IV&V engagement for additional qualities are also highlighted. These include:
(a) risk assessment and planning,
(b) security assessment and planning, and
(c) test automation.