To ensure the successful implementation of all three (3) PMO-TMO relationship models (previously discussed in Section 4.3), both PMO and TMO must manage a common ICT project. Four (4) typical lifecycles of ICT Projects that are observed in the Malaysian public sector can be classified as follows:
- Ground-up: reflects those ICT projects where manual processes are automated for the first time, and the new system is implemented, regardless whether it is custom-built or through licensing a Commercial Off-The-Shelf (COTS) product.
- Legacy Disposed: reflects those ICT projects where an existing (legacy) system is to be discontinued and replaced with a new ICT system which will be implemented, regardless whether it is custom-built or through licensing a Commercial Off-The-Shelf (COTS) software product.
- Legacy Enhanced: reflects ICT projects where an ICT system is still in production, and substantial enhancements and/or modifications are to be implemented (to enhance its usability and/or upgrade/update the features and functionalities), regardless whether it was originally custom-built or was a licensed Commercial Off-The-Shelf (COTS) software product.
- Production System: reflects an ICT system already in production which has been running and the system’s usage has evolved over time experiencing:
- increased user population,
- increased database size,
- increased number of transactions,
- changes to configurations of device interfaces,
- enhanced security settings, and/or
- changes to system settings in response to contractual or regulatory policies.
-
In summary;
An IV&V Provider may participate in any or all of the ICT Project Lifecycles in various ways that will be explained in the subsequent topics.
4.4.1 ICT Project Lifecycle (Ground-Up)
An IV&V Provider can provide services in the Ground-Up ICT Project Lifecycle by performing reviews and augmentation of testing activities as depicted in Figure 4.7 below.
Given the ICT system is entirely new with no predecessor system experience to leverage on, all core IV&V services are needed to ensure each stage of the ICT project leads to quality outcomes.
The IV&V Provider shall, at a minimum, perform specific core IV&V services related to the following:
The Agency should also consider adding other testing from a host of additional value-adding test types shown in the Section 4.5.2 of this IV&V Handbook, which are needed to ensure a high-quality ICT product is released for use.
Given the system has not been built before and may involve the need for new skills and competencies which The Agency may not have, the IV&V Provider can be requested to perform additional services, specifically the following:
- requirements elicitation workshop,
- data collection and data entry verification,
- services such as risk and security assessments, and
- test automation.
In this ICT Project Lifecycle, product certification is needed if the ICT project involves COTS, with certification occurring at the beginning of the lifecycle. However, certification at the end of the lifecycle is recommended because customization and other added features were modified.
4.4.2 ICT Project Lifecycle (Legacy Disposed)
An IV&V Provider can provide services in the Legacy Disposed ICT Project Lifecycle by performing reviews and evaluations of testing activities, as illustrated in Figure 4.8 below.
Given the system is new and replacing a predecessor system, the ICT project teams can learn from challenges and issues with the prior system. All core IV&V services are still needed to ensure each stage of the new ICT system being procured maintains a quality focus.
The IV&V Provider should, at a minimum, perform core IV&V services related to the following:
The Agency should also consider adding other testing from a host of additional value-adding test types shown in the Section 4.5.2 of this IV&V Handbook, which are needed to ensure a high-quality ICT product is released for use.
Given the system being created is new and may not be of the same technology as the legacy system to be disposed, The Agency may require new skills and competencies which The Agency may not currently have.
IV&V Provider can be requested to perform additional services, specifically the following:
- requirements elicitation workshop,
- data migration and conversion testing,
- services such as risk and security assessments, and
- test automation.
In this ICT Project Lifecycle, product certification is needed if the ICT project involves COTS, with certification occurring at the beginning and at the end of the lifecycle because customization and other added features were modified.
4.4.3 ICT Project Lifecycle (Legacy Enhanced)
An IV&V Provider can provide services in the Legacy Enhanced ICT Project Lifecycle by performing reviews and evaluations of testing activities as illustrated in Figure 4.9 below.
Given the system is not new but is going through a major enhancement, the ICT project teams can learn from the historical records of challenges and issues encountered throughout the system’s lifetime.
This learnings are good inputs for:
- modification planning, design, coding and testing by the Development Team.
- test strategy, planning, design and test results analysis by the IV&V Provider’s test teams.
- metrics analysis, and benchmarking of the system – before and after change.
All core IV&V services are still needed to ensure each stage of the new functional enhancements of the ICT system maintain a quality focus.
The IV&V Provider should, at a minimum, perform core IV&V services related to the following:
The Agency should also consider adding other testing from a host of additional value-adding test types shown in the Section 4.5.2 of this IV&V Handbook, which are needed to ensure a high-quality ICT product is released for use.
Given the system being created is an upgraded version of the legacy system, and may be incorporated with additional new technolog(ies), The Agency may require new skills and competencies which The Agency may not currently have.
IV&V Provider can be requested to perform additional services, specifically the following:
- requirements elicitation workshop,
- regression testing,
- data migration and conversion testing,
- services such as risk and security assessments,
- test automation, and
- maintenance testing to establish confidence that the upgraded version of the legacy system capability has not deteriorated and can continue to operate as expected.
4.4.4 ICT Project Lifecycle (Production System)
An IV&V Provider can provide services in the Production System ICT Project Lifecycle by performing reviews and evaluations of testing activities as illustrated in Figure 4.10 below.
Given the system is already in production and operational, the system’s usage has evolved over time and experiencing:
- increased user population,
- increased database size,
- increased number of transactions,
- changes to configurations of device interfaces,
- enhanced security settings, and/or
- changes to system settings in response to contractual or regulatory policies.
For critical and high-impact systems, even though already operational, it is necessary to conduct maintenance testing so as to establish “continued confidence” that it is reliable, and is able to continue its operation through systematic and objective V&V activities.
To achieve the above, IV&V Provider should be requested to perform, at a minimum, the following services related to maintenance testing:
The Agency should also consider adding other testing from a host of additional value-adding test types shown in the Section 4.5.2 of this IV&V Handbook, which are needed to ensure a high-quality ICT production system continues to be operational.
IV&V Provider can be requested to perform additional services, specifically the following:
- requirements elicitation workshop,
- regression testing,
- data backup and restore testing,
- services such as risk and security assessments,
- test automation, and
- maintenance testing to establish confidence that the production system capability has not deteriorated and can continue to operate as expected.