Upon starting the Post-Engagement phase, the following will take place;
- The Agency would typically do the following:
- MyTCoE remains involved with The Agency in areas of monitoring and Post-Engagement reviews.
- The Development Team will typically provide warranty period to The Agency for ICT they delivered.
- The IV&V Provider would have completed their IV&V Engagements activities, and clearing up its contractual obligations, by participating in the completion of the reports close-out, analysis of lessons learned, and transfers of test wares.
The Agency, must take advantage from this phase to learn from the experiences gained by The Agency and its stakeholders during the IV&V Engagement and subsequently promote knowledge sharing within the Malaysian public sector.
Figure 5.1 below lists the key activities to be performed during the Post-Engagement phase so as to be able to realise the knowledge sharing. 
5.1.1 Data Report Out
To perform the Data Report Out, The Agency must ensure the data are gathered and stored in dedicated project repositories from the start of the Engagement phase and continuously gathered/updated throughout the Engagement Phase. A complete, large and accurate set of data collected across the IV&V Engagement, shall facilitate a more encompassing, detailed and comprehensive Post-Engagement analysis to be performed.
Examples of data collected are:
- Function Point counts,
- data types related to test cases and defects encountered,
- project management-related data, such as effort estimations, planned vs actual effort etc.
- development activities related data, such as number of defects encountered, change requests etc.
As mentioned earlier during the Engagement phase, the Quality Gates 1-4 (QG1 – QG4) will review the status of data gathered and ensure its submission to MyTCoE for more advanced analysis.
Function Points count
Function Points count derived during this Post-Engagement phase are benchmark Function Point counts because:
- the system is ready for release and
- the calculation of Function Points directly reflects the actual system’s final and completed sizing.
It is to be noted that estimation and sizing Function Point counts obtained during Quality Gates 1, 2, 3, and 4, (i.e. where these counts reflect the system that is still work in progress), the final Function Point count is a more accurate sizing of the finalised system.
Defect Removal Efficiency (DRE)
As part of a data-driven management approach at the national level, both MyTCoE and The Agency must measure the Defect Removal Efficiency (DRE) as a rigorous calculation after the system goes live. The DRE calculation shall be measured during the Post-Engagement phase. Figure 5.2 below depicts the DRE Formula.
If the number of defects reported during the first 6 to 12 months of production is low, then the result is a high DRE score. MyTCoE and The Agency shall aim for high DRE scores because this encourages the Development Team to find and fix defects before the ICT system goes into production.
For example, a DRE score greater than 85% is a strong goal, which would put Malaysian organizations among the world’s best at ICT procurement. This leads to the economic benefits of an improvement in a reputation for quality for Malaysian development organizations and for public sector agencies.
5.1.2 Post-Engagement Review
The Post-Engagement Review, is usually carried out during the first 6 to 12 months after completion of the IV&V Engagement phase. During this review, MyTCoE and The Agency shall do the following:
- Compare outcomes against expected results, including DRE.
- MyTCoE and the Development Team discuss lessons learned and testing improvement ideas which can be considered for future ICT project(s), where:
- The Agency will benefit from the knowledge gained about internal workforce skills and updated understanding of testing capabilities with a quality focus for the entire ICT process.
- MyTCoE will benefit from sharing with the IV&V Providers about how to update their services based on lessons learned, raising quality for future IV&V Engagements.
5.1.3 Continual Improvements for Future IV&V Engagements
Continual Improvement is important because it ensures future projects are executed better than current. Various approaches can be used to implement continual improvements such as:
- adhering to international standards, such as ISO 9000,
- adhering to international practices, such as CMMi, TMMi,
- adhering to systems and software engineering quality assurance (SQA) and industry good practices.
Taking cognizance of all the above approaches, it can be summarised that continual improvement can be achieved in future engagements by meeting the following quality expectations of ICT projects:
- Consistency of service and product realisation.
- Repeatability of service and product realisation.
- Predictability of service and product realisation.
- High customer satisfaction index.
To achieve the above expectations, IV&V Engagement data from multiple projects must be gathered and then processed to form a Malaysian public sector’s Process Capability Baseline (PCB). This PCB can be used by MyTCoE and The Agency to measure the service performance of future Development Team and IV&V Provider in ICT project.
Thus, continual improvement can be measured (quantified) and controlled.