On 16 November 2020, an outage occurred shortly after a major upgrade to ASX’s equity trading platform, ASX Trade, called the ASX Trade Refresh project (the project).

ASIC and the Reserve Bank of Australia (RBA) (the regulators) view operational incidents of this nature with significant concern. To examine the issues, the regulators informed ASX of their expectation that an independent review of the project be conducted in the first half of 2021. In consultation with the regulators ASX appointed IBM Australia Limited to undertake an independent expert review.

The purpose of the independent expert review was to examine the project and assess whether it met internationally recognised standards or frameworks and relevant securities industry practices.

The independent expert, IBM Australia Limited, has reported its findings and a summary of its conclusions is attached.

Overall, the independent expert found that ASX met or exceeded leading industry practices in 58 out of 75 of the capabilities assessed, including:

• business case development and project change management, which exceeded accepted practices
• the project was provided with and had access to sufficient financial, time, people and technological resources at all stages of delivery to meet its objectives
• communications with key stakeholders were appropriately managed, and
• incident management actions taken by ASX were appropriate.

Significantly, the independent expert identified several key shortcomings in the project including:

• factors that suggested the ASX Trade system was not ready to go-live considering ASX’s near zero appetite for service disruption. This was the case even though the formal implementation readiness processes were completed and verified by multiple parties without objection to go-live
• there were gaps in the rigour applied to the project delivery risk and issue management process expected for a project of this nature, and
• risk and issue management, project compliance to ASX practices, project requirements and the project test strategy/planning did not meet accepted industry practices. It was not reasonable to expect the test plan used would meet the ASX’s near zero appetite for service disruption.

The independent expert made recommendations in seven key categories: risk, governance, delivery, requirements, vendor management, testing and incident management.

ASX has provided the regulators with its high-level response to the independent review.  The regulators note that ASX has agreed to address the recommendations from the review, and apologised for the disruption to the market.

ASIC Chair, Joe Longo said, 'The independent expert found that ASX met or exceeded leading industry practices in most areas, but the conclusion that the project was not ready for go-live is very disappointing.  ASX has acknowledged and accepted the need for improvement.  We do, however, require assurance that these improvements are implemented effectively and result in an overall improvement to ASX’s enterprise wide project management practices.'

The regulators expect ASX to apply the insights from IBM’s findings across the ASX Group to ensure existing and proposed projects, including the CHESS replacement program, are managed and implemented appropriately.

The regulators will consider the independent review, including the recommendations and ASX’s formal response. ASIC is also undertaking a separate investigation into the ASX Trade outage to determine whether ASX met its obligations under its Australian Market Licence, which is ongoing.

ASIC is continuing to engage with market operators, participants, institutional investors and other stakeholders on the impact of the incident and will work with the industry to identify what, if any, broader market adjustments might be necessary to reduce the impact of any future incidents.

Summary of independent review conclusions

1. If it was reasonable to expect the new trading platform was ready for successful production implementation and ongoing availability

Although the formal readiness processes were completed and verified by multiple parties without objection to go-live there were 7 factors suggesting the platform was not ready for go-live considering ASX’s near zero appetite for service disruption. These factors were:

• historical software product quality indicators
• additional testing needs noted
• the quantity of open defects
• gaps in end-to-end test coverage
• proximity to year-end change freeze windows for participants
• risk likelihood ratings
• a lack of evidence of challenges to the risk rating or to go-live

2. Whether the project had sufficient resources

The project was provided with and had access to sufficient financial, time, people and technological resources at all stages of delivery to meet its objectives. However, IBM also concluded the project could have benefited from additional and independent scrutiny.

3. The efficacy of the change control process

The project change request process was executed consistently with ASX standards. Project change requests were reviewed by the appropriate internal governance forums and were all deemed suitable for acceptance.

4. The robustness and rigour applied to risks and issues management

There were gaps in the rigour applied to the project delivery risk and issue management process expected for a project of this nature. These gaps included:

• opportunities to identify additional risks were missed
• differences between project delivery risk templates and the enterprise delivery risk processes
• the project would have benefited from involvement of risk resources with greater experience in technical projects  
• governance was shifted to a group that had a wide range of responsibilities, and the group did not include a key role. The shift diluted attention given to the project.

5. Whether it was reasonable to expect the project test plan would be effective, commensurate with the risk appetite and criticality of the ASX Trade system

It was not reasonable to expect the test plan used would meet the near zero appetite for service disruption for a systemically important national infrastructure as stated in the ASX test policy.

6. The implications of the project on stakeholders

There were negative implications experienced by both market participants and ASIC because of the incident, however ASX was unable to anticipate the emergence of these implications for stakeholders.  Communication with key stakeholders were appropriately managed during project delivery, pre go-live and post go-live.  The incident management actions taken by ASX were deemed appropriate and resulted in the correct course of action to reduce the impact upon project stakeholders.

7. Whether during the 2020 incident, ASX took into consideration the lessons learnt from the 2016 incident

The actions taken by ASX during the 2020 incident were appropriate and reflected the lessons learned from the 2016 incident.

8. The aspects that met or exceeded industry standards, frameworks or practices

The majority of the ASX project practices met the expectations of leading industry practices. Business case development and project change management stood out as areas that exceeded accepted practices. Evidence of continuous improvement, during and post the project, were also noted in enterprise practices, especially risk, project delivery and business continuity management.

There was good alignment to accepted practices for governance, risk management and project/program delivery. Whilst only an indication, of the 75 capabilities in the scope of the review, 58 met or exceeded industry practices for a project of this nature.

9. The aspects of the project that did not meet or exceeded industry standards, frameworks or practices

Whilst the majority of ASX project practices met the expectations of leading industry practices, IBM concluded that risk and issue management, project compliance to ASX practices, project requirements and the project test strategy/planning did not meet accepted industry practices. Regarding the test practices employed by ASX during project delivery, IBM concluded that ASX's test documentation and related process implementation were largely not consistent with leading industry practice expectations. The existing governance processes failed to act timely enough resulting in an extended period before an upgrade was performed, raising the operating risk.

Initial project governance processes did not exhibit sufficient program management scrutiny on the compliance to ASX delivery practices. IBM noted ASX did not have a formal quality management process and there were opportunities to improve ASX’s enterprise-wide project practices.

Read the Reserve Bank of Australia's media release.