Monitoring - The reload of the network device has now been completed and engineers are currently reviewing impacted services. Initial monitoring reports are promising.

Mar 17, 22:52 AEDT
Update - Engineers are now in the next stage of remediation works and are preparing to conduct a controlled reload of a network device. It is expected that once completed connectivity for disrupted services will be restored.

This is now commencing.
Mar 17, 22:27 AEDT
Update - Engineers are continuing remediation works that are well underway. The current estimation for work completion is within another 60 minutes, with applicable updates to be provided.
Mar 17, 21:45 AEDT
Update - Engineers have commenced works now. Updates will be provided as soon as possible.
Mar 17, 21:20 AEDT
Update - Engineers are finalising the planned remediation works now, with a few minor tasks remaining before the works commence to prevent any unexpected disruptions.

As mentioned in the previous update an additional notice will be provided once these works are about to commence.
Mar 17, 20:28 AEDT
Update - After further consultation with our hardware vendors, Engineers are planning to implement further remediation works which are expected to alleviate the remaining service disruption being experienced.

Final preparation is underway and this is expected to be initiated in the next 60 minutes. A further update will be provided in relation to this, prior to commencing.
Mar 17, 19:24 AEDT
Update - Hardware vendors are actively assisting the onsite team with reviewing affected network infrastructure and are working to provide the next steps towards a resolution for the remaining impacted services. Service restoration for all impacted clients remains our top priority.
Mar 17, 17:34 AEDT
Update - Engineers are currently working further with hardware vendors to investigate the cause of the remaining disruptions, with onsite engineers assisting with additional investigative efforts.

We are actively working to identify and resolve services still impacted and will continue this work into the evening.
Mar 17, 16:54 AEDT
Update - Network engineers are continuing their work to rectify disruptions for clients that are still impacted. At this point in time work is being directed to resolve connectivity for services utilising firewall appliances from our SY1 datacentre.

A further update will be provided once this has progressed further.
Mar 17, 16:18 AEDT


Update - Engineers have completed another round of changes and additional services are now reporting in with connectivity restored. We're continuing works this afternoon to bring online all remaining disrupted services.
Mar 17, 15:34 AEDT

Update - Engineers are working with our hardware vendors to further investigate affected network infrastructure in the SY1 datacentre which is believed to be the cause of disruption for remaining services. As we continue to coordinate onsite works we will provide further updates accordingly.
Mar 17, 14:34 AEDT
Update - Engineers are continuing to troubleshoot and review the remaining services that are still reporting network disruptions at this point in time.
Mar 17, 13:40 AEDT


Update - Engineers are preparing to shift some traffic within our Sydney infrastructure to resolve this disruption. This may cause a path change for some services with active networking along with TCP resets due to the path change.

An update will be provided again once this has been completed.
Mar 17, 12:48 AEDT

Update - Engineers have begun to implement additional changes that are seeing some affected services from within our Sydney region restored. We are continuing to work on restoring service connectivity for all impacted clients as our top priority.
Mar 17, 12:25 AEDT

Update - Engineers have completed initial works which have not proven successful in resolving experienced network disruptions. Engineers are continuing to work on a resolution as a matter of urgency.
Mar 17, 12:06 AEDT
Update - This maintenance window has been extended due to the complexity of the maintenance works required. At this time all troubleshooting effort is being invested to restore connectivity for affected services and is our utmost priority.

We continue to appreciate your cooperation during these emergency works.
Mar 17, 11:48 AEDT

Update - Engineers are continuing to conduct emergency works to rectify service disruptions being experienced by some clients in our Sydney locations. No ETA for work completion is available at this point in time.

Further updates will continue to be provided on a regular basis.
Mar 17, 11:31 AEDT
Update - Engineers are currently still working inside of this emergency maintenance window to rectify service disruptions being experienced. We appreciate your patience during this time.
Mar 17, 11:14 AEDT
Update - Engineers are currently undergoing the emergency maintenance window now. During this time some services may be impacted for several minutes.

An update will be provided once complete.
Mar 17, 10:58 AEDT
Update - Engineers are preparing to commence emergency maintenance works to resolve the network disruption being experienced by some clients. The emergency maintenance may result in a continuous disruption of several minutes while the work is underway.

An update will be provided once this maintenance is completed.
Mar 17, 10:48 AEDT
Update - Engineers believe they have identified the root cause being experienced currently and are working to organise an emergency fix to restore full connectivity for the remaining services. A further update will be provided again as soon as possible.
Mar 17, 10:20 AEDT
Update - Engineers are still conducting further troubleshooting to identify the root cause and restore connectivity for the remaining services that are experiencing a disruption.
Mar 17, 09:41 AEDT
Identified - Engineers have implemented a minor change to restore connectivity to affected services, and monitoring has confirmed network is restored for some services. Engineers are manually checking the remaining services.

Further analysis will be performed to identify the source of the disruption this morning.
Mar 17, 09:21 AEDT
Investigating - Engineers are reviewing further reports of network disruption that is impacting services in our SY4 locations. Further updates will be provided shortly.
Mar 17, 09:01 AEDT
Monitoring - Engineers have had to perform a controlled network event to restore connectivity to some services this morning. During the connectivity resynchronisation, services in our SYD2 location may have experienced a disruption of up to a few minutes. Further investigation is underway to review the cause of this disruption
Mar 17, 06:29 AEDT
Investigating - Engineers are reviewing monitoring alerts in relation to a potential network disruption experienced by a small portion of clients within the Sydney region this morning.

Further updates on the matter will be supplied when available
Mar 17, 05:49 AEDT


Wednesday, March 17, 2021







« Back