We are pleased to announce the closure of this incident. The immediate issue has been resolved, and services are operating normally.
However, please note that the root cause investigation is still underway. We are actively collaborating with our database vendor to identify and address the underlying cause of the incident. A separate post-incident review will be issued once the investigation is complete, providing a comprehensive analysis and any necessary preventive measures.
We appreciate your patience and understanding throughout this process. As always, we remain committed to maintaining the reliability and stability of our services.
Posted May 11, 2023 - 08:29 UTC
Monitoring
Currently, we are pleased to report that there are no ongoing troubles with the outage of our Production API endpoint (api.paydock.com). The API is functioning smoothly without any disruptions.
However, we are actively monitoring the situation and awaiting a response from our database vendor. Our objective is to identify the root cause of the failure that occurred previously. We are committed to working closely with our database vendor to gain insights and address any underlying issues.
Rest assured, we remain vigilant in monitoring the situation to ensure the continued stability and reliability of our services.
Posted May 11, 2023 - 07:16 UTC
Identified
We've identified the cause of the API outage we experienced earlier:
One of our back end database instances experienced an unexpected failure, at which point a failover process was initiated automatically, but had caused performance degradation while this was happening.
Currently all database instances have recovered, so performance has been restored to normal levels, however we are still working with our database vendor to identify the root cause of the failure.
More updates will be provided as we learn more.
Posted May 11, 2023 - 05:58 UTC
Investigating
We have received notifications on our monitoring systems that there was an outage of our Production API endpoint (api.paydock.com) between 2:21pm and 2:27pm AEST.
We are currently investigating the root cause of this, and will provide more information on this page.
Posted May 11, 2023 - 04:50 UTC
This incident affected: Paydock Platform (Paydock API).