Maintenance Windows for Sauce Labs
We are currently aware of an ongoing issue impacting the Talkback functionality on our Real Device Cloud. Our team is actively working to resolve this as quickly as possible.
2024-09-15 Resolved Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Saturday September 15th, 2024 17:02 - 20:15 UTC
What happened:
Our Real Device (RDC) availability dropped below 80% across our US West and US East regions.
Why it happened:
The issue was caused by a failure in our certificate renewal process set to occur automatically, but failed to run as expected.
How we fixed it:
We fixed the problem by restarting the impacted service, which automatically renewed the necessary certificates. This brought our device availability back to normal.
What we are doing to prevent it from happening again:
We have initiated an investigation to review and fix the renewal process script to ensure it functions correctly in the future.
Resolved
Between 17:02 and 20:15 UTC, automated tests executed on our real devices in the US West Data Center experienced an increased error rate due to reduced availability of the devices. The issue has been identified. All services are back to normal.