Maintenance Windows for Sauce Labs
We are currently performing maintenance that will affect the initiation of new Sauce Connect tunnel instances. The maintenance is expected to be completed by 2:00 PM PST. We appreciate your understanding and patience as we work to enhance our service.
2025-March-6 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Thursday March 6th 2025, 12:00 - 12:22 UTC
What happened:
Customers were unable to start new Android Real device tests in all datacentres.
Why it happened:
During a deployment, a race condition occurred during the reallocation of devices from "old" device pools to "new" device pools which caused devices to become unavailable in both pools.
How we fixed it:
The "old" pools were shutdown releasing the device lock, allowing the "new" pools to acquire the lock.
What we are doing to prevent it from happening again:
We are improving our device allocation processes.
Resolved
After taking remedial action, Android Real Devices are now available in all our datacenters. This incident is resolved
Investigating
We are currently experiencing significantly reduced availability of Android Real Devices in all our datacenters. We are investigating.
This incident affected: Automated Real Device Testing (US-West, EU-Central, US-East) and Live Real Device Testing (US-West, EU-Central, US-East).