Maintenance Windows for Sauce Labs
We are currently performing maintenance that will affect the initiation of new Sauce Connect tunnel instances. The maintenance is expected to be completed by 2:00 PM PST. We appreciate your understanding and patience as we work to enhance our service.
2024-March-27 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Wednesday March 27th 2024, 10:20 - 11:55 UTC
What happened:
Availability of real devices running iOS 14.x - 16.x was decreased, which may have resulted in an inability to obtain a device during periods of increased usage.
Why it happened:
A third-party library was updated. This update introduced a minor bug that exposed another non-recoverable bug, resulting in specific versions of iOS appearing offline in our system.
How we fixed it:
We rolled back the library update.
What we are doing to prevent it from happening again:
We are fixing the triggered defect in our code as well as improving the detection of offline devices during deployment of the affected software.
Resolved
After taking remedial action, iOS Real devices are available in all our data centers. This incident is now resolved.
Investigating
We are experiencing reduced availability of iOS Real devices older than iOS 17 in all our data centers. We are investigating.
This incident affected: Automated Real Device Testing (US-West, EU-Central, US-East) and Live Real Device Testing (US-West, EU-Central, US-East).