Partially Degraded Service
We are currently experiencing a slight reduction in available public and private iOS devices in the EU-Central Data Center. We expect to restore full availability for all iOS users in the EU-Central Data Center by tomorrow, Thursday, July 3, 2025, end of day
2021-Dec-02 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Thursday December 2nd 2021, 2:00 - 4:30 UTC
What happened:
Android devices in the US-West-1 data center failed during regularly recurring health checks and became disabled.
Why it happened:
Two services, one responsible for running health checks on our real devices, the other responsible for installing Android applications were installed on the same VM. The installation service has a disk cache that used most of the disk space and impacted the functionality of the service running the health checks. The devices that were checked failed the health check and they were disabled for customer tests.
How we fixed it:
The services were restarted.
What we are doing to prevent it from happening again:
The two services were separated into different VMs. Also the time-to-live of the files in the installation cache was reduced.
Resolved
This incident has been resolved.
Investigating
We are seeing an elevated number of unavailable real devices in our US-West data center. We are investigating.
This incident affected: Automated Real Device Testing (US-West) and Live Real Device Testing (US-West).