Partially Degraded Service
We are currently experiencing a slight reduction in available public and private iOS devices in the EU-Central Data Center. We expect to restore full availability for all iOS users in the EU-Central Data Center by tomorrow, Thursday, July 3, 2025, end of day
2023-April-29 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Saturday April 29th 2023, 07:03 - 09:06 UTC
What happened:
Customers experienced elevated wait times and error rates for tests running on macOS and iOS simulators for the duration of the incident.
Why it happened:
A third-party automated upgrade erroneously halted services supporting service discovery for one of our data centers. Without service discovery, VMs were not able to be restarted.
How we fixed it:
We restarted the service discovery services, and normal operation resumed.
What we are doing to prevent it from happening again:
We are preventing live upgrades of the service in question, moving it to our regular release workflow.
Resolved
After taking remedial action, MacOS Desktop and iOS Virtual mobile device test wait times and error rates in our US-West-1 Data center have returned to normal levels. This issue is resolved.
Monitoring
After taking remedial action, MacOS Desktop and iOS Virtual mobile device tests wait times and error rates in our US-West-1 Data center are returning to normal levels. We are currently monitoring.
Investigating
MacOS Desktop and iOS Virtual mobile device tests are seeing long wait times and a high rate of errors up in our US-West-1 Data center. We are investigating.
This incident affected: Automated Browser Testing (US-West), Automated Virtual Mobile Device Testing (US-West), Live Browser Testing (US-West), and Live Virtual Mobile Device Testing (US-West).