Maintenance Window
We are currently experiencing a slight reduction in available public and private iOS devices in the EU-Central Data Center. We expect to restore full availability for all iOS users in the EU-Central Data Center by tomorrow, Thursday, July 3, 2025, end of day
2025-January-21 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Tuesday January 21st 2025, 12:40 - 13:44 UTC
What happened:
Tests using iOS Real Devices experienced failures to download and install apps for the eu-central01 and us-west-1 datacenters.
Why it happened:
A misconfiguration with the resigner service caused errors when communicating with the app storage.
How we fixed it:
A rollback was executed, restoring configuration to the previous working state.
What we are doing to prevent it from happening again:
- Improve canary deployments for each Real Device Cloud region.
- Improve end to end testing for canary deployments.
- Improve SLOs for the Real Device application installs
Resolved
After taking remedial action, apps are now installing correctly in all datacenters. This incident is resolved.
Investigating
We are currently seeing App installation errors when trying to run iOS tests on our US-West-1 & EU-Central-1 Datacenter. We are investigating
This incident affected: Automated Virtual Mobile Device Testing (US-West, EU-Central), Automated Real Device Testing (US-West, EU-Central), Live Virtual Mobile Device Testing (US-West, EU-Central), Live Real Device Testing (US-West, EU-Central), and Native Framework Mobile App Testing (US-West, EU-Central).