Sauce Labs Maintenance Windows for Sauce Labs
We are aware of and working to resolve intermittent WiFi connectivity and slowness issues affecting Android Real Devices in our US-West-1 data center. During a maintenance window on September 25, we made changes to our wireless infrastructure which has greatly improved the situation, but we are still monitoring and working on further improvements. Please contact our Support team should you wish to report any such issues.

2021-Dec-01 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Friday December 1st 2021, 02:40 - 04:24 UTC

What happened:

Tests running on  iOS simulators, Android emulators, and desktop browser/OSs in our US-West and EU-Central data centers experienced high wait times and errors.

Why it happened:

Unforeseen issues in our deployment flow caused the cloud capacity to become unavailable. Safeguards were not in place to prevent this from rolling out to both the EU and US.

How we fixed it:

Services were restarted to restore capacity. Some Mac/iOS tests were terminated to expedite recovery in the US-West data center.

What we are doing to prevent it from happening again:

We are introducing safeguards and adding additional checks in our services to increase resiliency while at the same time working with our cloud providers to improve traffic flows.

Posted Dec 09, 2021 - 16:02 UTC

Resolved
iOS Simulators and MacOS Desktop tests have fully recovered in US Data Center. All services are fully operational.
Posted Dec 01, 2021 - 04:29 UTC
Investigating
Our US and EU Regions experienced high wait times and errors between 02:40 - 03:10 UTC and have since recovered.
iOS Simulators and MacOS Desktop tests are still experiencing high wait times in US Data Center. During this time, tests may fail to start. We are taking remedial action.
Posted Dec 01, 2021 - 03:46 UTC
This incident affected: Native Mobile App Testing (US-West), Live Virtual Mobile Device Testing (US-West), Automated Virtual Mobile Device Testing (US-West), Automated Browser Testing (US-West), and Live Browser Testing (US-West).