Sauce Labs Maintenance Windows for Sauce Labs
Sauce Labs has consolidated onto one status page for all data centers.

The old status pages for US-West, US-East and EU-Central are still available to be able to view incident history. Anyone who was subscribed to notifications from one of the old status pages has been migrated onto this consolidated page. If you received an email saying that you had been unsubscribed from notifications, that was from one of the old pages and can be disregarded.

If you would like to select which components that you receive notifications for (e.g. if you are only interested in receiving notifications about incidents affecting Real Device Testing), you can access the subscription management page by clicking "Subscribe To Updates" above and enter the email address you are subscribed with. Additionally there is a "Manage Your Subscription" link on any email you will receive from the status page going forward.
2021-March-8 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Monday, March 8, 18:39 - 23:30 UTC

What happened:

We were seeing elevated wait times for automated and manual MacOS tests in our US-West-1 data center. Customers may have seen MacOS and iOS Simulator tests taking a long time to start, or failing to start. Windows, Linux and Android tests were unaffected.

Why it happened:

Due to high sustained peak usage combined with intensive CPU operations from a customer application our Mac Cloud CPU performance became degraded.

How we fixed it:

Additional data center caching servers were deployed to improve performance and distribution of application storage.  We also implemented throttling of targeted jobs to reduce the CPU load on our US-West Mac Cloud.

What we are doing to prevent it from happening again:

In the future more aggressive throttling will be implemented on a per customer basis to keep the performance of our services within established baselines.  A Sev1 issue will be opened with affected customers to work with them to resolve the issue specific to their usage.

Posted Mar 16, 2021 - 11:11 UTC

Resolved
This incident has been resolved. All services are fully operational.
Posted Mar 09, 2021 - 13:25 UTC
Monitoring
Wait times have returned to normal levels. We are continuing to monitor the situation.
Posted Mar 09, 2021 - 11:55 UTC
Update
Wait times continue to be elevated for automated and manual MacOS and iOS Simulator tests in our US-West-1 Data Center. We are still investigating.
Posted Mar 09, 2021 - 09:15 UTC
Update
We are seeing elevated wait times for automated and manual MacOS tests, including iOS Simulators, in the US-West Data Centre only. Customers may see tests taking a long time to start, or failing to start. Windows, Linux and Android tests are unaffected. We are investigating.
Posted Mar 09, 2021 - 07:16 UTC
Investigating
We are seeing elevated wait times for automated and manual MacOS tests, including iOS Simulators. Customers may see tests taking a long time to start, or failing to start. Windows, Linux and Android tests are unaffected. We are investigating.
Posted Mar 09, 2021 - 07:15 UTC
This incident affected: Automated Virtual Mobile Device Testing (US-West), Live Virtual Mobile Device Testing (US-West), and Automated Browser Testing (US-West).