Sauce Labs Maintenance Windows for Sauce Labs
We are investigating an issue with a third-party provider that is causing intermittent "Connection timed out", "pool communication" and "Unknown error while proxying appium request" errors when running tests in our EU and US datacenters. We will be performing emergency maintenance to our EU-Central and US-West datacenters this week to address this issue. Emergency maintenance windows will be posted to this page.
2022-October-27 Resolved Service Incident
Incident Report for Sauce Labs


Friday October 27th 2022, 09:15 - 09:45 UTC

What happened:

We observed higher than normal response times from our user service, which impacted all functionality that depends on it. The largest impact was elevated wait times, errors when attempting to start virtual device tests, and errors in the dashboard and test results in our US West data center. 

Why it happened:

Over the past year, we have migrated most of the services responsible for test orchestration to Google Cloud. One of these services is our user service, which relies on a data store with a component still configured to run in our on-premise data center. 

While cleaning up resources for our on-premise deployments, we removed a configuration that pointed the user service to an on-premise component that increased latency when accessing Forgerock DS. 

How we fixed it:

Since this component was no longer needed, we removed the configuration and the on-premise resources. Then we restarted the service, and the latency went away as everything was running in Google Cloud. 

What we are doing to prevent it from happening again:

Now that all components are cleaned up, we are safe from this reoccurring.

Posted Dec 20, 2022 - 14:09 UTC

There were elevated wait times and errors when attempting to start virtual device tests, as well as errors in the dashboard and test results in our US-West-1 data center between 9:06 and 9:47 UTC. This is now resolved and wait times and error rates are back to normal levels.
Posted Oct 27, 2022 - 09:50 UTC