Sauce Labs Maintenance Windows for Sauce Labs
We are investigating an issue with a third-party provider that is causing intermittent "Connection timed out", "pool communication" and "Unknown error while proxying appium request" errors when running tests in our EU and US datacenters. We will be performing emergency maintenance to our EU-Central and US-West datacenters this week to address this issue. Emergency maintenance windows will be posted to this page.
2023-January-11 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Thursday January 11th 2024, 23:17 - Friday January 12th 2024, 01:09 UTC

What happened:

Tests running on Chrome version 120 for any platform and Firefox on Mac experienced failures.

Why it happened:

During a new browser release, a critical update to a dependency for Firefox was not identified due to an irregular and infrequent update cycle for that dependency.  Additionally, during the initial rollback, an inconsistent system state affected Chrome.

How we fixed it:

The deployment was reverted.  Following the initial revert, an additional deploy was performed to resolve an inconsistent system state.

What we are doing to prevent it from happening again:

We are improving our internal automated testing coverage to include the previously missed component.

Posted Jan 29, 2024 - 15:17 UTC

Resolved
Between 23:48 UTC and 01:09 UTC, we saw an elevated rate of Infrastructure Errors on our platform impacting browser tests on macOS and Windows in our US West Data Center. We have taken remedial action. All services are fully operational.
Posted Jan 12, 2024 - 01:20 UTC