Maintenance Windows for Sauce Labs
Sauce Labs will have a four hour previously scheduled maintenance window on Wednesday, July 10th, starting at 04:00 UTC and ending at 08:00 UTC. During this maintenance window, we will be making updates to our infrastructure and services in our US West data center. These maintenance actions may cause portions of the service (including running automated and manual tests) to be unavailable for up to four hours.
2023-January-11 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Thursday January 11th 2024, 23:17 - Friday January 12th 2024, 01:09 UTC
What happened:
Tests running on Chrome version 120 for any platform and Firefox on Mac experienced failures.
Why it happened:
During a new browser release, a critical update to a dependency for Firefox was not identified due to an irregular and infrequent update cycle for that dependency. Additionally, during the initial rollback, an inconsistent system state affected Chrome.
How we fixed it:
The deployment was reverted. Following the initial revert, an additional deploy was performed to resolve an inconsistent system state.
What we are doing to prevent it from happening again:
We are improving our internal automated testing coverage to include the previously missed component.
Resolved
Between 23:48 UTC and 01:09 UTC, we saw an elevated rate of Infrastructure Errors on our platform impacting browser tests on macOS and Windows in our US West Data Center. We have taken remedial action. All services are fully operational.