Maintenance Windows for Sauce Labs
Customers may experience intermittent errors during automated browser and virtual mobile device tests in our US-West-1 datacenter. We are closely monitoring and investigating the affected services.
2024-February-20 Resolved Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Monday February 19th 2024, 11:20 UTC - Tuesday February 20th, 10:15 UTC
What happened:
Some Emulator and Simulator sessions with typically long startup times were unable to start.
Why it happened:
A service was updated to improve its performance. This service handles waiting additional time for a VM to become available. Its internal logic was modified to process a single wait request, but sometimes additional wait requests are necessary.
How we fixed it:
The deployment was reverted.
What we are doing to prevent it from happening again:
We are investigating simulating slower startup times to ensure we can effectively identify similar regressions in the future.
Resolved
Between 2024-Feb-19 11:20 UTC and 2024-Feb-20 10:20 UTC, error rates for desktop and virtual mobile device tests were elevated in our US-West-1 and EU-Central-1 data centers. We have taken remedial action. All services are fully operational.