Sauce Labs Maintenance Windows for Sauce Labs
Customers may experience intermittent errors during automated browser and virtual mobile device tests in our US-West-1 datacenter. We are closely monitoring and investigating the affected services.
2021-November-15 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Monday November 15th 2021, 20:17 - 22:30 UTC

What happened:

Automated and Live browser tests as well as virtual and real mobile device tests were failing to start in our EU-Central-1 data center. 

Why it happened:

An upstream provider pushed a change to a cluster that disabled a feature that enabled critical internal communication flows. 

How we fixed it:

A workaround was implemented that bypassed the feature that was removed.

What we are doing to prevent it from happening again:

We are working with the provider to prevent future removals of the affected service.

Posted Nov 19, 2021 - 18:23 UTC

Resolved
We have taken remedial action. Tests in the EU-central data center are passing as intended. All services are fully operational.
Posted Nov 15, 2021 - 22:59 UTC
Update
We are still experiencing failures starting tests in our EU-Central data center affecting Automated and Live browser and Virtual Mobile device testing. We continue to investigate
Posted Nov 15, 2021 - 22:17 UTC
Investigating
We are seeing failures in our EU-Central data center when starting Automated and Live browser as well as virtual mobile device tests. We are investigating.
Posted Nov 15, 2021 - 21:01 UTC
This incident affected: Automated Virtual Mobile Device Testing (EU-Central), Live Browser Testing (EU-Central), Live Virtual Mobile Device Testing (EU-Central), and Automated Browser Testing (EU-Central).