Sauce Labs Maintenance Windows for Sauce Labs
Customers may experience intermittent errors during automated browser and virtual mobile device tests in our US-West-1 datacenter. We are closely monitoring and investigating the affected services.
2021-November-09 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Tuesday November 9th 2021, 17:35 - 21:15 UTC

What happened:

Automated and Live browser tests as well as virtual and real mobile device tests were failing to start in our EU-Central-1 data center. 

Why it happened:

Due to an invalid deployment state of one of our services, two services in our EU-Central-1 data center were unable to communicate with each other properly. 

How we fixed it:

The affected service was redeployed.

What we are doing to prevent it from happening again:

We will be adding additional logging and monitoring to our services.

Posted Nov 18, 2021 - 11:33 UTC

Resolved
We have taken remedial action. Tests in EU-Central data center are passing as intended. All services are fully operational.
Posted Nov 09, 2021 - 22:01 UTC
Monitoring
We've taken remedial action and have seen testing is back to normal. We are monitoring.
Posted Nov 09, 2021 - 21:25 UTC
Update
We are still experiencing elevated failure rate in our EU-Central data center affecting Automated and Live browser and Mobile testing. We continue to investigate
Posted Nov 09, 2021 - 20:58 UTC
Investigating
We are seeing an elevated failure rate in our EU-Central data center when starting Automated and Live browser tests as well as virtual and real mobile device tests. We are investigating.
Posted Nov 09, 2021 - 19:33 UTC
This incident affected: Automated Virtual Mobile Device Testing (EU-Central), Automated Real Device Testing (EU-Central), Live Browser Testing (EU-Central), Live Virtual Mobile Device Testing (EU-Central), Live Real Device Testing (EU-Central), and Automated Browser Testing (EU-Central).