Sauce Labs Maintenance Windows for Sauce Labs
Customers may experience intermittent errors during automated browser and virtual mobile device tests in our US-West-1 datacenter. We are closely monitoring and investigating the affected services.
2021-September-13 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Monday September 13th 2021, 06:00 - 07:03 UTC

What happened:

Services in our US-West-1 data center, including Sauce Connect tunnels, the Sauce Labs REST API, and the Sauce Labs dashboard were not starting.

Why it happened:

An update to the database used to establish connections by one of our microservices had not been fully applied. 

How we fixed it:

The credentials used to establish a database connection were reapplied.

What we are doing to prevent it from happening again:

Monitoring mechanisms for the affected database were disabled. API monitoring tests will now be run more frequently so that we can detect a similar issue in the future with less delay.

Posted Sep 21, 2021 - 10:18 UTC

Resolved
This incident has been resolved, all services are fully operational.
Posted Sep 13, 2021 - 07:39 UTC
Identified
We have identified the issue and taken action. Services are beginning to recover but we are still monitoring the issue.
Posted Sep 13, 2021 - 07:17 UTC
Update
We are continuing to investigate this issue.
Posted Sep 13, 2021 - 06:55 UTC
Investigating
Automated Browser, Virtual Mobile Device and Real Device tests are not starting. In addition, Sauce Connect Tunnels are not starting, the Sauce Labs REST API is not responding, and the Sauce Labs dashboard is not loading, all in our US-West-1 datacenter.
Posted Sep 13, 2021 - 06:53 UTC
This incident affected: Automated Browser Testing (US-West), Automated Virtual Mobile Device Testing (US-West), Automated Real Device Testing (US-West), Live Browser Testing (US-West), Live Virtual Mobile Device Testing (US-West), Live Real Device Testing (US-West), Sauce Connect (US-West), Sauce Labs REST API (US-West), and Sauce Labs Dashboard (US-West).