Sauce Labs Maintenance Windows for Sauce Labs
Sauce Labs will have a four hour previously scheduled maintenance window on Wednesday, July 10th, starting at 04:00 UTC and ending at 08:00 UTC. During this maintenance window, we will be making updates to our infrastructure and services in our US West data center. These maintenance actions may cause portions of the service (including running automated and manual tests) to be unavailable for up to four hours.
2023-September-7 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Wednesday September 6th 2023, 18:35 UTC - Thursday September 7th 19:05 UTC

What happened:

A subset of customers were unable to run tests with specific uploaded apps in the EU region intermittently for the duration of the incident. 

Why it happened:

During migrating users from regional to global teams, several organizations in the EU had their users moved to the default team, meaning they were unable to run tests using apps uploaded by another team. Various scripts were created to prepare and execute this migration, one of which was meant to determine a user’s region. This script had a bug in it and misconfigured 55 organizations.

How we fixed it:

We were able to restore the user configuration to restore service, and then we addressed the bug in the script and completed the migration. 

What we are doing to prevent it from happening again:

We have addressed bugs in the migration pipeline and the scripts it leverages for future runs.

Posted Sep 13, 2023 - 10:30 UTC
Resolved
After taking remedial action, impacted subset of user accounts are restored to correct teams. All services are fully operational.
Posted Sep 07, 2023 - 19:06 UTC
Identified
Sauce Labs accounts assigned to custom teams were moved to the default team for a small subset of customers. We have identified the root cause and are taking remedial action.
Posted Sep 07, 2023 - 15:03 UTC
← Current Status Powered by Atlassian Statuspage