Maintenance Windows for Sauce Labs
Sauce Labs will have a four hour previously scheduled maintenance window on Wednesday, July 10th, starting at 04:00 UTC and ending at 08:00 UTC. During this maintenance window, we will be making updates to our infrastructure and services in our US West data center. These maintenance actions may cause portions of the service (including running automated and manual tests) to be unavailable for up to four hours.
2023-June-26 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Monday June 26th 2023, 12:43 - 15:04 UTC
What happened:
In the EU region, results for tests running on Real Devices were not captured for the duration of the incident. This resulted in these results not being available in the front-end “Test Results” section of the Sauce Labs’ UI.
Why it happened:
The cause of the test results not being captured was an incomplete dataset in our database which was caused by a bug in how data is consumed from an internal event stream.
How we fixed it:
To restore service, we addressed and deployed a fix for the bug. We were also able to reprocess missed events, which restored all test results in the Test Results UI.
What we are doing to prevent it from happening again:
We are extending our testing around this area to better catch these types of issues moving forward.
Resolved
After taking remedial action, Live and Automated Real Device test results are displayed on the EU-Central-1 Sauce Labs UI. The issue is now resolved.
Identified
We have identified and fixed the root cause, and Real Device test results are recovering on our EU-Central-1 Sauce Labs UI. We are monitoring and working on restoring the missing data.
Investigating
Recent Live and Automated Real Device tests are not displayed on the EU-Central-1 Sauce Labs UI. We are investigating.
This incident affected: Sauce Labs Dashboard (EU-Central).