Maintenance Windows for Sauce Labs
We are aware of and working to resolve intermittent WiFi connectivity and slowness issues affecting Android Real Devices in our US-West-1 data center. During a maintenance window on September 25, we made changes to our wireless infrastructure which has greatly improved the situation, but we are still monitoring and working on further improvements. Please contact our Support team should you wish to report any such issues.
2021-September-20 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Monday September 20th 2021, 11:30 - 19:50 UTC
What happened:
Sauce Labs UI pages and Real Device testing were not fully functional in our US-West-1 instance.
Why it happened:
Due to an increased amount of test report data for real devices, one of our long running queries caused the database to become unresponsive, resulting in real device testing and the Sauce Labs UI not being fully functional.
How we fixed it:
We improved the efficiency of the database query for large amounts of data.
What we are doing to prevent it from happening again:
We are improving our data storage limitations and optimizing the database setup, as well as extending our monitoring capabilities.
Resolved
We have identified the issue and deployed a fix. Real Device tests are working as expected.
Identified
We have identified the issue and taken corrective action. Live and Automated Real Device tests are now running correctly but we are still experiencing issues displaying Real Device test history. We continue to work on the issue.
Investigating
Live and Automated Real Device Testing and the Sauce Labs dashboard in our US-West-1 data center are running at a degraded level of performance, and errors are intermittently being experienced. We are investigating. This is a continuation of the issue experienced earlier today.
This incident affected: Sauce Labs UI (US-West), Live Real Device Testing (US-West), and Automated Real Device Testing (US-West).