Maintenance Windows for Sauce Labs
We are aware of and working to resolve intermittent WiFi connectivity and slowness issues affecting Android Real Devices in our US-West-1 data center. During a maintenance window on September 25, we made changes to our wireless infrastructure which has greatly improved the situation, but we are still monitoring and working on further improvements. Please contact our Support team should you wish to report any such issues.
2021-September-22 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Wednesday September 22nd 2021, 10:40 - 13:30 UTC
What happened:
Some Android devices on the public cloud in the US-West-1 data center were not available for running tests.
Why it happened:
A number of devices had lost their connection to the internet, resulting in the devices being automatically disabled. Our automated cleaning and setup process was not successful in recovering and restoring internet connectivity.
How we fixed it:
We improved the logic that allows the system to reset the internet connection for devices.
What we are doing to prevent it from happening again:
We added additional monitoring steps to give us better data about when and how devices lose their connection to the internet, while also improving the wifi configuration inside of our data centers.
Resolved
After making changes to resolve the issue, device availability is back to normal levels. This incident is resolved.
Investigating
We are seeing an elevated proportion of unavailable Real Devices in our US-West-1 data center due to wifi issues. Customers may also be experiencing connectivity issues whilst running tests on these devices. We are investigating.
This incident affected: Live Real Device Testing (US-West) and Automated Real Device Testing (US-West).