Maintenance Windows for Sauce Labs
We are experiencing cellular connectivity issues affecting 10% of private RDC devices in our US datacenter. A carrier technician is currently on site investigating the issue.
2023-April-11 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Tuesday April 11th 2023, 09:09 - 10:05 UTC
What happened:
During the incident, any requests to start Appium sessions on Android devices requiring chromedriver version 112 or higher failed.
Why it happened:
This was caused due to an outdated version of chromedriver in our real device pool. The process that is responsible for keeping chromedriver up to date failed due to a bug. The bug also caused the alert mechanism to fail, causing a delay in our response.
How we fixed it:
We deployed a new version of the real device pool with the latest chromedriver versions.
What we are doing to prevent it from happening again:
We are addressing the bug in the chromedriver update process, which caused chromedriver to be out of date and delayed our response.
Resolved
After taking remedial action, all Android Real Device sessions are starting as expected on both US-West-1 and EU-Central-1 data centers. This issue is now resolved, all services are fully operational.
Investigating
Automated Android real device tests are intermittently failing to launch in our US-West-1 and EU-Central-1 data centers. We are actively investigating.
This incident affected: Automated Real Device Testing (US-West, EU-Central).