Maintenance Windows for Sauce Labs
We are experiencing cellular connectivity issues affecting 10% of private RDC devices in our US datacenter. A carrier technician is currently on site investigating the issue.
2023-April-26 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Tuesday April 25th 2023, 09:38 - Wednesday April 26th, 15:04 UTC
What happened:
For the duration of the incident, freemium users could not run live tests on public real devices. The issue did not spread to automated tests.
Why it happened:
The incident was caused by a deployment that introduced a bug that changed how devices were filtered to freemium users.
How we fixed it:
Since the deployment included other changes and the issue affected only live freemium users, extra care was taken when rolling back to an earlier version, which did not include the faulty change.
What we are doing to prevent it from happening again:
Expanding our monitoring and test coverage for freemium users is underway and will help prevent and detect this issue going forward.
Resolved
After taking remedial action, device availability is back to normal in both US and EU datacenters. All services are fully operational.
Update
We are continuing to investigate this issue.
Investigating
Real Devices are not available during Live Testing in our US-West-1 and EU-Central-1 datacenters for users on free plan.
We are investigating.
We are investigating.
This incident affected: Live Real Device Testing (US-West, EU-Central).