Partially Degraded Service
We are currently experiencing a slight reduction in available public and private iOS devices in the EU-Central Data Center. We expect to restore full availability for all iOS users in the EU-Central Data Center by tomorrow, Thursday, July 3, 2025, end of day
2023-November-28 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Monday November 27 2023, 20:00 - Tuesday November 28 2023, 15:36 UTC
What happened:
Pixel devices in Europe failed to recover from a specific test scenario, resulting in decreased availability of Pixel devices over time.
Why it happened:
A combination of defects resulted in devices failing to be properly cleaned following a specific customer test scenario involving audio capture.
How we fixed it:
Multiple software defects were fixed and deployed.
What we are doing to prevent it from happening again:
The defects were resolved. Additionally, new observability work is under investigation to improve detection time for similar availability situations in the future.
Resolved
After taking remedial action, all Google Pixel devices are now available in the EU-Central-1 data center.
Identified
Public Google Pixel devices continue to be unavailable in the EU-Central-1 data center. We have identified the root cause and are taking remedial action.
Investigating
Public Google Pixel devices are currently unavailable in the EU-Central-1 data center. We are investigating.
This incident affected: Automated Real Device Testing (EU-Central) and Live Real Device Testing (EU-Central).