Partially Degraded Service
We are currently experiencing a slight reduction in available public and private iOS devices in the EU-Central Data Center. We expect to restore full availability for all iOS users in the EU-Central Data Center by tomorrow, Thursday, July 3, 2025, end of day
2022-May-25 Service Incident 2
Incident
Report for Sauce Labs
Postmortem
Dates:
Wednesday May 25th 2022, 07:49 - 09:51 UTC.
What happened:
Sauce Connect tunnels intermittently failed to start in our EU-Central-1 data center.
Why it happened:
Unusually high load caused our back-end systems to respond slower than usual.
How we fixed it:
We enabled a more strict rate-limiting to reduce the load caused by a small number of users. The system restored its normal behavior after the load was decreased.
What we are doing to prevent it from happening again:
- We are planning to improve our back-end system performance in EU-Central to be able to handle this workload.
- We are adding more tunnel capacity in our EU datacenter.
Resolved
This incident has been resolved.
Monitoring
Sauce Connect tunnels are now starting normally. We are continuing to monitor the situation.
Identified
The issue has been identified and we are taking steps to recover.
Investigating
Sauce Connect tunnels in our EU-Central data center are intermittently failing to start. We are investigating.
This incident affected: Sauce Connect (EU-Central).