Sauce Labs Partially Degraded Service
We are currently experiencing a slight reduction in available public and private iOS devices in the EU-Central Data Center. We expect to restore full availability for all iOS users in the EU-Central Data Center by tomorrow, Thursday, July 3, 2025, end of day

2021-September-19 Service Incident

Incident Report for Sauce Labs

Postmortem

Dates:

Sunday September 19th 2021, 15:42 - 16:35 UTC

What happened:

Automated and Live Mac and iOS tests in our EU-Central-1 data center were failing to start.

Why it happened:

We experienced a combination of router software failure caused by a third-party bug as well as with a failure to properly route traffic. 

How we fixed it:

The issue was resolved by initiating a manual failover of traffic to the secondary circuit.

What we are doing to prevent it from happening again:

We are investigating potential options to avoid the failure to automatically failover traffic routing in the future. Additionally, we are removing a configuration from our edge routers to ensure we do not hit this particular software bug again.

Posted Sep 28, 2021 - 10:04 UTC

Resolved

We have taken remedial measures and all Mac testing is back and running. All services are fully operational.
Posted Sep 19, 2021 - 16:47 UTC

Investigating

Mac tests (Live and Automated) are not starting in our EU data center. We are investigating.
Posted Sep 19, 2021 - 16:14 UTC
This incident affected: Automated Browser Testing (US-West) and Live Browser Testing (US-West).