Maintenance Windows for Sauce Labs
We are currently performing maintenance that will affect the initiation of new Sauce Connect tunnel instances. The maintenance is expected to be completed by 2:00 PM PST. We appreciate your understanding and patience as we work to enhance our service.
2022-October-19 Resolved Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Wednesday October 19th 2022, 11:45 - 13:10 UTC
What happened:
All iOS devices lost internet connection after a firewall rule was removed.
Why it happened:
iOS devices lost internet connection after a firewall rule was applied that prevented public access to https://app.testobject.com. Our iOS devices have a proxy configured that points to this domain for downloading a “Proxy auto-config” (PAC), and since that failed they lost internet connection.
How we fixed it:
We restored the firewall rule, and connectivity was re-established.
What we are doing to prevent it from happening again:
The team is investigating ways to extract the parts of the system that need internet connectivity. For example, storing the PAC file in publicly accessible cloud storage instead of hosted under https://app.testobject.com.
Resolved
Between 12:45 UTC and 13:15 UTC We experienced decreased availability and internet connectivity in iOS real devices in both US-West-1 & EU-Central-1 datacenters. This issue is now resolved, all services are fully operational.