Sauce Labs Maintenance Windows for Sauce Labs
Sauce Labs will have a four hour previously scheduled maintenance window on Wednesday, July 10th, starting at 04:00 UTC and ending at 08:00 UTC. During this maintenance window, we will be making updates to our infrastructure and services in our US West data center. These maintenance actions may cause portions of the service (including running automated and manual tests) to be unavailable for up to four hours.
2024-August-12 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Saturday August 10th 2024, 19:00 - Monday August 12th 2024, 21:13 UTC

What happened:

Following scheduled network upgrades, connectivity between Real Devices and our IPsec VMs was disrupted.  Additionally, a misconfiguration in Sauce Connect caused iOS Real Device Live Testing and app installations to fail.

Why it happened:

During the network upgrade, a community tag was missing from a subset of production routes causing connectivity issues between IPsec VMs and Real Devices.  Additionally, a misconfiguration in Sauce Connect caused iOS Live Testing connectivity issues.

How we fixed it:

To resolve the IPsec issue, we added the missing tags.  To resolve the Sauce Connect with iOS Real Devices issue, we rolled back the Sauce Connect service configuration.

What we are doing to prevent it from happening again:

We’re adding additional end to end testing and alerting for our IPsec and Sauce Connect on iOS products.

Posted Sep 23, 2024 - 10:11 UTC
Resolved
After taking remedial action, all services are operating as normal. This incident is resolved.
Posted Aug 12, 2024 - 22:44 UTC
Investigating
We are currently seeing real device iOS installation failures, showing "Unable to Verify App" in the US-West Data Center. We are investigating.
Posted Aug 12, 2024 - 21:13 UTC
This incident affected: Automated Real Device Testing (US-West) and Live Real Device Testing (US-West).
← Current Status Powered by Atlassian Statuspage