Sauce Labs Maintenance Windows for Sauce Labs
Sauce Labs will have a four hour previously scheduled maintenance window on Wednesday, July 10th, starting at 04:00 UTC and ending at 08:00 UTC. During this maintenance window, we will be making updates to our infrastructure and services in our US West data center. These maintenance actions may cause portions of the service (including running automated and manual tests) to be unavailable for up to four hours.
2023-September-20 Resolved Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Wednesday September 20th 2023, 09:39 - 15:38 UTC

What happened:

Sauce Connect users could not load their websites behind Sauce Connect tunnels until they closed a pop-up informing them, "Your connection is not private”.

Why it happened:

Windows virtual machines did not import the certificates needed to run tunnels properly due to a change in how they were stored on the virtual machine. A startup script references several locations to load certificates from, and not all of them were updated to reflect the new location on the virtual machine. 

How we fixed it:

The startup script was updated to reflect the new certificate locations. 

What we are doing to prevent it from happening again:

We reviewed the tunnel startup code across all platforms to ensure no other areas referenced the old locations. We also added unit tests that check whether there is a mismatch between where the certificates are stored and their references in the startup script.

Posted Sep 28, 2023 - 15:06 UTC

Resolved
Between 10:39 UTC and 15:38 UTC our desktop tests that use Sauce Connect certificate resign feature experienced failures due to invalid certificates on US-West-1 and EU-Central-1 datacenters . We have identified the issue and taken remedial action.
Posted Sep 20, 2023 - 17:59 UTC