Sauce Labs Maintenance Windows for Sauce Labs
We are aware of and working to resolve intermittent WiFi connectivity and slowness issues affecting Android Real Devices in our US-West-1 data center. During a maintenance window on September 25, we made changes to our wireless infrastructure which has greatly improved the situation, but we are still monitoring and working on further improvements. Please contact our Support team should you wish to report any such issues.

2021-September-30 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Thursday September 30th 2021, 18:00 - Friday October 1st 2021, 01:00 UTC

What happened:

In Visual Testing (Screener), component snapshots were being captured with the “error: certificate has expired” showing on screen. All component tests that were run using ngrok tunnels were affected.

Why it happened:

Screener has a cache layer to avoid downloading resources on every component request. This cache layer has some issues related to an expired third-party certificate. 

How we fixed it:

To mitigate the issue the cache layer was removed as fast as possible. After the issue was later addressed, the cache layer was reinstated the next day on Oct 1st.

What we are doing to prevent it from happening again:

We’re adding additional monitoring checks to detect any issues with the caching layer.

Posted Oct 04, 2021 - 12:29 UTC

Resolved
We have taken remedial action and have seen tests pass as intended. All services are fully operational.
Posted Oct 01, 2021 - 01:04 UTC
Update
We continue to investigate the certificate errors and mitigate its impact.
Posted Sep 30, 2021 - 23:27 UTC
Investigating
Screener tests using ngrok might get an invalid certificate error when running. We are investigating.
Posted Sep 30, 2021 - 22:25 UTC
This incident affected: Visual Testing Hub and Visual Testing Infrastructure.