Sauce Labs Maintenance Windows for Sauce Labs
We are investigating an issue with a third-party provider that is causing intermittent "Connection timed out", "pool communication" and "Unknown error while proxying appium request" errors when running tests in our EU and US datacenters. We will be performing emergency maintenance to our EU-Central and US-West datacenters this week to address this issue. Emergency maintenance windows will be posted to this page.
2023-September-28 Service Incident 2
Incident Report for Sauce Labs
Postmortem

Dates:

Thursday September 28th 2023, 21:10 UTC - Friday September 29th 2023, 00:43 UTC

What happened:

Users attempting to use live test sessions in the Sauce Labs UI were either given an error or a blank screen for the duration of the incident.

Why it happened:

When attempting to insert a non-serializable value into our memory store, an error was thrown, preventing actions such as starting a live session. This error began occurring after a middleware was added to our memory store that attempted to serialize any payload, which, upon failing to do so, would throw an error preventing that action from being committed.

How we fixed it:

We rolled back to the previous version of live testing to restore service. 

What we are doing to prevent it from happening again:

We are adding a blocking stage to our pipeline to check for these issues by running live sessions for both real devices and web sessions.

Posted Oct 10, 2023 - 13:29 UTC
Resolved
We have identified and resolved the issue that caused Live sessions on Real Device Cloud to fail to start in US and EU datacenters. All services are fully operational.
Posted Sep 29, 2023 - 00:45 UTC
Update
We continue investigating the issue with running Real Device app and browser tests in US West and EU datacenters. Cross-browser sessions are running in both US and EU datacenters.
Posted Sep 28, 2023 - 23:48 UTC
Investigating
We are seeing an issue with running Live Sessions in US West and EU datacenters. Real device and Web browser Live tests fail to start. We are investigating.
Posted Sep 28, 2023 - 23:00 UTC
This incident affected: Live Browser Testing (US-West, EU-Central) and Live Real Device Testing (US-West, EU-Central).
← Current Status Powered by Atlassian Statuspage