Sauce Labs Maintenance Windows for Sauce Labs
We are investigating an issue with a third-party provider that is causing intermittent "Connection timed out", "pool communication" and "Unknown error while proxying appium request" errors when running tests in our EU and US datacenters. We will be performing emergency maintenance to our EU-Central and US-West datacenters this week to address this issue. Emergency maintenance windows will be posted to this page.
2023-October-20 Service Incident
Incident Report for Sauce Labs
Postmortem

Dates:

Friday October 20th 2023, 09:19 - 13:36 UTC

What happened:

For the duration of the incident, customers experienced errors when starting Live and Automated iOS Simulator sessions targeting “latest”, “current_major”, and “previous_major” simulator aliases in all regions. 

Why it happened:

In order to bring early availability of iOS 17 testing to our customers, we deployed a release candidate of the iOS 17.0 Simulator. We released it with an alias of “rc” to avoid mapping it to the “current_major” alias so customers wouldn’t accidentally start using it. This new mapping caused an issue with the service that orchestrates tests in our system. Due to this issue, customers running simulator tests with an alias had issues running their tests. Since most customers target specific iOS Simulator versions, there wasn’t a widespread impact. 

How we fixed it:

The team rolled back the iOS 17.0.rc Simulator release, which resolved the issue.

What we are doing to prevent it from happening again:

We are working on a standard way to deploy iOS Simulator release candidates.

Posted Nov 20, 2023 - 16:30 UTC
Resolved
We have resolved the issue which caused errors when running Live and Automated iOS Simulator tests in our US-West-1 and EU-Central-1 datacenters. All services are fully operational.
Posted Oct 20, 2023 - 13:36 UTC
Identified
We are continuing to see errors starting Live & Automated iOS Simulator sessions in all datacenters. We have identified the root cause and are taking remedial action.
Posted Oct 20, 2023 - 12:52 UTC
Update
We are currently seeing errors starting Live & Automated iOS Simulator sessions in all datacenters. We are still investigating.
Posted Oct 20, 2023 - 11:32 UTC
Update
We are currently seeing errors starting Live & Automated iOS Simulator sessions in all datacenters. We are investigating.
Posted Oct 20, 2023 - 10:28 UTC
Investigating
We are currently seeing errors starting Live iOS Simulator sessions in all datacenters. We are investigating.
Posted Oct 20, 2023 - 09:58 UTC
This incident affected: Automated Virtual Mobile Device Testing (US-West, EU-Central) and Live Virtual Mobile Device Testing (US-West, EU-Central).
← Current Status Powered by Atlassian Statuspage