2024-April-14 Service Incident

Incident Report for Sauce Labs

Postmortem

Dates:

Sunday April 14th 2024, 17:05 - 18:40 UTC

What happened:

Customers were unable to access the Web UI or API for Sauce Labs services.

Why it happened:

Our internal and external API Gateways were unable to process traffic.

How we fixed it:

We restarted the API Gateway service.

What we are doing to prevent it from happening again:

We are working to improve our runbooks for similar issues in the future.

Posted Apr 24, 2024 - 02:57 UTC

Resolved

After taking remedial action, services in all datacenters are operating as normal. This incident is resolved.

Posted Apr 14, 2024 - 18:46 UTC

Investigating

We are seeing issues accessing and running tests on the Sauce Labs platform in all datacenters, this is affecting all services. We are currently investigating.

Posted Apr 14, 2024 - 17:55 UTC

This incident affected: Automated Browser Testing (US-West, EU-Central), Automated Virtual Mobile Device Testing (US-West, EU-Central), Automated Real Device Testing (US-West, EU-Central, US-East), Live Browser Testing (US-West, EU-Central), Live Virtual Mobile Device Testing (US-West, EU-Central), Live Real Device Testing (US-West, EU-Central, US-East), API Testing (US-West, EU-Central), Sauce Connect (US-West, EU-Central), Visual Testing (Visual Testing Hub, Visual Testing Infrastructure, Visual Testing REST API), Sauce Labs REST API (US-West, EU-Central), Sauce Labs Dashboard (US-West, EU-Central), Insights (US-West, EU-Central), Billing (US-West, EU-Central), Other (saucelabs.com, Sauce Labs Documentation), IPSec VPN (US-West, EU-Central), Native Framework Mobile App Testing (US-West, EU-Central), Mobile App Distribution (Mobile App Distribution Platform, Mobile App Distribution UI), and US-West, EU-Central.