Maintenance Windows for Sauce Labs
Sauce Labs has consolidated onto one status page for all data centers.
The old status pages for US-West, US-East and EU-Central are still available to be able to view incident history. Anyone who was subscribed to notifications from one of the old status pages has been migrated onto this consolidated page. If you received an email saying that you had been unsubscribed from notifications, that was from one of the old pages and can be disregarded.
If you would like to select which components that you receive notifications for (e.g. if you are only interested in receiving notifications about incidents affecting Real Device Testing), you can access the subscription management page by clicking "Subscribe To Updates" above and enter the email address you are subscribed with. Additionally there is a "Manage Your Subscription" link on any email you will receive from the status page going forward.
The old status pages for US-West, US-East and EU-Central are still available to be able to view incident history. Anyone who was subscribed to notifications from one of the old status pages has been migrated onto this consolidated page. If you received an email saying that you had been unsubscribed from notifications, that was from one of the old pages and can be disregarded.
If you would like to select which components that you receive notifications for (e.g. if you are only interested in receiving notifications about incidents affecting Real Device Testing), you can access the subscription management page by clicking "Subscribe To Updates" above and enter the email address you are subscribed with. Additionally there is a "Manage Your Subscription" link on any email you will receive from the status page going forward.
2021-April-13 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates
Tue April 13th, 2021 19:20-20:24 UTC
What happened
Customers using Sauce Headless experienced elevated wait times and an increased rate of tests unexpectedly terminating.
Why it happened
Our Google Kubernetes Engine (GKE) cluster underwent automated upgrades during the incident window, causing services to unexpectedly terminate.
How we fixed it
The automated cluster upgrade was completed and service returned to normal operations.
What we are doing to prevent it from happening again
We’re increasing system robustness by adding graceful shutdown mechanisms and creating a highly available setup for identified services.
Resolved
We have taken remedial action. us-headless-1 (Headless Cloud) should be running jobs without error.
Investigating
us-east-1 (Headless Cloud) jobs are failing intermittently. We are taking remedial action.
This incident affected: Automated Browser Testing (US-East (Headless)).