Maintenance Windows for Sauce Labs
Sauce Labs will have a four hour previously scheduled maintenance window on Wednesday, July 10th, starting at 04:00 UTC and ending at 08:00 UTC. During this maintenance window, we will be making updates to our infrastructure and services in our US West data center. These maintenance actions may cause portions of the service (including running automated and manual tests) to be unavailable for up to four hours.
2024-February-26 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Monday, February 26th, 2024 23:08 UTC - Tuesday, February 27th, 2024 02:56 UTC
What happened:
Increased network load which led to test slowness or failures affecting Real Device tests in our US-West-1 data center.
Why it happened:
During the troubleshooting of an increase in network load in our US-West-1 data center, a network failover event was executed..
How we fixed it:
The network failover and additional traffic shaping stabilized services.
What we are doing to prevent it from happening again:
Improvements to the runbook for manually initiated failovers.
Resolved
After taking remedial action, Real device availability and error rates are back to normal in the US-West-1 data center. All services are fully operational.
Monitoring
After taking remedial action, Real device availability and error rates are returning to normal levels in the US-West-1 data center. We are currently monitoring.
Investigating
We are currently seeing elevated error rates and device availability in Real device tests in our US-West-1 datacenter. We are currently investigating
This incident affected: Automated Real Device Testing (US-West), Live Real Device Testing (US-West), and Native Framework Mobile App Testing (US-West).