Maintenance Windows for Sauce Labs
We are currently performing maintenance that will affect the initiation of new Sauce Connect tunnel instances. The maintenance is expected to be completed by 2:00 PM PST. We appreciate your understanding and patience as we work to enhance our service.
2024-October-30 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Wednesday October 30th 2023, 03:45 - 21:06 UTC
What happened:
Intermittent errors occurred when starting or using Sauce Connect tunnel in the US West datacenter.
Why it happened:
A service responsible for creating new bindings between the tunnel endpoints and test VMs experienced timeouts on some hosts.
How we fixed it:
Service was restored after clearing bindings on the affected hosts.
What we are doing to prevent it from happening again:
Monitoring and alerting for when the service is unable to create new bindings has been improved. The root cause of the condition is under investigation.
Resolved
After taking remedial action all services are operating as normal. This incident is resolved.
Monitoring
We have taken remedial action and are seeing an improvement with Sauce Connect tunnel startup and allocation in the US-West-1 data center. We are monitoring
Investigating
We are currently seeing intermittent issues with Sauce Connect tunnel startup and allocation in the US-West-1 data center. We are investigating.
This incident affected: Automated Browser Testing (US-West), Automated Virtual Mobile Device Testing (US-West), Automated Real Device Testing (US-West), Live Browser Testing (US-West), Live Virtual Mobile Device Testing (US-West), Live Real Device Testing (US-West), API Testing (US-West), Sauce Connect (US-West), Visual Testing (Visual Testing Hub, Visual Testing Infrastructure), Native Framework Mobile App Testing (US-West), and Sauce Orchestrate (US-West).