Maintenance Windows for Sauce Labs
Sauce Labs will have a four hour previously scheduled maintenance window on Wednesday, July 10th, starting at 04:00 UTC and ending at 08:00 UTC. During this maintenance window, we will be making updates to our infrastructure and services in our US West data center. These maintenance actions may cause portions of the service (including running automated and manual tests) to be unavailable for up to four hours.
2024-March-6 Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Wednesday March 6th 2024, 12:43 - 19:00 UTC
What happened:
Test artifact uploads and downloads intermittently failed or were slower to complete, resulting in issues displaying test results.
Why it happened:
A subsystem involved in artifact uploads would not properly recover from some failures, resulting in leaked connections and connection pool exhaustion. Once fully exhausted, uploads were no longer possible.
How we fixed it:
We identified the bug in the code and deployed a fix.
What we are doing to prevent it from happening again:
We have fixed the specific bug and implemented monitoring for similar symptoms.
Resolved
This incident has been resolved.
Investigating
We are currently experiencing intermittent issues loading real device test reports on the dashboard in our US West Data Center. We are investigating.
This incident affected: Automated Real Device Testing (US-West), Live Real Device Testing (US-West), and Native Framework Mobile App Testing (US-West).