Maintenance Windows for Sauce Labs
We are currently performing maintenance that will affect the initiation of new Sauce Connect tunnel instances. The maintenance is expected to be completed by 2:00 PM PST. We appreciate your understanding and patience as we work to enhance our service.
2023-Feb-14 Resolved Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Tuesday February 14th 2023, 21:00 - 23:32 UTC
What happened:
Throughout the incident, several customers could not resolve ondemand.saucelabs.com, which caused any API calls to that URL for those impacted customers to fail.
Why it happened:
Adding a third DNS provider coincided with a few customers reporting that they could not resolve ondemand.saucelabs.com. The new DNS provider was removed out of caution; further investigation did not reveal a direct link from this change to the outage.
How we fixed it:
No specific restorative action was taken. A DNS change made during the incident was reverted out of caution.
What we are doing to prevent it from happening again:
In our first maintenance window of March, we successfully added the new DNS provider and have not seen any issues.
Resolved
Between 20:45 - 22:15 UTC, a configuration change on our side resulted in name resolution issue reaching ondemand.us-west-1.saucelabs.com causing test startup failures. We have identified the issue and taken remedial action.