Maintenance Windows for Sauce Labs
We are experiencing an intermittent issue in our US-West-1 datacenter that causes elevated wait times and error rates when starting desktop browser and virtual mobile device tests. Until we address the underlying causes of this issue, we expect this issue to continue to reoccur. We will post incidents here on our status page as each instance of this issue occurs whilst we continue to work to address the underlying cause.
2023-Feb-14 Resolved Service Incident
Incident
Report for Sauce Labs
Postmortem
Dates:
Tuesday February 14th 2023, 21:00 - 23:32 UTC
What happened:
Throughout the incident, several customers could not resolve ondemand.saucelabs.com, which caused any API calls to that URL for those impacted customers to fail.
Why it happened:
Adding a third DNS provider coincided with a few customers reporting that they could not resolve ondemand.saucelabs.com. The new DNS provider was removed out of caution; further investigation did not reveal a direct link from this change to the outage.
How we fixed it:
No specific restorative action was taken. A DNS change made during the incident was reverted out of caution.
What we are doing to prevent it from happening again:
In our first maintenance window of March, we successfully added the new DNS provider and have not seen any issues.
Resolved
Between 20:45 - 22:15 UTC, a configuration change on our side resulted in name resolution issue reaching ondemand.us-west-1.saucelabs.com causing test startup failures. We have identified the issue and taken remedial action.