2018-11-27 Resolved Service Incident
Incident Report for Sauce Labs Inc
Postmortem

Date: November 27, 2018
Time: Intermittent between 11:08am and 1:50pm PST

*What happened:
*
Wait times for Windows and Linux VMs in our Virtual Cloud exceeded 30 seconds.

*Why it happened:
*
An unusually large increase in testing load led to high wait times.

*How we fixed it:
*
As load decreased, the service recovered.

What we are doing to prevent it from happening again:
We have already deployed additional capacity to meet customer demand. We are also adjusting our capacity planning models in light of new usage patterns.

Posted 6 days ago. Dec 06, 2018 - 07:26 PST

Resolved
On Tuesday, November 27th between 11:20 am and 1:50 pm PST our service had several sustained intervals of high wait times in our PC cloud. The root cause was identified as high demand exacerbated by a lack of resources on a proxy server cluster used to store test assets. Our long standing plan to increase capacity of both PC and Mac clouds will be executed tomorrow to prevent this type of incident in the future. Additionally, more resources have been added to the proxy cluster. All services are now fully operational.
Posted 14 days ago. Nov 27, 2018 - 18:41 PST