2018-February 6 Service Incident
Incident Report for Sauce Labs Inc
Postmortem

Date: February 6, 2018
Time: 10:57 am - 11:52 am PST

What Happened:
Wait times for Windows and Linux VMs were high, which led to some tests timing out.

Why did it happen:
A transient network issue led to a temporary drop in capacity, causing wait times to increase.

What did we do to fix it:
The problem recovered on its own.

What we are doing to prevent this from happening again:
We’re reviewing our network configurations and adding more network logging to facilitate further investigation. Additionally, we’re adding logging and instrumentation to the service responsible for managing the cloud to more quickly predict network issues and react more effectively when they occur.

Posted 6 months ago. Feb 13, 2018 - 15:23 PST

Resolved
Our PC cloud has fully recovered and tests are starting normally. All systems are fully operational.
Posted 6 months ago. Feb 06, 2018 - 12:20 PST
Monitoring
Wait times on the PC Cloud have gone down to normal levels. We are monitoring all cloud activity closely.
Posted 6 months ago. Feb 06, 2018 - 12:04 PST
Investigating
Tests on our PC cloud are experiencing high wait times. We are investigating the cause.
Posted 6 months ago. Feb 06, 2018 - 11:29 PST
This incident affected: Automated VM Testing (Automated PC Testing) and Manual Testing (Manual VM Testing).