Friday November 7th 2022, 08:30 - 19:30 UTC
A portion of newly signed-up users saw an error in the Sauce Labs frontend as they tried to log in after signing up.
As new users were signing up in Sauce Labs, a backend service responsible for managing user data wasn’t being updated fast enough. This caused the user’s subsequent login attempts to fail until that data was updated.
This race condition was introduced as a change to the service managing user concurrency was made and pushed to production.
That change was reverted to address the race condition, and the team also changed to the front end to respond more gracefully in case a backend service error is thrown, as we saw in this incident.
The team is exploring more areas where the Sauce Labs front end could run into this scenario and addressing those areas.