Thursday October 26th 2023, 17:03 - 17:30 UTC
For the duration of the incident, customers could not upload new files or list files/groups through the Web UI. Some users received an HTTP error code of 429 due to them being rate-limited.
Customers began to have issues uploading assets to our storage service due to a high number of concurrent uploads. This spike in upload requests caused a delay in acquiring a backend connection for the storage service.
We recycled the storage service pods, which cleared the connectivity issue with our backend service.
We are looking at the following areas: