Resolved -
This incident has been resolved.
Oct 21, 11:30 UTC
Update -
The patches for running the recorder in a new region had some unexpected side effects causing some recorders to continue to fail. Further mitigations have been deployed and processing appears to be moving forward successfully.
Oct 20, 18:03 UTC
Update -
We've deployed a patch to start scheduling the recording in a new AWS Region, we're seeing successful recording runs.
Oct 20, 16:07 UTC
Update -
AWS continues to be have failures launching new instances which is preventing scheduled and ad-hoc recorders from booting up. The team is working on spinning up resources in a new region.
Oct 20, 14:55 UTC
Update -
We are continuing to monitor for any further issues.
Oct 20, 13:59 UTC
Monitoring -
AWS has deployed mitigations but we are still observing some issues with upstream providers being able to launch instances. We are continuing to monitor the situation.
Oct 20, 13:09 UTC