No components marked as affected
Resolved
This incident has been resolved.
Monitoring
When we scaled up for the evening load we used an old copy of a binary that recreated the indexes we deleted earlier today. Replacing this with the newer version and restarting some boxes has solved the problem.
We're very sorry for how this issue has continued to be a problem today.
Identified
The mistaken re-creation of an index in Postgres is spiking our IOPS, causing our services to timeout. We've identified the problem and are pushing a fix.