We are continuing to monitor for any further issues.
Posted Jan 25, 2019 - 09:59 GMT
Monitoring
We have identified that at 4:21am several machines in different zones simultaneously stopped responding. It appears that GCP attempted to restart each of those machines multiple times for about 2 hours. Some machines did eventually come back online. Deleting those flakey instances saw them replaced with fully functional instances.
Normal service has since been resumed. We will continue to monitor and have an incident report soon.
Posted Jan 25, 2019 - 09:00 GMT
Investigating
We are experiencing elevated responses times and 500 rates from our API, affecting ~2% of requests. We are investigating to identify the cause. Please check back here for further updates.