May 9, 18:34 UTC
Resolved – Based on the monitoring, all packet routers have fully stabilized and rollout is complete.
May 9, 16:43 UTC
Update – Based on the monitoring, the core team is rolling out the GRPC connection stability update to all packet routers.
May 9, 16:28 UTC
Update – The U.S. packet routers are back up. We are now evaluating the patch performance on GRPC connection stability in the selected few packet routers. After confirmation, we will rollout to the other packet routers.
May 9, 16:11 UTC
Monitoring – The core team is doing a staged rollout for packet routers to optimize GRPC process leaks. We’ll start with a rollout on a few packet routers, monitor for a bit before releasing to all packet routers. The rollout is doing fine in other regions, but causing memory issues in U.S. The core team is upgrading the packet router instances in U.S., the expected downtime is 15-20 minutes.