System Latency on GCP Asia region

Incident Report for EdCast by Cornerstone

Postmortem

Issue Summary:
On February 26, 2026, at 07:48 PM, customers experienced significantly elevated response times across the platform, which in several cases resulted in temporary service unavailability. Users reported slow page loads, delayed API responses, and intermittent timeouts while accessing the system. The incident impacted multiple customer environments and lasted approximately one hour before full service restoration.

Root Cause:
The issue was attributed to a sudden and unanticipated spike in database requests, which caused a sharp increase in database CPU and connection utilization. This surge led to resource contention at the database layer, resulting in slower query execution times and increased request queuing.

Corrective Action:
Upon identification of the issue, the engineering team took immediate action to scale up database resources. This included increasing compute capacity and optimizing database throughput to handle the elevated request volume.

Preventive Measures:

  • Enhance monitoring and alerting mechanisms with lower thresholds for key database performance indicators (CPU, memory, connections, query latency) to enable earlier detection and response.
  • Optimize high-frequency and resource-intensive database queries to reduce overall load and improve efficiency.
Posted Apr 24, 2026 - 09:49 PDT

Resolved

The elevated latency issue affecting GCP Asis has been successfully resolved.

We have verified that latency metrics have returned to expected thresholds and all impacted services are now operating normally.

The RCA for the issue will be shared within 7 to 10 business days.

Thank you for your patience and understanding during this incident.
Posted Mar 03, 2026 - 02:18 PST

Update

The system performance on the GCP ASIA swimlane has remained stable post resolution. We did not record any latency concerns on GCP Asia and we continue to monitor the environments to ensure continued stability before concluding a closure of this incident.
Posted Mar 01, 2026 - 22:09 PST

Update

The system performance on the GCP ASIA swimlane has remained stable post resolution. We did not record any latency concerns on GCP Asia and we continue to monitor the environments to ensure continued stability before concluding a closure of this incident.
Posted Mar 01, 2026 - 14:33 PST

Update

The system performance on the GCP ASIA swimlane has remained stable post resolution. We did not record any latency concerns on GCP Asia and we continue to monitor the environments to ensure continued stability before concluding a closure of this incident.
Posted Mar 01, 2026 - 14:30 PST

Monitoring

The performance issue with the GCP ASIA swimlane has been resolved, and the sites are now loading as expected. We are currently monitoring the environments to ensure continued stability.
Posted Feb 26, 2026 - 20:55 PST

Investigating

We are currently experiencing performance slowness on the GCP Asia Production environment. Our Operations team is actively working to restore the instance to normal performance levels.

We appreciate your time and patience.
Posted Feb 26, 2026 - 20:34 PST
This incident affected: Mumbai (GCP) (Web, API, Data and Analytics, Content Integrations).