Latency in US West Production

Incident Report for EdCast by Cornerstone

Postmortem

Issue Summary:
Across two separate incidents on September 1 and 2, 2026, customers in the US West region experienced application performance degradation, including increased response times and intermittent Gateway Timeout (504) errors.

The incidents were associated with an unexpected and sustained increase in API traffic within the environment, which placed significant demand on shared infrastructure resources and affected overall platform performance.

Root Cause:
The incidents were caused by an unusually high and sustained volume of API requests originating from a single integration. The request volume exceeded expected usage patterns and consumed a significant amount of available infrastructure resources, resulting in resource contention and degraded application performance.

Corrective Action:
Cornerstone increased infrastructure capacity to stabilize platform performance and investigated the source and pattern of the increased API traffic. Once the source was identified, the abnormal traffic was restricted to prevent further impact. Following these actions, platform performance returned to normal, and the environment was monitored to confirm continued stability.

Preventive Measures:
To reduce the risk of a similar issue, Cornerstone has taken the following actions:

  • Implemented additional monitoring and alerting to provide earlier notification when abnormal resource consumption or API traffic patterns are detected.
  • Initiated a review of additional usage-based controls, including throttling mechanisms, to better contain unusually high resource consumption and reduce the potential for broader platform impact.
Posted Sep 09, 2026 - 13:10 PDT

Resolved

This incident has been resolved.
Posted Sep 03, 2026 - 11:04 PDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Sep 02, 2026 - 10:20 PDT

Update

A fix has been implemented and we are monitoring the results.
Posted Sep 02, 2026 - 10:20 PDT

Identified

Latency has been observed in the US West Production region. Users and monitoring systems have reported degraded response times, potentially impacting service availability and user experience.
Engineering team is working on the solution.
Posted Sep 02, 2026 - 07:38 PDT
This incident affected: US West (Web).