Gcore - Global CDN and API Incident Details – Incident details

All systems operational

Global CDN and API Incident Details

Resolved
Major outage
Started 12 days agoLasted about 13 hours

Affected

Gcore Systems

Major outage from 7:24 PM to 8:15 PM, Operational from 8:05 PM to 8:41 AM

API

Major outage from 7:24 PM to 8:05 PM, Operational from 8:05 PM to 8:41 AM

Billing System

Major outage from 7:24 PM to 8:15 PM, Operational from 8:15 PM to 8:41 AM

Customer Portal

Major outage from 7:24 PM to 8:15 PM, Operational from 8:15 PM to 8:41 AM

Website

Major outage from 7:24 PM to 8:15 PM, Operational from 8:15 PM to 8:41 AM

CDN

Major outage from 7:24 PM to 8:15 PM, Operational from 8:05 PM to 8:15 PM, Major outage from 8:15 PM to 8:23 PM, Operational from 8:15 PM to 8:23 PM, Partial outage from 8:23 PM to 8:27 PM, Operational from 8:23 PM to 8:41 AM

Updates
  • Postmortem
    Postmortem

    Public RCA about the Incident:

    Date: May 13, 2026 | Duration: 19:08 – 20:14 UTC (1 hour 6 minutes)


    Summary

    On May 13, 2026, from 19:08 to 20:14 UTC, part of Gcore's CDN service experienced a global disruption. CDN edges worldwide failed to process requests and returned HTTP 502 errors for a subset of client resources. The disruption affected gcore.com, the Customer Portal, public API endpoints, and CDN delivery on part of the infrastructure. 


    Impact

    • gcore.com and portal.gcore.com were unreachable.

    • api.gcore.com returned 502 errors, affecting API-based operations across CDN, Cloud, DNS, Streaming, Storage, WAAP, and IAM services.

    • SSO/SAML-based authentication to the portal was disrupted during and briefly after the window.

    • Customers with CDN resources served through the affected infrastructure saw 502 errors for their end-user traffic.


    Root Cause

    This was a stacked-defect incident — three independent gaps in the CDN configuration pipeline combined to turn a single configuration change into a global edge failure. Any one of the three defects, had it been absent, would have prevented the outage.

    1. API input validation gap: An internal origin routing field, originally intended as an admin-only configuration knob, lost its access restriction in a 2023 API rewrite and was later published in the public API documentation (March 2026) without specifying its allowed values. This allowed a non-standard value to be submitted and accepted via the API.

    2. Configuration generation logic error: When the CDN configuration pipeline processed the resource with the non-standard value, a bug in the rule-level config generation silently dropped all origin servers — producing a configuration with an empty upstream list. 

    3. Edge initialization crash: When a CDN edge node received a configuration with an empty upstream list, an edge-side script crashed during the initialization phase. Because the configuration file is global (shared across all resources on a node), this single malformed entry caused the entire node to fail initialization — returning HTTP 502 for all traffic, not just the affected resource. This crash propagated across all edge nodes on the affected infrastructure.


    Timeline (UTC)

    Time

    Event

    19:08

    CDN configuration containing the malformed resource pushed to edge nodes globally

    19:08–19:14

    Edge nodes begin returning HTTP 502 globally

    19:15

    P1 incident declared

    19:24

    Public status page incident posted

    19:42

    Mitigation begins: critical services routed via alternative edge infrastructure

    19:53

    Customer portal migrated to alternative infrastructure

    20:01

    Additional resources migrated

    20:14

    Fix applied, Offending resource disabled via API; edge nodes recover

    22:05

    API-level validation fix merged

    23:06

    API fix deployed to production


    Resolution

    Service was restored at 20:14 UTC by disabling the resource containing the malformed configuration. Engineers had been mitigating the impact since 19:42 UTC by routing critical control-plane services (API, portal) through alternative edge infrastructure.


    Corrective Actions

    #

    Action

    Status

    1

    API-level input validation: reject non-allowed values for the origin routing field

    Deployed 

    2

    Fix config generation logic to correctly handle inherited origin groups, eliminating the silent origin-drop bug

    In progress

    3

    Harden edge initialization to degrade gracefully (rule-level 502) instead of crashing the entire node on empty upstream configuration

    In progress

    4

    Audit related API fields from the 2023 rewrite for similar access control regressions

    In progress

    5

    Review and update API documentation to clearly specify allowed values for all origin configuration fields

    In progress


    We sincerely apologize for the disruption this caused. We are committed to completing the remaining fixes and implementing additional safeguards to prevent a similar configuration pipeline failure from causing a global impact in the future.

  • Resolved
    Resolved

    We are happy to inform you that the Major outage with our website, Global CDN Delivery, API access for all services and Customer Portal has been resolved. However, if you continue to experience any issues, please do not hesitate to contact our support team. Our team will be happy to assist you and ensure that any further concerns are addressed promptly.

    We will also provide a detailed Root Cause Analysis (RCA) once it becomes available.

    We appreciate your patience and understanding throughout this incident, and we thank you for your cooperation.

    For further assistance, please contact our support team via support@gcore.com

  • Monitoring
    Monitoring

    The issue has been resolved, and all services have fully recovered. We are continuing to monitor the situation to ensure stability. We apologise for the inconvenience.

  • Update
    Update

    We are recovering, and all services are up and running. CDN service has mostly recovered; however, it may still be partially unavailable for some users. We are continuing to monitor the situation and working on the fix.

  • Update
    Update

    Website and Customer Portal are up again. We are continuing to fix other services.

  • Identified
    Identified

    The API access has been recovered, and we are continuing to fix other services. We will keep you updated.

  • Investigating
    Investigating

    We are currently experiencing a major outage with our website, Global CDN Delivery, API access for all services and Customer Portal, resulting in its complete unavailability. We sincerely apologise for any inconvenience this may cause and greatly appreciate your patience and understanding during this critical time.

    Our engineering team is actively working to identify the root cause and implement a resolution as quickly as possible. We will provide regular updates as we receive more information on the progress of the resolution

    Thank you for your understanding and cooperation.