Gcore - Cloud, DDoS Protection | Frankurt-2 Incident Details – Incident details

All systems operational

Cloud, DDoS Protection | Frankurt-2 Incident Details

Resolved
Partial outage
Started 27 days agoLasted about 16 hours
Updates
  • Postmortem
    Postmortem

    Root Cause Analysis

    Issue:

    On February 9, 2026, A network synchronization issue, combined with a targeted traffic surge, temporarily impacted VM availability in the Frankfurt-2 region.

    Timeline (UTC)

    • February 9, 2026 - 11:04 : Unknown unicast flooding begins

    • February 9, 2026 - 11:07: First compute node crashes (pcb19)

    • February 9, 2026 - 11:37: Incident / status page created

    • February 9, 2026 - 12:04: ARP entry expires; flooding stops

    • February 9, 2026 - 12:30: VM evacuation completes; environment stabilizes

    • February 9, 2026 - 16:32: Kubernetes clusters fully restored

    • February 9, 2026 - 20:09–20:16: Customer VM restoration completed

    • February 9, 2026 - 21:09: Incident closed

    Root-cause

    There is a condition in the network workflow where the MAC entry is removed from the access (ToR) switches while the corresponding ARP entry remains active on the upstream gateway until its natural expiration.

    This temporary inconsistency between Layer 2 and Layer 3 states can result in unknown unicast flooding within the affected VLAN.

    Under normal traffic conditions, this state mismatch does not cause significant impact. However, in this case, the server was simultaneously subjected to a high-volume DDoS attack, which amplified the flooding behavior and led to widespread compute instability.

    Action-Items

    • Implement mechanisms to promptly clear outdated network entries upon port changes to maintain accurate routing information.

    • Develop an automated process to clear network entries when certain withdrawal events occur on central network devices.

    • Review device behavior related to static bindings and explore alternative configurations to improve synchronization of network address mappings.

    • Assess and adjust existing evacuation procedures to align with storage integrity and recovery processes.

    • Improve monitoring capabilities to detect abnormal traffic conditions at the network segment level, including traffic rate thresholds and system health indicators.

  • Resolved
    Resolved

    We are happy to inform you that the partial outage in our Cloud and DDoS protection services in Frankurt-2 has been resolved. However, if you continue to experience any issues, please do not hesitate to contact our support team. Our team will be happy to assist you and ensure that any further concerns are addressed promptly.

    We appreciate your patience and understanding throughout this incident, and we thank you for your cooperation.
    A formal Root Cause Analysis (RCA) is currently being prepared and will be published once available.

    For further assistance, please contact our support team via support@gcore.com

  • Monitoring
    Monitoring

    We are pleased to inform you that our engineering team has implemented a fix to resolve partial outage in our Cloud and DDoS protection services in Frankurt-2. However, we are still closely monitoring the situation to ensure stable performance.

    We will provide you with an update as soon as we have confirmed that the issue has been completely resolved.

  • Identified
    Identified

    We have identified the issue and are working to find the root cause in order to resolve the network problem.

  • Investigating
    Investigating

    We are currently experiencing a partial outage in our Cloud and DDoS protection services, affecting both our private and public networks. This may result in partial unavailability for users. We apologize for any inconvenience this may cause and appreciate your patience and understanding during this time.

    We will provide updates as soon as more information becomes available on the progress of the resolution. Thank you for your understanding and cooperation.