Cloud | Singapore-1 | Major Outage incident details
Incident Report for Gcore
Postmortem

Cloud | Singapore-1 | Major Outage incident details

Incident Description: The client's infrastructure sustained a massive DDoS attack that significantly impacted the control plane of the cloud API, leading to potential network connectivity issues.
Root Cause: The incident resulted from current infrastructure design limitations and the lack of advanced automated systems to quickly detect and mitigate large-scale attack attempts.
Action Items:

  1. Architectural Improvements: Redesign the infrastructure with enhanced protection to better withstand such DDoS attacks.
  2. Automation Upgrades: Strengthen automated detection systems to identify attack patterns early and isolate them without manual intervention.
  3. Preventive Measures: Develop a more comprehensive strategy for DDoS mitigation to reduce the risk of future service disruptions.
Posted Oct 31, 2024 - 14:14 UTC

Resolved
We'd like to inform you that the issue has been resolved, and we are closely monitoring the performance to ensure there are no further disruptions. We will provide a Root Cause Analysis (RCA) report in the coming days to help you understand what caused the incident and the steps we have taken to prevent it from happening again in the future.

We apologize for any inconvenience this may have caused you, and want to thank you for your patience and understanding throughout this process.
Posted Oct 25, 2024 - 16:43 UTC
Identified
We'd like to let you know that our engineering team has successfully located the issue that is affecting the control plane of our Cloud Singapore region.

Our team is now fully focused on rectifying the situation.
Posted Oct 25, 2024 - 15:58 UTC
Investigating
We are currently experiencing a degradation in performance, which may result in service unavailability. We apologize for any inconvenience this may cause and appreciate your patience and understanding during this time.

We will provide you with an update as soon as we have more information on the progress of the resolution. Thank you for your understanding and cooperation.
Posted Oct 25, 2024 - 15:27 UTC
This incident affected: Cloud | Singapore-1 (Compute - Instances, Compute - Boot Volumes, Compute - Custom Images, Compute - Tags, Marketplace, Block Storage - Block Volume, Block Storage - Snapshots, Networking - Private Network, Networking - Router Management, Networking - Floating IPs, Networking - Firewall, Networking - Load Balancing, Managed Kubernetes, Function as a Service, File Shares as a Service, Managed Logging, Managed PostgreSQL, Containers, Container Registry).