In a recent incident, Cloudflare, a leading cybersecurity company, fell victim to a suspected state-sponsored attacker. The breach, discovered on November 23rd, occurred after the threat actor used compromised credentials from the October 2023 Okta hack to gain unauthorized access to Cloudflare’s internal systems. Although the incident was promptly identified and mitigated, it serves as a reminder of the evolving landscape of cybersecurity threats. In this blog post, we will delve into the details of the breach, highlight the measures taken by Cloudflare, and discuss the lessons that can be learned from this unfortunate incident.
The Breach:
The compromised login information, including an access token and three service account credentials, were not rotated following the Okta hack. This oversight provided an opportunity for the attackers to conduct reconnaissance and probe Cloudflare’s systems starting from November 14th. Despite their access being limited due to network segmentation, they managed to infiltrate an AWS environment and gain access to Atlassian Jira and Confluence. The attackers actively searched for specific information related to the Cloudflare network and accessed 36 Jira tickets and 202 wiki pages during their unauthorized access period.
Actions Taken:
Cloudflare swiftly responded to the breach, taking measures to contain the attack and prevent further compromise. They discovered that the attacker had installed the Sliver Adversary Emulation Framework to gain persistent access to the compromised Atlassian server. Once this was identified, the server was promptly secured and the framework was removed. Cloudflare also terminated the unauthorized Atlassian account within 35 minutes of identifying it, ensuring that the intruder’s access was cut off. Additionally, firewall rules were established to block the attacker’s known IP addresses.
Security Improvements:
In the aftermath of the breach, Cloudflare initiated comprehensive security enhancements to fortify their systems against future threats. Over 5,000 production credentials were rotated, while close to 5,000 systems underwent triage and evaluation. Furthermore, test and staging systems were physically segmented, and every machine within the Cloudflare global network was reimaged and rebooted. An investigation was conducted to understand the extent of the intrusion and to ensure that the attacker no longer had persistent access to any systems.
Lessons Learned:
Cloudflare’s response to the breach highlights the importance of proactive security practices and the continuous assessment of access controls. Key lessons learned from this incident include:
01. Regular Credential Rotation: Rotating credentials is crucial to prevent unauthorized access, especially following any security incidents or compromise of credentials outside the organization.
02. Network Segmentation and Firewall Rules: Proper network segmentation and well-defined firewall rules can limit the lateral movement of threat actors, reducing the impact of a breach.
03. Vigilance in Access Control: Promptly terminating unauthorized accounts and conducting regular reviews of user access can help minimize the risk of insider threats and unauthorized access.
04. Embracing Zero Trust Principles: Implementing Zero Trust tools, such as hard security keys, can ensure that only authorized users gain access to sensitive systems.
Conclusion:
The breach experienced by Cloudflare serves as a stark reminder that even industry-leading cybersecurity companies are not immune to the evolving threat landscape. However, the swift response and extensive security enhancements undertaken by Cloudflare demonstrate the importance of proactive measures and ongoing evaluation of security controls. By learning from this incident and implementing the lessons derived from it, organizations can strive to enhance their cybersecurity posture and defend against state-sponsored attackers and other emerging threats.
Disclaimer: The information and events mentioned in this blog are based on the article published on SecurityWeek.