The continued impact of cyberattacks on federal agencies has led to a massive investment in cybersecurity. Last year, when President Biden announced the 2023 budget, it came with an 11 percent overall increase in cybersecurity. The Department of Defense (DoD) was allocated $11.2 billion for cybersecurity this year, and with the 2024 budget now under discussion, the DoD is requesting $13.5 billion for cybersecurity. Vice Admiral Sara Joyner, Director of Force Structure, Resources, and Assessments for the Joint Staff, said in a recent Pentagon press briefing that the budget request enables the DoD to “continue to modernize network defense capabilities to build a secure and resilient cyber architecture.” But as billions are spent on cyber defenses, how can the DoD quantify the expenditures and measure their effectiveness?

One approach is to map the organization’s cyber investments on a kill chain such as Lockheed Martin’s Cyber Kill Chain or MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK). Both are frameworks that attempt to explain the anatomy of an attack and potential mitigation. Part one of this series explored with the help of Rico Cody, Solutions Architect at Verizon, the cybersecurity strategy for Defense In-Depth, which requires that tools and rules be layered tacitly. The second part of this series considered the importance of an effective logging strategy in cyber defense. In this final article of the series, we put it all together to examine how agencies can identify what aspects of these strategies are successful.

Challenges Managing Multiple Security Solutions

Increasing budgets for cybersecurity have led the DoD to adopt new strategies and solutions. While many of these solutions have become integral to fighting cyberattacks, the challenge today is how to determine what’s effective and what isn’t. “Agencies are investing a great deal of money in cybersecurity, but it is difficult to understand the full value of tools to stop threats,” Cody explained. Despite significant data collection, it is challenging for the DOD to hone in on the right set metrics to gain an understanding of a specific tool’s capability to stop or mitigate threats, so “the metric is basically meaningless,” he added.

For example, the data might show that in January, a firewall stopped 1,000 threats from getting to the enterprise. In February, data may show 2,000 were stopped by the firewall. It’s useful to know that the firewall is doing its job, but what does this data really mean? Did the attackers decide to go on vacation in February? Were network defenders more alert in February than in January? Was there an update to the firewall rules with signatures that were not available the previous month? Collecting a metric that trends the number of threats stopped month to month provides little to no use to an organization.  It is difficult to identify actionable information provided as to what aspect of the security was effective, and hard to determine the value or ROI on the tools being used. What the DoD needs is a sophisticated way to better measure and understand security.

What is the Cyber Kill Chain and Its Benefits?

One solution that has added visibility into the efficacy of cybersecurity solutions is to utilize a cyber kill chain framework. Based on the military concept of kill chains, the cyber kill chain framework was created by Lockheed Martin in 2011. In a kill chain, the various steps of an attack are broken down to establish defenses for multiple stages; not only does this prevent threats from becoming attacks, but it also pinpoints where any problems occur in the defense. The cyber kill chain uses this concept toward understanding cyberattacks.

There are seven steps in the DoD cyber kill chain:

1. Reconnaissance: Before an attack can happen, bad actors must research and identify targets and vulnerabilities.
2. Weaponization: Bad actors then create backdoor access to deliver the payload.
3. Delivery: The weaponized payload is delivered to the intended target, initiating the attack.
4. Exploitation: Once delivered, the malware or malicious code finds vulnerabilities to exploit.
5. Installation: A backdoor is installed, granting the bad actors easy access whenever they need.
6. Command and Control (C2): The bad actors can now have control of the target’s server using remote access.
7. Actions on Objectives: With this access, the bad actor now has the ability to deliver on their objectives, including stealing, manipulating, or deleting data.

The table below illustrates the seven stages of the kill chain, examples of attack methods at each stage, and capabilities organizations may invest in at each stage to mitigate against an attack.

---

By laying out the steps that bad actors take in an attack, the DoD can gain a more extensive understanding of their process and methodology. Leveraging this deeper understanding, agencies can incorporate the right tools of defense at each of the seven stages to provide the most efficient security. Additionally, they are able to see where exactly the weak points are or where the defense fails, ensuring all tools are as effective as possible. “Using metrics and data, agencies can determine where threats have been stopped, and then determine if improvements can be made at other stages,” Cody explained.

Organizations can begin to understand the value of their investments by following these five steps:

1. Map an organization’s tools (cyber investments) on a cyber kill chain
2. Measure over time which tools successfully stopped each attack
3. Measure over time where on the kill chain each attack was stopped
4. Measure mean time to detect an attack
5. Measure time to resolve attack

These are more meaningful metrics that may drive investments to strengthen the defensive posture of the enterprise.  Some additional questions that agencies can ask over time include:

1. Where on the kill chain was the attack stopped?
2. Why was the attack stopped at a certain stage of the kill chain?
3. Why was the attack not stopped at an earlier stage of the kill change?
4. What can organizations do to detect and mitigate threats more quickly?
5. Should the organization increase investments at earlier stages of the kill chain?
6. Should the organization decrease investments at later stages in the kill chain?
7. Should the organization shift defensive capabilities from later to earlier stages in the kill chain?

Building a More Effective Cyber Defense

It is expensive to implement cybersecurity strategies and solutions, but it is even more expensive to deal with a cyberattack. It’s vital for DoD agencies to dive deeper into the security measures they are taking to understand their effectiveness. Spending more money on advanced solutions doesn’t truly fix the problem nor automatically make the DoD more secure. Even the best offenses and defenses are meaningless unless they are constantly being evaluated for performance and value. The cyber kill chain allows agencies to create defenses that address the steps bad actors take in an attack so that the best defense can be constructed against them.