A metric is “A system or standard of measurement,”1 and we generally combine a target and a metric to define success or failure. In this way, metrics quantify our goals. For example, we might define success as sales of more than $1 million (the target) dollars (the metric), or we might define failure as damage to at least 10 (the target) systems (the metric). When red teaming, it is essential to understand both the defender’s and the attacker’s metrics.
Inexperienced red teams are inclined to adopt the defender’s metrics as their own. For example, if the defender is most concerned about the loss of lives, the unit “casualties” might be the defender’s key metric. The target might be zero casualties over the course of the coming year. In this case, the inexperienced red team simply reverses this target/metric combination and seeks to inflict the maximum number of casualties on the defender.
This certainly simplifies the equation, but at what cost? What if the real-world adversary’s (RWA’s) metric differs? What if, for instance, the RWA’s key metric is not casualties but system downtime? While casualties can result in system downtime, other methods can as well. Perhaps the RWA has identified 10 courses of action that achieve a sufficient degree of system downtime, and not one of these courses of action involves casualties. Thus, while the red team focuses on maximizing casualties, the RWA pursues other courses of action. Further, when the resulting red team report highlights courses of action that yield casualties, the defender—looking elsewhere—is surprised when the RWA executes an unexpected attack designed primarily to take down the defender’s system. (The surprise is compounded by the fact that the red team reinforced the defender’s expectations by employing the defender’s metrics.)
This, then, is the first point: the red team must always seek to identify and employ the RWA’s metrics. If the RWA’s metrics differ from the defender’s metrics and the red team persists in using the defender’s metrics, the red team can mislead the defender.
The second point is this: The defender’s metrics signify the defender’s focus, a feature the savvy red team will seek to exploit to the education and benefit of the defender (who may in turn seek to exploit this dynamic).
Let’s make this point concrete by extending the previous example. If the RWA is indeed focused on courses of action that maximize system downtime, then the red team accounts for this by incorporating the RWA’s metrics in its assessment. Does this mean the red team should also include courses of action that yield casualties? Actually, yes—but it shouldn’t limit itself to this metric. After all, the red team must account for uncertainty. Maybe the RWA is focused on maximizing system downtime, but maybe it isn’t, or maybe an overlooked RWA is focused on maximizing casualties. As always, the red team’s job isn’t to answer one question with certainty but to weigh a range of questions while accounting for uncertainty.
Note that this more balanced approach also informs the defender regarding potential opportunities to broadcast a misleading set of preferred metrics. In other words, the defender can exploit the attacker’s expectations, but this consideration is only likely to emerge from a balanced red teaming assessment. To pull this off, the defender must understand the full range of possible RWA metrics. It also helps tremendously if the defender has some window into the RWA’s perceptions.
To summarize, the red team should always seek to incorporate the RWA’s metrics while acknowledging the defender’s concerns and accounting for the overriding presence of uncertainty. To conclude, we’ve distilled the principles into a short checklist:
- Identify and define the defender’s key metrics;
- Identify and define the potential real-world attackers’ (RWA’s) key metrics;
- Account for uncertainty;
- Identify the gaps between key metrics, and account for these gaps during the analysis;
- As appropriate, exploit the defender’s key metrics;
- Clearly explain the gaps and how they influenced your analysis when communicating your findings to the customer; and
- As appropriate, educate the defender regarding opportunities to exploit the attackers’ expectations.
- http://www.oxforddictionaries.com/us/definition/american_english/metric, definition noun/technical. [↩]