I‘ve written elsewhere about the need to red team major national policies. As U.S. legislators again consider adopting an expensive and consequential program—in this case, health care reform—I once more question the apparent lack of structured challenge analysis within the arena of public discourse. From my perspective, this lack underscores the chronic wishful thinking and demonizing that animates national debates.
Many possible approaches exist. I illustrate one here. The framework took me about 15 minutes to build, the basic description perhaps another 15 minutes to flesh out. It’s not perfect, but I do believe it’s a better framework for analyzing consequences than those adopted (or not adopted) in most of what passes for analysis on news channels.
The framework employs the scenario analysis technique Peter Schwartz champions in The Art of the Long View. Using this approach, the analyst or team first assembles a list of driving forces, or factors that are likely to frame whatever future that emerges. Some of these forces will be deterministic, others more uncertain. The most uncertain and consequential forces are labeled critical uncertainties. The two most interesting or significant critical uncertainties may be crossed to form a simple, four-quadrant matrix.1 Each quadrant defines one possible future.
The goal is not to predict the future but to frame the scenarios in which alternatives or strategies may be tested. A proposal that fares well in one scenario but fails miserably in the others may be described as fragile. It should be questioned. A proposal that fares well in all scenarios is robust. Whether a proposal is fragile or robust is not the sole factor a decision maker should consider, but it does provide a simple and fairly intuitive measure of risk. Of course, a fragile and risky strategy may pay off handsomely, while a robust strategy may yield only modest rewards.
Perhaps surprisingly, many analysts and decision makers never get this far. Proponents and critics alike rarely specify the future context in which their proposals or criticisms are likely to play out. As one would expect, proponents tend to assume a best-case scenario; critics assume the worst case. Again, however, the logic of these contexts is rarely specified, even when understood.
For this quick-look analysis, I have settled on two critical uncertainties: financial stability and external threat. Financial stability characterizes the health of the U.S. economy. High financial stability denotes a strong economy, and low financial stability denotes economic weakness and insecurity. External threat characterizes the state of U.S. security. High external threat denotes a condition in which the U.S. is threatened on many fronts, and low external threat denotes a relatively peaceful condition. In a state of high external threat, the country may be either (1) under the direct threat of attack or conflict or (2) under attack or engaged in an active conflict.
Crossing these axes yields the following scenario space:
I have characterized each scenario using a short label. The scenario defined by high financial stability/low external threat is good times; high financial stability/high external threat is under siege; low financial stability/low external threat is the unraveling; low financial stability/high external threat is dystopia.
Even a simple analysis should trigger a pause. The financial and policy consequences of the proposed health care legislation are likely to be problematic in under siege, the unraveling, and dystopia. These consequences may be measured in many ways, from tangible metrics such as direct and indirect costs to less tangible metrics such as policy focus and national energy. (In either scenario involving high external threat, for example, policy makers’ limited time will be heavily taxed by pressing national security concerns.) This indicates a potentially fragile strategy.
Proponents may reasonably counter that the costs will be covered by the program, that the payoff is worth the risk, or that doing nothing yields even more problematic consequences. Regardless, even a back-of-the-envelope scenario framework helps structure the debate and ensure that key factors are examined systematically, thereby reducing the chance that we will stumble into a disaster. Whether you are a proponent or a critic of the current bill, it should be clear that the give-and-take that attends systematic challenge analysis is likely to yield better policy.
Unfortunately, the situation is somewhat more complex than the figure above indicates. The scope and scale of the proposed legislation is so massive that—if adopted—it could actually influence the critical uncertainties. In other words, if we adopt the proposal, we modify the probability of landing within each of the framing scenarios. This feedback effect merely compounds the need to weigh the risks and consequences systematically.
In my classes, we often discuss the practical limits of systems analysis. One of the most consistent frustrations students express is the seemingly inescapable polarity that characterizes major policy debates. Even when policy makers apply systems analysis to major policy questions, the analysis is typically skewed in one direction or another. Proponents undertake analysis to prove their case; critics undertake analysis to prove theirs. Very few objective sponsors exist. The cynic in me argues that this condition will always hold. The optimist in me believes that red teaming or, more broadly, challenge analysis can help inject more integrity into consequential policy debates. For better or worse, the red teamer in me usually trusts the cynic.
- Crossing three critical uncertainties yields a cube. A scenario logic incorporating more than three critical uncertainties is typically unmanageable. [↩]