Asking the right question at the right time can help avert disaster. In fact, one could characterize red teaming and alternative analysis as the practice of posing and answering contrarian questions—questions such as “How could our adversary circumvent our defenses?” or “What flaws in our new product line might we have overlooked?”
A good question can help a decision maker challenge hidden assumptions and expose unexpected vulnerabilities. A poor question can foreclose options and reinforce biases. So, how does a red teamer pose good questions and avoid poor ones?
One approach is to learn by watching and doing. Good red teamers and alternative analysts often exhibit an intuitive ability to pose probing and insightful questions that slice through uncertainty, disorder, and overconfidence. Rarely does this skill emerge spontaneously from a junior analyst; rather, the analyst gains it over time by listening to experienced analysts and decision makers, assessing what works and what doesn’t, and testing the skill in a variety of contexts.
While the implicit approach of watching and doing does work in many cases, it is possible to build the skill more explicitly and systematically. Of the many resources available, I focus on one here: dialogue mapping. Specifically, I discuss the art of framing good questions within the dialogue mapping approach. While dialogue mapping can fill a book (see Jeff Conklin’s Dialogue Mapping), the question-framing portion of the approach is worthwhile as a standalone framework.
As background, dialogue mapping is designed to address the challenges that attend wicked problems and multiple stakeholders.1 It is based on Horst Rittel’s “Issue Based Information System” (IBIS), a technique that encourages analysts to structure problems and dialogue as questions, ideas, and arguments (pro and con).2 Questions are the heart of IBIS and dialogue mapping, and good questions go a long way toward ensuring success when using the approaches.
Conklin identifies seven types of questions and discusses each in the sixth chapter of his book. Here, I review the seven types and discuss briefly how they apply to red teaming.
- Deontic Questions. Deontic questions address what you should do. For example, a decision maker [BLUE] might ask “What should we do about our delayed product release?” A red teamer might ask the same question from the competitor’s perspective (“What should we [RED] do about BLUE’s delayed product release?”).3
- Instrumental Questions. These questions address how you should achieve your aim. For example, a decision maker might ask “How should we accelerate the product release?” The red teamer’s instrumental questions will address how the competitor or adversary can achieve the competitor or adversary’s goal. A good instrumental question from the red teamer might be “How should we [the competitor] exploit the delayed product release?”
- Criterial Questions. These questions address your criteria–your goals, requirements, metrics, and so on. Of note is the possibility that RED’s criteria might not simply be the obverse of BLUE’s. The red teamer can help expose potential asymmetry by asking good criterial questions from RED’s perspective. For example, BLUE might believe that the goal is to maximize profit while RED might be willing to accept a loss in a certain market niche in order to enhance its overall brand recognition.
- Meaning or Conceptual Questions. These questions address meaning, which may differ by participant or stakeholder. Most wicked problems involve multiple stakeholders and these stakeholders are likely to understand various aspects of the problem differently. This is true within both the RED and BLUE organizations and doubly true across the RED/BLUE divide. A red teamer can explicate important perceptual differences across this divide using good meaning questions.
- Factual Questions. These questions address issues of fact. Of course, what is fact can vary by stakeholder and perspective–another opportunity for the red teamer to broaden and enrich the dialogue by asking good questions from RED’s perspective.
- Background Questions. These questions address the problem’s background, which, like questions of meaning and fact, are likely to lead to different answers for RED and BLUE. RED’s perspective on the problem’s background should inform BLUE’s perspective and vice versa.
- Stakeholder Questions. These questions address the problem’s stakeholders–namely, “Who are the stakeholders on this project?” As suggested above, BLUE should not assume that RED’s decision making apparatus is any more monolithic than BLUE’s. Clearly, the red teamer should be asking “Who are RED’s stakeholders?” just as the decision maker’s team should be asking “Who are BLUE’s stakeholders?”
Conklin notes that deontic (what) and instrumental (how) questions ground most analysis. The other five types serve to deepen the analysis and broaden the decision maker’s perspective. Each type of question is just as appropriate for RED as for BLUE, and the analyst that asks only one or two types of questions from a single perspective is risking failure and surprise.
To the seven types of questions, I suggest adding an eighth:
- Hypothetical Questions. These questions, phrased in what if form, serve as a meta-category that cuts across the previous seven question types. The primary purpose of a hypothetical question is to encourage the analyst to broaden the context of the analysis to include drivers and effects that might otherwise be overlooked but could swing an outcome in one direction or another. For example, the red teamer might first ask a deontic question such as “What should we do about BLUE’s new product launch?” and then follow up with a hypothetical question such as “What if BLUE’s criteria differ significantly from what we now assume?” or “What if RED’s stakeholder D is more influential than we now assume?”
Dialogue mapping handles what I call hypothetical questions within its existing framework, so no need exists to add any new symbols or structures. I add the type here primarily to emphasize the important role of hypotheticals within red teaming and alternative analysis.
Before closing, it is worth mentioning Conklin’s three aspects of an artful question within the dialogue mapping framework:
- An artful question “must be simple, not compound.”
- “Artful questions do not try to ‘sneak’ major assumptions into their statement….”
- And finally, “artful questions are open, not closed. ‘What should we do?’ is open, ‘Should we cut costs?’ is closed.”
Of these three criteria, the second and third are arguably most interesting to red teamers and alternative analysts. A question that violates criteria two can help preserve bad assumptions, while a question that violates criteria three can foreclose good options. It takes practice to recognize poor questions just as it takes practice to pose an appropriate variety of question types, but the effort can be very worthwhile.
Dialog Mapping: Building Shared Understanding of Wicked Problems by Jeff Conklin.
- For more on wicked problems, see Horst Rittel’s 1973 paper, “Dilemmas in a General Theory of Planning.” For more on how wicked problems and social complexity combine to compound the challenge, see the first chapter of Jeff Conklin’s book, which is available online here. [↩]
- Find more on IBIS here. [↩]
- I use the following convention to refer to the client and competitor’s perspectives: BLUE represents the client decision maker’s perspective and RED represents the competitor, opponent, or adversary’s perspective. [↩]