Finding Clarity in Complexity HOW TO USE INTERVIEWS TO LISTEN TO A SYSTEM

By Jeff Mohr & Theo Gibbs 1

TABLE OF CONTENTS Why interview? Framing the interview process

3! 4!

Decide who to interview Rule 1: Go broad Rule 2: Go deep Rule 3: Go extreme Rule 4: Keep iterating

5! 6! 7! 7! 8!

Develop an interview guide Techniques for bird’s-eye stakeholders Techniques for ground-level stakeholders

9! 9! 11!

Conduct interviews Before the interview During the interview Toward the end of the interview After the interview

13! 13! 14! 15! 15!

Create an interview-capture process Their persona Their relationships Their perception of the system

16! 16! 16! 17!

Synthesize your insights Build stakeholder personas Analyze relationships between stakeholders Summarize initial system insights

18! 18! 20! 21!

Next steps

22!

2

INTRODUCTION

Why interview? Systems mapping is a powerful approach for making sense of complexity. Being able to visually represent ideas about causality and influence frees up much needed mental capacity to do further sensemaking, while also making our assumptions visible to others. But the value of creating a systems map is dependent on the diversity of perspectives and insights that are part of the mapping process. Creating a map based purely on our own vantage point won’t yield breakthrough insights, even if we are “experts” on the topic. So what’s the best way to bring additional perspectives and insights to your mapping efforts? One path is to involve a representative set of stakeholders in the actual building of the map. While ideal, “getting the whole system in the room” isn’t always practical. Rather, we’ve found it effective to combine 1) the building of a system map through an in-person convening of diverse stakeholders (especially including those who disagree with us and each other) with 2) a structured interview process that taps into a wider range of perspectives across all parts and levels of the system. We’ve joined forces with Theo Gibbs (Lecturer at the Stanford d.School, cofounder of Blue Heart and previous Systems Design Strategist at ChangeLabs) to develop a guide to support you in structuring your own interview process to better understand complex systems.

3

Framing the interview process Interviews for understanding a complex system go beyond traditional needfinding interviews you may be familiar with if you’ve used design thinking or human-centered design. Instead of focusing primarily on user needs, these interviews combine depth with breadth, looking for insight about the behavior and motivation of individuals while also exploring the relationships among stakeholders and broader dynamics playing out across the system. At its core, the interview process is about answering three questions: • Who’s involved in the system? • What’s happening? • Why is it happening? We’ve broken the process into five key steps: 1. Decide who to interview 2. Develop an interview guide 3. Conduct interviews 4. Create an interview-capture process 5. Synthesize your insights

Science Partnerships Enabling Rapid Response (SPERR) Theo co-led the SPERR project, which analyzed the obstacles to effective scientific collaboration during the Deepwater Horizon (DWH) oil spill. Her team used a blend of human-centered design and systems analysis to identify ways to enable scientific exchange between government agency responders and non-governmental scientists from multiple disciplines – before and during large-scale crises like the 2005 oil spill. You’ll find examples from this project throughout the guide that show you exactly how this is applied in the real world.

4

PART 1

Decide who to interview Take 30 minutes to list all stakeholder groups that might be involved in the challenge you’re working on. Think of stakeholders who are directly affected and also those who are indirectly affected. It’s often useful to start very specific (i.e., naming individuals), and then get more general as needed (i.e., organizations or categories of people). Starting specific grounds your list of stakeholders in real stories, faces, and relationships. This helps you avoid making untested generalizations about entire groups or organizations, and can highlight potentially interesting differences or tensions within groups that otherwise seem monolithic. In the case of the SPERR project, we could have said “scientists” were one stakeholder group. But in the system around the DWH oil spill it was important for us to distinguish government agency scientists from academic scientists from private industry scientists. Each had very different goals, behaviors, incentives, and constraints from one another. For example, data-sharing and transparency is a fundamental norm among most academic scientists, while guarded protection of datasets is the norm for most industry scientists. Once you’ve listed specific individuals who represent affected and involved groups, then go broader. Which companies and organizations are involved? Government agencies? Nonprofits? Community organizations and movements? What about non-human stakeholders, like affected species or ecosystems? You may feel you don’t know all the stakeholders surrounding your focus challenge – that’s probably true, and that’s ok. During your interviews, you will learn about other stakeholders and about important differences within stakeholder groups (a la the “scientists” example).

5

Once you have this first draft list, it’s time to figure out who to interview within these groups. Here are four rules to guide you: 1. Go broad 2. Go deep 3. Go extreme 4. Keep iterating

Rule 1: Go broad Find people who are likely to see the whole system. Bird’s-eye stakeholders allow you to quickly get a sense of what is happening in a system and what the key dynamics are. You’ll learn a lot about the challenge quickly. An example of a bird’s-eye stakeholder group for the SPERR project was the conflict mediators who facilitated multi-stakeholder negotiations during and after the disaster response. We also treated senior leaders in federal government agencies (e.g., the U.S. Environmental Protection Agency) as bird’s-eye stakeholders, since they often had 20+ years of experience working from multiple senior positions within government and could quickly characterize relationship dynamics among government agencies and external institutions (e.g., oil companies). It’s important to remember, however, that every stakeholder’s perspective is biased and inherently limited. Your job as the interviewer is to seek out many perspectives, identify contradictions within a given stakeholder’s perspective and across multiple stakeholder perspectives, and then piece together something closer to the “real” story of what’s happening.

6

Rule 2: Go deep Find people who are as close to the challenge as possible. Ground-level stakeholders are important as they help us unpack the motivations and beliefs for each stakeholder group. An example of a ground-level stakeholder for the SPERR project was the emergency responders who managed the on-the-ground operations during the oil spill response in the Gulf. Another example was the academic scientists who wanted to contribute their expertise to the response effort, but ran into barriers. With both of these groups, we sought to understand their goals, what prevented them from achieving these goals, and how their achievements were rewarded. The interview approach for a ground-level stakeholder is similar to a typical “need-finding” interview that you would conduct in a traditional human-centered design project.

Rule 3: Go extreme Rather than trying to find people to interview who seem like the most representative of the greater population, look for people who are at the extreme ends of the behavior or context you’re looking to understand.

“When you speak with and observe extreme users, their needs are amplified and their work-arounds are often more notable.” – d.school

7

Other examples of “extreme” stakeholders include: • Movers & shakers. Who is extremely enthusiastic about your challenge? • Experts & mavens. Who has particularly extensive knowledge or expertise? • Connectors. Who has a large social network related to your challenge? • Decision-makers & influencers. Who holds unusually high power related to your challenge? In the SPERR project, one example of an extreme user was an academic research scientist who was highly passionate about connecting his research findings to the government’s oil spill response tactics. He was well-connected with government emergency response managers and had a high level of interpersonal trust with them – a relative anomaly. Because trust was a central theme we were interested in, we interviewed him multiple times to better understand how he was able to build those relationships and what challenges he experienced in maintaining them. This helped us identify the barriers that more “mainstream” research scientists would likely encounter.

Rule 4: Keep iterating If you’ve gotten this far, you have a great starting point. Don’t spend too much time trying to perfect your interview lists at the beginning because as you conduct interviews and learn more about the system, you’ll discover other stakeholder groups and actors that may provide valuable insights. Keep returning to your list and updating it as you deepen your understanding. By the end of the SPERR project, Theo’s team had interviewed 35 academic scientists, 12 NGO scientists, 5 oil industry scientists, and 50 government agency employees. That’s over 100 interviews. Don’t underestimate how many people you need to get at the root of the challenge and never assume one interview per stakeholder group will be sufficient. 8

PART 2

Develop an interview guide Take the time to build an interview guide before you start interviewing. You’ll likely have a core set of questions that you ask everyone you interview and then others that differ based on who you talk to. When you create questions, make sure to use open-ended questions (“Why do you think…?”) rather than yes/no questions. Scenario questions are also great (“If you had $10 million dollars…”) because they engage our brains in different ways that can lead to more honest, “real life” responses. Your interview approach will differ most on whether you’re interviewing a bird’seye stakeholder or a ground-level stakeholder...

Techniques for bird’s-eye stakeholders As you’re thinking about questions to ask bird’s-eye stakeholders, keep in mind that your goal is to paint a picture of the current system dynamics and key relationships among stakeholders. Try to come up with questions that get at: • Major players and influencers • Power dynamics (who has power, who doesn’t) • Problem frames (how do they articulate what the “problem” is) • Drivers of current outcomes (what do they think is maintaining the status quo)

9

Here’s a set of general interview questions you can adapt when creating your interview guide for bird’s-eye stakeholders: • What changes have occurred in this challenge over the last 10 years? • Who sets the rules? Who are the major influencers of what happens? Whose voices/opinions usually get left out or drowned out? • What changes and outcomes are you working toward? Which barriers prevent that from happening? • When you look at the path ahead, what are you most concerned about? What gives you hope and energy? • In your work on this challenge over X years, what has surprised you? • If you were in my position, what additional questions would you ask? • Who else would be important for us to talk to? Below are the bird’s-eye questions used in the DWH project and the types of insights the team was after for each question: • If you had a billion dollars, what two critical success components would you focus on to ensure effective response to oil spills in the future? (Getting at: what they think the root of the “problem” is) • What is preventing that ideal from being a reality? (Getting at: What constraints shape stakeholder behavior? What rules govern the system?) • If you were in my position, looking into this challenge, what additional questions would you be asking? Who is most critical to talk to? (Getting at: how they frame the problem; what power hierarchies exist within the system)

10

Techniques for ground-level stakeholders Each ground-level stakeholder sees a different part of the system. For this reason, it’s important to talk to a wide diversity of stakeholders and keep an eye out for larger themes that emerge. As you’re thinking about the questions to ask ground-level stakeholders, keep in mind that your goal is to deeply understand the motivations and constraints of each stakeholder group. Create questions that get at: • Rules and constraints • Rewards and motivations • Beliefs • Problem framing Here’s a set of general interview questions you can adapt when creating your interview guide for ground-level stakeholders: • Can you describe your day yesterday from the start to end? • Follow-up question: On a scale of 1-10, how ‘typical’ was that day for you? • What are the most rewarding parts of your job? What are the most frustrating? • Can you tell me about the last time you did X (key behavior of the challenge issue you are interested in)? • Whom do you interact with on a daily basis? • If you had $10 million dollars, what would you focus on to address the challenge? Why do you think those aren’t being focused on right now? • If you were in my position, what additional questions would you ask? • Who else would be important for us to talk to? Your questions for ground-level stakeholders will be very specific to your focus challenge, and the behaviors associated with that challenge. For example, the questions you ask wheat farmers about water pollution issues would be pretty

11

different from the questions you ask pediatricians about patient-doctor relationship challenges. In contrast, the questions we ask bird’s-eye stakeholders often look similar across challenges, since they are observing dynamics from a system level. Below are the ground-level questions used in the DWH project and the types of insights the team was after for each question: • What are the most rewarding parts of your job? The most frustrating? (Getting at: their motivations and their constraints) • During the spill, what did your typical day look like? Who did you interact with most frequently? What types of decisions did you need to make? (Getting at: What types of connectivity they had; Where they were in the power hierarchy) • Tell me about a time during the DWH spill when you had to make a decision, but weren’t sure what to do? How did you go about making that decision? (Getting at: Current problem-solving frameworks and resources) • Can you give me an example of success when you felt proud of your work? (Getting at: how are they rewarded and what motivates them)

12

PART 3

Conduct interviews Now that you’re armed with a list of stakeholders and questions to ask them, go find people and talk to them! Finding people who agree to talk will be much easier if you have a warm introduction. Make sure it is clear why you want to talk and what you intend to do with whatever you learn through the interviews. Ask for 45-60 minutes, and be respectful of people’s time.

Before the interview Bring along someone else to take notes, or get permission to record the interview. Only one person should be leading the interview, so that he or she can build rapport with the interviewee. In-person interviews are best because they allow you to capture non-verbal cues and information about the interviewee’s environment. Make sure to block out 20-30 minutes following the interview to immediately process and capture key insights with your notetaking partner.

13

During the interview Here are some general tips to help you be more successful during your interview: • Your interview guide is just that – a guide. Use it as a general outline and stay open to unexpected surprises and opportunities. • Whether it’s your first interview or your 100th, try to maintain a beginner’s mindset. Stay curious and open to alternative explanations. • Build an emotional connection with the interviewee and state your interest in their points of view. Draw out their thoughts, emotions, and underlying motivations. You want to really know how someone feels. • Ask open-ended questions, such as “Why do you think…?” or “What would you ask if you were me?” Follow the answer to a why question with another “Why?” to uncover root causes. • Ask people to share stories about success, failure, and times they felt proud of their work or their team. These can be windows into underlying values that are powerful shapers of behavior. • Look for contradictions between what people say and what they actually do. Take note of those, and gently probe people about them (if you feel comfortable). • Pay attention to the language people use. Write down exact phrases instead of summarizing or inserting your own language. • Leave room for silence. People will speak.

14

Toward the end of the interview Don’t be in a rush to leave. The end of the interview is a special time where people tend to relax and let their guard down. Always try to “finish” 5-10 minutes early and end with an open-ended question like, “Is there anything else that you want to share?” and see what comes up. Some of your most valuable insights may come after the interview is technically finished. Don’t forget to thank the person for spending time with you. The people you interview may be useful thought-partners and collaborators as you continue your analysis.

After the interview It’s best to reflect in a structured format to make it easier to compare notes and share with the rest of your team. Continue to Part 4 to learn how to best capture your interviews.

15

PART 4

Create an interview-capture process Ideally each member of your team will conduct multiple interviews to develop a more grounded and nuanced understanding of the challenge. It’s important to have a structured way to capture the insights and initial synthesis from each interview. Having team members thoroughly capture their individual interviews will make the team synthesis far more efficient and effective. When building your interview-capture process, it’s best to start with a question like “What were the most interesting insights you learned from this interview?” Capture these on post-its (or in a Google Doc if you aren’t in the same place) for all your team members to reflect on and reference. From there, we recommend organizing the capture process into three buckets:

Their persona • How would you describe this person in 3 words? • What were they motivated by? What are their goals? • What values and mindsets were hinted at?

Their relationships • What other stakeholders is this person connected to? • What value do they provide and/or receive from other stakeholders? • How do they perceive other stakeholders?

16

Their perception of the system • • • •

How does this stakeholder characterize “the problem”? What role do they play in the system? What resources do they control? What are the key dynamics in the system?

Google Forms is a great way to structure the interview-capture. As an example, here’s the form from the SPERR project. Your form doesn’t need to be this extensive – try out questions and see what works for you. A good interviewcapture form will feel like it’s helping you quickly reflect and extract deeper levels of insight from the conversation. The most important thing is that you do this within 24 hours of each interview. If you find that you are falling behind on capturing your insights and initial synthesis, you probably need to cut out a few of the questions. After capturing many interviews in the form, you can use Google Sheets or download it as an Excel file and quickly scan across categories to pick out patterns and contradictions that you and your team want to explore further.

17

PART 5

Synthesize your insights Before starting this step, make sure all interviews have been captured (through a Google Form or other method) and each team member has read them all. You’ll then work as a team to synthesize and develop initial insights about what is driving the behavior and current outcomes of the system.

Build stakeholder personas Identify the 8-10 stakeholders who seem most important to your analysis and create personas for each one. We recommend looking at four areas: • Motivations. What are the stated and unstated goals of the stakeholder? What unmet needs does this stakeholder have? • Beliefs. What mental models seem to be at play? What habits does this stakeholder fall into and which beliefs drive those habits? • Agency. Does the stakeholder perceive they have the ability to act? And do they have the knowledge/capabilities for those actions to have a reasonable chance of success? • Constraints. What constraints are preventing the stakeholder from achieving their stated motivations? What would they say is holding them back? Try to come up with a few key adjectives that succinctly describe the stakeholder as well as a few quotes that were especially telling.

18

Below is an example of a stakeholder persona from the SPERR project. Note the commentary on how the stakeholder perceives other groups at the bottom of the page. In this case, “local” means local to the DWH spill site (e.g., academic scientists in the coastal Gulf states).

Persona profile from the SPERR project 19

Analyze relationships between stakeholders Once you’ve built personas, you’ll then want to analyze the relationships between stakeholders. The goal of this step is to move beyond a focus on individual stakeholders and create a deeper understanding of how stakeholders interact with each other. Start this exercise by picking two stakeholders who interact regularly and consider these four areas: 1. Collaboration. How often do the stakeholders interact? What is the strength and nature of the relationship? What results from their collaboration (intentionally or unintentionally)? 2. Perceptions. How does one stakeholder group perceive another? What adjectives would they use to describe each other? Is that perception positive? Negative? Mixed? 3. Value flow. What value flows between the two stakeholders? Does it flow both ways? What value could flow between them? 4. Power. If you had to distribute 10 “power” points among the two stakeholders, how much would each get? Is it balanced (e.g., 5 and 5)? Skewed (e.g., 8 and 2)? You don’t need to do this for every possible pair of stakeholders, but try to walk through this analysis for any pair of stakeholders that interacts regularly. Also consider analyzing stakeholder pairs where building or strengthening relationships might create new, desirable system behavior or reduce undesirable behavior. Using Kumu, you can then build a visual map of these interactions. Start by adding each stakeholder as an element and then drawing connections anywhere there is significant interaction or collaboration. You can use the profile of the connection in Kumu to add additional details from the four areas above and optionally decorate connections to highlight strong vs. weak connections, negative perceptions, or skewed power dynamics.

20

If you decide to include in your map those connections you’d potentially like to build or strengthen, make sure to distinguish them visually so people are aware those connections don’t yet exist. You may want to create two versions of the map, one that reflects the current state of the system and another that reflects the desired future state.

Summarize initial system insights Much of the synthesis until this point has focused on individual stakeholders or relationships between stakeholders. While this is critical to understanding why a given system behaves the way it does, it is important to also reflect on the larger patterns and structures that emerged across the various interviews. We’ve found it most effective to treat this as a brainstorming session that takes place after your team has completed the rest of the synthesis. The results of this step will serve as the starting point for you to begin building a systems map from. Find a large whiteboard (or a Google Doc) and follow these steps: 1. List enablers and inhibitors. What patterns and trends do you see that are supporting more desirable system behavior (enablers)? What patterns and trends are preventing more desirable system behavior (inhibitors)? An enabler for the SPERR project might be “expectation of data sharing and transparency” while an inhibitor might be “fear of litigation”. Try to list as many enablers and inhibitors as you can. 2. Group into common themes. Once you’ve generated a significant number of enablers and inhibitors (20-30), try to group them into themes. These themes help surface larger dynamics and often hint at possible feedback loops that may become part of your system map.

21

CLOSING

Next steps Congrats! If you’ve followed these steps, you now have far more insight into who’s involved in the system, what’s happening, and why it is happening. Your next step is to build a system map, ideally through an in-person convening of diverse stakeholders. Share the results of your synthesis (personas, stakeholder relationships, enablers and inhibitors) with anyone who will be involved in the mapping to broaden their perspective and jumpstart their understanding of the system. If you’d like to get started using Kumu, you can sign up for a free account by visiting https://kumu.io/join. Head over to the Kumu help docs to learn the basics. If you’re looking for further guidance on applying these tips within your organization, project, or community, reach out to Jeff Mohr ([email protected]) or Theo Gibbs ([email protected]).

22

ABOUT THE AUTHORS

Jeff Mohr Jeff is cofounder & CEO of Kumu, a powerful data visualization platform that combines system and network mapping to tackle complex issues. He works with changemakers across the globe on a range of topics including strategy, governance, systems mapping, and network analysis.

Theo Gibbs Theo is co-founder of Blue Heart, lecturer at the Stanford d.School, and previous Systems Design Strategist at ChangeLabs. She works at the intersection of systems science and human-centered design. At Blue Heart she is working to build the political power of communities most impacted by climate change in the U.S.

Thank you to Banny Banerjee, founder of Stanford ChangeLabs and creator of the Deep Change Method (DCM). The exercises and frameworks presented in this article draw from and are inspired by the DCM. Thank you also to Karen Grattan, Laura Kwong, and Anna Xu for providing valuable feedback and to Rob Ricigliano for continuing to innovate on how an effective systems practice can unlock the potential for deep and lasting impact.

23