I muttered and cursed on the subway the first time I read Paul Brest’s The Power of Theories of Change. It was the heyday of the Strategic Philanthropy movement, talk of measuring impact was everywhere, and Brest’s article seemed to sum up something essential to the movement. It was not just deeply, but triumphantly wrong.

Strategic Philanthropy may have lost some steam since Brest’s clarion call for more rigorous impact evaluation, but the era of Big Data is only beginning; calls for social change nonprofits to gather proof of their effectiveness remain constant.

The article was, among other things, optimistic. ‘Theories of change’, in Brest’s view, should be measurable, and when they appear to succeed, replicated, and measured again at scale. This is power, indeed.

Photo credit: Unsplash

I am a nonprofit geek, and I love getting into the mud to pull together a theory of change. It is fascinating and exciting work. The results can be a powerful decision-making tool for NGO leaders facing complex problems, competing demands, and fraught choices. Theories of change provide a north star for avoiding mission creep, or for making decisions to change course in tumultuous times. And they are a clear and compelling way to tell the world what your organization does.

But there is one more thing to know about your theory of change: It is wrong.

Theories simplify complex phenomena to gain explanatory power, and the simplifications eventually catch up to us. All theories fail. Even the Standard Model of physics has holes that resist patching; eventually, a new theory will capture the facts that it does not. Getting there is the work of thousands of physicists working perhaps millions of hours over many years. Theories of change, by contrast, are often developed in a conference room over a week-long retreat.

Looking under the hood, the moving parts of our theories of change are alarmingly fragile.

Typically, theories of change are one part descriptive, telling a story about the root causes of a problem, and one part prescriptive, offering a set of steps to solve it.

Both sides are a minefield. On the descriptive side, causality is complex and elusive: even with a great deal of scholarship, consensus around the root causes of most social problems is thin and subject to the strong winds of politics. Rarely do nonprofit strategic planners have access to the latest social science, nor time to digest and adjudicate between competing theories. It is folly to think that we theorize with a complete understanding of the problem. We always risk offering a cure for the wrong diagnosis.

These difficulties are mirrored on the prescriptive side: for trenchant problems, there is no clear answer to how they may be solved. The task is dizzying: designing effective interventions, anticipating and ameliorating resistance, accounting for unintended consequences – these are separately very difficult and together well-nigh impossible. Even if you have the right diagnosis, every step forward offers countless, unpredictable opportunities to go astray. All the more so when we consider that there are many other actors working on the same issue, some pushing against us, others alongside but with strategies that conflict with or even nullify ours.

I am not sure there is a solution to the challenges on either side. We might wait for social scientists to agree to a descriptive theory. Or wait to build consensus among social justice actors around a prescriptive approach. We ought not hold our breath.

This is the nature of social change. Brest offered the example of the Knowledge Is Power Program, a charter school with demonstrated results in improving educational outcomes (as measured by test results). But as hard as teaching is, this is a relatively modest problem. Unlike direct services approaches that identify a surface problem and apply resources to meet those needs, social change organizations look upstream, and try to address root causes. These problems are intractable, because of their complex causes, but also because it is difficult to change minds on a societal scale. They can be broken down into smaller problems, but tend to shatter as they do – is educational attainment just what happens in the classroom, or must we tackle housing discrimination, food insecurity, and declining wages, too? A theory that encompasses all factors will produce an infinite logical framework.

As funders, the goal should be to get as many hands on the board as you can, pushing in the desired direction. The right theory is the wrong measure of future success. A large number of aligned but wrong theories is how we eventually arrive.

Acting on the theories we have is risky. But insisting that they be measurable brings even thornier problems.

Brest thinks of a theory of change like a scientific hypothesis, one subject to confirmation by experiment. But as in quantum physics, measurement sometimes alters the outcome.

No single organization moves society in a new direction; change is brought by many advocates, and other unpredictable factors like news events or even television shows. Do we ask grantees to measure just their contribution, and if so, how are they supposed to isolate it? Or do we expect them to show the impact of the full range of causal factors affecting the outcome?

Impact measurement is resource intensive, if it is possible at all. And in our sector as everywhere, resources are not shared equally.

Brest’s example involves a randomized control trial. If this were to be a general requirement, two results seem obvious. The first is that philanthropy will privilege larger, well-resourced organizations over their smaller, scrappier counterparts. Along with issues of fairness and equity, this would also stifle innovation by leaving many promising approaches unfunded for lack of monitoring and evaluation capacity.

Second, It will lead to a failure of ambition. Inevitably, grantees will feel pressure to stick to goals they can demonstrably meet. This would shift the focus from bold efforts to tackle wicked problems to cautious strategies that will probably succeed but may not produce meaningful change. It is well and good to say that goals should be SMART (specific, measurable, achievable, relevant, and time-bound), but the problems we face often seem insuperable, require decades-long engagement, and cannot be solved by one organization or from one angle alone. Our goals must be Relevant, they may be Specific, and one hopes they are Achievable, but few are Measurable, and none are Time-bound.

Theories of change are unreliable, but every social change NGO needs one – indeed, has one whether it is explicit or not. The key is to understand their limits. This is easy from the front lines, where experience teaches that, to paraphrase, no plan survives contact with the problem. Or as Eisenhower rephrased it, “I have always found that plans are useless, but planning is indispensable.”

The problem comes when we put too much faith in the theories we make. If our theories are apt to go awry, and measurement introduces distortions that undermine effectiveness, how is a philanthropist to be strategic?

First, we must give up the hubris of thinking that we can measure all things. There is no map for the complex, interconnected, contested and controversial injustices we decry. And we won’t herd all the cats onto one path. But change will happen, as it always does. Change is blind, the result of many different actors, many with incompatible interests and strategies pushing in every opposing direction.

Change is not a logframe but a Ouija Board. It is random because of the complexity of inputs, but hopefully intelligible in the end. As funders, the goal should be to get as many hands on the board as you can, pushing in the desired direction. The right theory is the wrong measure of future success. A large number of aligned but wrong theories is how we eventually arrive.

The way to be strategic about your philanthropy is to not to have a theory, but rather to have a geography of theory of change. Look at the field and map the plausible theories of change in play. Then, as far as possible, fund all of them.

That is how social change is won.

Devon Kearney has worked with human rights and social justice nonprofits around the world for more than 20 years.