Measuring Impact – Introduction to the Series
For all our recent talk of designing for flux, one thing is certain: Designers who pursue social impact aren’t in it for the money. Ask people in most other professions why they do what they do and eventually they’ll most likely admit that it pays the bills, or they really like the look of that shiny new car. There’s nothing wrong with such motivations of course, but the efforts of everyone from oil companies to investment bankers to reframe their industries as doing noble service to their fellow humans are hard to take at face value. The sincerity of the impact designer, on the other hand, is not in doubt. However difficult and frustrating it may be to articulate the reason they get up in the morning (and often work long into the night), dollars and cents aren’t what count. Impact designers don’t primarily want to make money; they want to make a difference, a better world.
But how do we know we’re making a difference, or making the world any better? The pursuit of money may be morally ambiguous, but it is stubbornly arithmetic — you’re either making it or you’re not, and there’s an easy way to keep score. Conversely, the pursuit of impact is quite definitely ethically virtuous (in intent anyway), but its practical effect is ambiguous as can be. And even if one could clearly measure an outcome, “impact” is a far more complex and tricky concept. What if that outcome seems to be the positive impact we strive for, but turns out to be disastrous or destructive, whether in the long-term or in some other way we can’t discern? Our challenges are increasingly so complex that their solutions defy simple measurement — wicked problems snicker at straightforward metrics.
Evaluation is also power. The definition of success reflects the values and priorities of those who define it. Who decides what gets measured, who gets to do the measuring, and how resources will be distributed based on those measurements — these are all structural and political questions as important as who gets to design, and for (or with) whom. As in quantum mechanics, measurement itself can define and even distort the outcome as well as the substance. Rigid frameworks may obscure the diversity of circumstances and efforts of a given project and and even lead us to “designing to the test,” but without standardization we run the risk of getting lost in a chaos of subjective outputs, with no firm basis from which to decide what approaches are more effective.
If there’s one metric the impact design field is never short on it’s the number of dilemmas like this one. But it’s crucial precisely because we need to find a way to show that our approach works without hampering or destroying the work itself. Perhaps the greatest obstacle to impact design gaining traction as a profession and field of practice is the difficulty we have demonstrating the actual impact of our designs. Funding dollars will flow where positive effects can be quantified — metrics means money, and credibility, and, so, greater impact.
In this series, we explore the possibilities and quandaries of striving to measure what we do, from Acumen’s “Lean Data” model to gauging the effects of massive multi-player immersive gaming worlds on teaching social change principles, to examining how all this evaluation feels for its supposed beneficiaries. Our hope is that debate and discussion around impact metrics can serve to move the field forward with an understanding of metrics that’s empowering and ethical, and that gives us enough certainty to do better while instilling enough humility to recognize that we can always know more. That won’t change why we get up in the morning, but it might mean our days are better spent, and just might help the world wake up to the power of design to create a better future for us all.