Can We Measure Impact From Design?
Can social impact from design be measured? We asked leading practitioners working in the social impact realm to answer our challenge as part of our Measuring Impact series. What do you think? Let us know by adding your voice to the comments section or get in touch at firstname.lastname@example.org
YES WE CAN ….
My academic training was in economics and quantitative policy analysis, where the assumption was always that we could — and, in fact, had to — find ways to measure things, even if it meant settling for imperfect proxies. Economists are in the business of measuring things, after all. I’ve carried the conviction that “everything can be measured” with me as I’ve delved into the realm of social impact design.
There’s a lot at stake in answering this question. Suggesting that our social impact simply can’t be measured removes designers from an ongoing conversation in the social sector, a conversation with people who speak the language of measurement. It’s a language that often feels cold or impersonal — the opposite of excellent design. Further, it’s outside the traditional training and comfort zone of many designers. But refusing to adapt that language and its methods means losing an opportunity to influence the conversation, and the whole sector misses out. Designers’ deep knowledge of human behavior makes them uniquely suited to designing new and creative ways to track impact.
One of the wonderful things about design is its insistence not on whether something can be done, but rather on how we might do it. Measuring social impact is certainly difficult, but since when has that stopped designers from taking on a brief?
Kerry Brennan is an Organizational Designer at IDEO.org, where she designs experiences, processes, and resources to support innovation within social sector organizations.
NO, UNLESS ….
City of Detroit Housing and Revitalization Department
Can social impact from design be measured? Perhaps not. I used to believe that there was a way to effectively measure social impact: The original idea that led to SEED stemmed from the notion that USBGC’s LEED rating system could somehow be adapted to help improve social outcomes and enhance the quality of life for people living in challenged neighborhoods.
However, I now honestly have to admit that my 21-year old self had it slightly wrong. As well intentioned as I was at the time when SEED began to germinate, I had not yet dealt with the financial reality of building anything. I will always have a deep appreciation for the power of design, but design is effectively powerless without the resources to bring the built version of a designer’s vision to fruition. There will always be a tension between what the developer will spend and what the architect will design.
But there is a way that architects and designers can reverse the equation and actually use social impact measurement not only to validate their work, but get projects built as well. For example, if a developer is concerned about costs, an architect can offer potential strategies to obtain funding from the various philanthropic or government sources who do care about impact and measurement when awarding funds. If architects can leverage impact metrics to help developers find a broader set of investors, then the likelihood of getting great design work actually built will be much higher.
Prior to joining the City of Detroit’s Housing and Revitalization team, Kimberly was a Sheila C. Johnson Leadership Fellow within the Center for Public Leadership at Harvard’s John F. Kennedy School of Government.
NOT THE WAY YOU MIGHT THINK ….
Justin Garrett Moore
Executive Director, New York City Public Design Commission
Maya Angelou famously stated, “People will forget what you said, people will forget what you did, but people will never forget how you made them feel.” One might not immediately think of Angelou when talking about design and metrics, but her point highlights a critical measure that we should not diminish. How does our built environment make us feel? How do the changes in someone’s community affect them — do they still feel a part of it? These measures are hard to get into a database but they are critical for us to think about in the work of transforming the built environment for our social contexts and values.
In New York City, the de Blasio administration has made equity a defining value in its policies and decision-making, and it is focused on helping all New Yorkers, in every demographic group and all five boroughs to share in the opportunities and betterment of the city. It is important that design and design thinking be employed to develop the tools, measures, and approaches to better gauge and promote the impact that design has on social contexts, how places change, and how people feel about their neighborhoods, their parks, their streets, and their cities.
Justin Garrett Moore is an urban designer and the Executive Director of the New York City Public Design Commission. He has extensive experience in urban design and city planning—from large-scale urban systems, policies, and projects to grassroots and community-focused planning, design, and arts initiatives.
Often the biggest social impacts are unexpected. There are special moments of social impact on all of our projects, but sometimes they don’t appear until the end, until years afterwards, or you don’t hear about them until somebody tells you something anecdotal. I find it is these stories that resonate most when we talk about the value of our work, but it would be easy to pick holes in the numerical data we collect on the impact and therefore I question why we even do it in the first place.
We recently did a series of audio interviews at the end of a project with a client and members of the community. The client was excited how the process of being involved in a project to think about a big physical change in the neighbourhood had brought people together, fostered unexpected relationships, and made them start thinking about how they would collaborate to use a new community space. Through the client witnessing these conversations first hand, they saw value and are now an advocate for a codesign approach. This is not something we predicted we would achieve — it was not included in our brief and not something we can easily measure, but it is powerful and ultimately what will foster design that works for a community and create space for further social impact.
Catherine Greig is founder and director of make:good, a design and architecture studio involving people in shaping neighbourhood change. She champions meaningful local participation and top quality design.
AS LONG AS WE CAN LEARN FROM OTHERS ….
CEO, Creative Reaction Lab
Can social impact from design be measured? Yes. Do most designers know how? No. And, that’s the problem.
Discounting a few exceptions, emerging and practicing designers are not being taught how to determine research benchmarks, the necessity of pre- and post-testing, analysis of data, and so on. While social science, “hard science,” and non-profit management/social entrepreneurship programs integrate ethnographic research, measurement, and evaluation into their curriculum, the creative field only recently began to explore these topics through the creation of design research and social impact design specializations (primarily within top design schools). Not only is the ability to show impact a necessity within the social impact field (especially with the purpose of our work being the improvement of a social or community issue), it is pertinent for the design field overall.
Antionette Carroll is the President and CEO of Creative Reaction Lab, a social enterprise cultivating creative leadership to improve the human experience through immersive challenges. She is currently the president of AIGA Saint Louis and Chair of AIGA: The Professional Association of Design’s Diversity and Inclusion Task Force.
As designers, we often work from our “gut,” bringing design intuition to tangible and technical problems in search of ephemeral, intrinsic, and relational outcomes. We strive for improvements to quality of life, personal health, and social systems, yet rarely provide a mechanism for feedback from our clients and end users. It is quite curious how we assume our original good intentions align with the end products of our design process.
With tools of measurement in use by other fields and adaptable for our needs, we could develop a robust capability to measure the social impacts of good design. Through multidisciplinary collaborations, participation from project stakeholders, and cultural shifts within the profession, we could move beyond anecdotes to draw more concrete and comprehensive conclusions. While we see many social impact design-focused individuals and organizations making progress, this pursuit would be a new endeavor for most architects and designers – and a skill set that would require cultivation.
Creating a viable system of measurement would require agreement, on some level, within the profession on our values and priorities. Time would be a continual hurdle, as it often takes years for a project to move from the design process through to completion and sometimes decades for impacts to be felt by a community. While challenging, I believe measuring impact, including social aspects, is in our best interests as designers and citizens. Linking the tangible with the intangible, our investment in measuring social impacts is consistent with our desire to serve clients well and fosters our relevance within society as integral contributors to the built environment.
Sarah Gamble is a principal and architect at GO collaborative, an Austin, TX – based firm integrating elements of architecture, planning, public art, and community engagement to connect people with place. Gamble is a research fellow and adjunct faculty member at the University of Texas at Austin School of Architecture.
YES, WITH ENOUGH PATIENCE ….
Detroit Collaborative Design Center
The challenge and beauty of measuring public interest design is the far-reaching nature of our work. Meaningful impact (a.k.a. “success”) could mean one family has a home that meets their needs. Or it could mean that a neighborhood has a plan that represents their vision for sustainable growth. Or it could even mean that residents have access to the information they need to advocate for their community. The essence of this work is qualitative, which muddles measuring attempts but cannot be lost in attempts to establish metrics. The key is to develop a measuring system that also helps tell the bigger story about how design contributes to communities. This story is being told project by project and practice by practice but would benefit from greater sharing across the field. Developing a measuring system that includes the satisfaction of numbers as well as the richness of experience is much like the rest of our work: it requires a bit more time, collaborative thinking, and an emphasis on participation.
Ceara O’Leary is a Senior Design and Project Director at the Detroit Collaborative Design Center where she works on neighborhood revitalization projects at a range of scales. She is a former Enterprise Rose Architectural Fellow and previously worked for bcWorkshop in Brownsville, Texas and the Gulf Coast Community Design Studio in Biloxi, Mississippi.
Architect and Researcher
Yes, social impact from design can be measured.
However, as solutions-focused people, designers are often keen to start the next project rather than undertake evaluation. To address this, we must set up evaluation processes that allow us to learn things we care about. Too often evaluation is a tickbox exercise to satisfy a donor or for marketing purposes, and it tells us nothing about how to improve our practice. We may be correct in saying a school project has been successful because attendance rates have gone up, but that doesn’t tell us what things we did well and where we could improve.
Another obstacle is that few designers have been trained in evaluation. We may ‘feel’ what worked, but we must be carefully self-reflective because our assumptions are often coloured in ways we are not aware of. We should often collect qualitative information, but even as we do this, we need to reflect on who we are, how our presence influences the situation, and what methods to use.
Kate Ferguson is an architect and researcher who works on design and infrastructure projects as avenues for learning, empowerment and community strengthening. Her current PhD project involves working with teenagers in Sydney’s western suburbs to strengthen youth participation in the shaping of public spaces.
Measuring social impact in design is a notoriously hard thing. Who decides what and where the impact has been made? Do photographs of mud brick buildings with smiling multicultural faces constitute social design? I would argue not necessarily. Measurement must be done by systematically consulting the key stakeholders (the project end-user, the community in which the project lives, the architect and the builder, and so on) in each design project at least one year after the project has been finished. This way we can begin to avoid having our projects up as the content of the newest glossy photograph of a project that has not really been lived in or tested by its users, and, therefore, cannot really be considered a success at all.
Esther Charlesworth is a Professor in Architecture and Academic Director of the Master of Disaster, Design and Development (MoDDD) at RMIT University, Melbourne.
Civic Projects LLC
It difficult to quantify the value of human and social capital and subsequent growth and development, but one way is to use economic tools. As economics does, we need to measure both the current conditions and the changes that occur as a result of the design and we need both the tools and the funding to measure it. These are not short-term metrics — it is inherently difficult to measure social impact because change occurs slowly and happens over long periods of time. While grants and other funding mechanisms in the arts and architecture have historically been tied to short timelines, this is changing.
While it’s not easy to figure out what to begin to measure, some examples of potential metrics include looking at revenue and taxes collected in an area before and after project implementation, educational attainment, and neighborhood GDP growth. Has the project led to further growth in a neighborhood? Has human capital been developed? There are even projects such as the rebirth of the Livernois Corridor in Detroit and the development of the Revolve Detroit Guidebook that have successfully measured variables such as these. It’s possible — we just need the tools and support.
Monica Chadha is a LEED certified, licensed architect who has been practicing for over 20 years. She is the Founder of Civic Projects LLC, focusing on the design, building and engagement for community led projects.
Deanna Van Buren
Designing Justice Designing Spaces
First we cannot do this alone — we need partnership. We need to work across disciplines with researchers who understand these methodologies so we can understand them ourselves. Second, I believe the social impact from design can be measured only when designers, program providers, institutional partners, operational staff, and all stakeholders are committed to doing the project while planning for, and following through with evaluation. The places and projects we design take so much time and money to complete that sometimes it’s easy to put down the evaluation baton — it takes a great deal of commitment from all parties to follow through.
Lastly, beyond partnership and commitment, we need bold and innovative funders. If grantmakers and foundations require proof of impact, they need to provide both the financial support to build and iterate prototypes as well as the long term funding to evaluate them. The social problems we face are immense and if we want good design to play a role in the solution we have to stay the course with close collaborations and partners who are open to exploring the unknown.
Deanna is the co-founder and design director of Designing Justice Designing Spaces. She is a recent awardee of the Rauschenberg Artist as Activist grant, which she is using to develop a prototype mobile resource village, and the founder and principal of FOURM design studio.
SCALEAfrica + SCALEStudio
Before we ask “can social impact be measured,” particularly as it relates to built projects in rural environments, we should ask whom does the data serve and what would it tell us of value. While well-designed environments most certainly enrich an inhabitant’s experience, numerical data struggles to capture that experience. If the data is not conveying a compelling narrative, its collection becomes a process that primarily serves donors’ interests and diminishes a small organization’s capacity.
More importantly, it can obscure the importance of building history, deep personal relationships, and trust with stakeholders. After eight years working in one community in Zambia, I have found listening and observing reveal the most telling indicators of a project’s success or failure – and that numbers wouldn’t tell us more than we can know by putting in the time. I can see how the buildings hold up, how they are used, what adaptations the community makes, and what skills the building team is mastering. I hear what everyone shares — good and bad. There is real value in and much to be gleaned from an organic, qualitative, human-centered process that unearths what numbers never could.
Erinn McGurn is a licensed architect and the Founder and Executive Director of SCALEAfrica + SCALEStudio, a for-benefit design firm designing and building green school and health infrastructure in sub-Saharan Africa & Asia.
Royal College of Art
We often assume that social impact manifests in tangible rather than tacit outcomes– in products, services, and technologies rather than whether people work, live, or feel better. And we’re impatient about the results too. Many entrepreneurs and social investors adopt a Tinder-like approach to social impact projects: Rather than get involved in something that requires a longer-term commitment before the results can be realized, they go for a left-swipe in search of something new if results don’t come quickly. In reality however, social impact is seldom the consequence of a ‘big-bang’ moment, but instead requires time to co-design, prototype, test, refine and share, before meaningful and lasting social impact can be achieved — or measured.
In the case of my students’ refugee wearable shelter, its ability to turn from a coat into a sleeping bag and into a tent means that while it might fail in one of these functions, it could still excel in the other two — there is more than one facet to be analyzed for impact. And there’s always an option to make people the metric by how a product, space, or service makes people feel or how it’s changed lives on an intrinsic level, and then crucially consider how it will continue to do so.
Dr Harriet Harriss is a Senior Tutor in Interior Design & leads the Architecture School PhD program at the Royal College of Art in London. Her teaching, research and writing are largely focused on pioneering new pedagogic models for design education.
IT’S COMPLICATED ….
Measurement is a bit of a deceitful thing, really. Anyone who’s ever been on a diet to lose weight or gain muscle has experienced the deceit of measurement, and all statisticians know that it’s easy to fudge individual metrics to fit a desired narrative. Even when you are doing the right thing and making progress, you can be easily deceived by measurement. You look at the kilos rising and miss the inches falling away, or focus on the body fat percentage remaining static and miss the strength gains. And in all the numbers, you might miss the fact that your skin looks clearer, you have more energy, your mood has improved and you feel better about yourself. But if you look at your goal, and look across all the measures available to you, quantitative and qualitative, they tell a story that is much harder to fudge or frame – you are either generally heading towards your desired outcome or you aren’t, and you can get a good sense of how quickly or effectively that’s happening.
When we talk about measuring social impact, we should be mindful of the human preference for simple and clear indicators. We need to recognize that the most powerful story of change – whether success or failure – is one that harnesses a range of indicators in concert, and is interpreted clearly and honestly by someone who is equipped with the contextual knowledge and skills to tell that story.
Steph Mellor works at the intersection between government, society and emerging technologies. With a combined background in sociology and internet sciences, Steph spends her time delivering small and large-scale public value projects for ThinkPlace – a global user-centered research and co-design consultancy.
Design Impact Group at Dalberg
One reason that the social impact of design is difficult to measure systematically is because truly impactful solutions work concurrently across multiple levels, and as such, they must be measured across all those levels as well. A well-designed solution addresses not only the product level issues that can be measured – such as number of users of an app that allows people to give feedback about services in a public clinic – but also the ecosystem level, such as whether the clinics can be allocated enough resources to function properly in the first place.
How you measure impact starts with how you measure the problem. While traditional indicators tend to focus on numbers, design research tools such as personas and journey maps highlight nuanced stories that statistics cannot describe. Similar techniques can be used to support impact measurement after an intervention, so as to provide a basis for comparison. However, while the tools you use may be the same across challenges, the stories you tell will be different each time, making them harder to collect systematically. Another difficulty is that stories and images are often seen as “soft” material, but combined with the “hard” statistics, they can illustrate causal pathways and shed additional insight into impact.
Priyanka Pathak is a Senior Designer at the Design Impact Group at Dalberg whose work focuses on co-designing and co-creating technologies for social good around the world, with a special focus on women’s issues, health, and education.
Texas Venture Labs
Measurement brings its own baggage. We live in an age where we measure our steps, the number of minutes of deep sleep, what we ate for dinner, clicks, likes, shares and systems and devices that measure these things for us in the palm of our hand. The overwhelming pressure and tendency is to measure more.
However, one of my greatest mentors introduced me to the concept that “better is better than more” — a plea for quality in an age of productivity. If we are to develop a system of social impact measurement, there are elements on which we first need alignment and validation: Who is our audience? What matters most to them and why? (This is often different from what matters most to us and it takes patience, discipline, and listening in order to get this right.) How do we communicate with our audience? Are there expert subgroups? When do we communicate with our audience?
These are all much more important questions than “what should we measure,” which is where many of us start. These questions force us to see ourselves through the eyes of our audience — and it is our audience that will judge our impact.
Ryan McKeeman is currently a Venture Partner at Texas Venture Labs in Austin, Texas. Prior to his work at Texas Venture Labs, Ryan was a sustainability consultant for 10 years specializing in developing social performance measures for Fortune 100 clients.
Designer and Urban Planner
I’d argue that social change can most successfully be quantified when a design has clear social goals from the beginning, and when the variables can be related in real numbers. Imagine a low-income family who can’t afford municipally-supplied heat. Good sustainable design could provide a home that harvests solar energy to heat itself, and therefore eliminates expensive energy bills for the family. In this case, the impact of good design can be measured by the number of low-income families with access to heat.
It’s important to remember, though, that not all important social issues can be so easily quantified: it’s relatively easy to measure people, uses, or changes in physical health, but it’s very difficult to measure a shift in emotional state or mental well-being. How can one be sure that something is happening because of the design, and not because of another economic, political, or social factor? Isolating the way a design impacts a community’s social fabric requires an intimate and comprehensive understanding of the cultural landscape. But while measuring any social condition is complex, and often foggy, it’s our responsibility as a new age of architects and designers to be confident nonetheless, and use data to show the power good design has to solve our trickiest social problems.
Marissa Feddema is a designer and urban planner focused on green technologies and projects with bold social and environmental agendas. She is currently part of Brooklyn-based design firm Cycle Architecture + Planning and independently consults on projects locally and internationally.
Design Specialist, A Community of Friends
In the same way that we can measure environmental or economic impacts, we can also measure social impacts from design. Unlike these other impacts, however, the metrics for social impacts are not as readily defined or universally applied. One of the biggest challenges preventing the adoption of a system of impact measurement is that each project generally defines its social impact in a different way based on the type and intent of the project. A permanent supportive housing development in South Los Angeles, for instance, will focus on a different set of measures than a clinic in rural Mississippi.
The SEED Evaluation and Certification framework attempts to capture these differences while being relevant to a wide range of design projects. Within the framework it remains the imperative of the project team to identify the intended impacts and metrics with which to evaluate them. The risk in this approach, however, is that by allowing project teams to define their own impacts, they have the freedom to focus solely on achievable, positive outcomes and ignore others that could potentially be negative.
Using the environmental model as an example, if a building project is designed to be zero net energy but uses excessive water, has poor indoor air quality, incorporates toxic materials, and destroys acres of old-growth forest in the process, it would probably not be considered an environmentally sustainable building. Simply because a project sets out with good intentions, or specific social outcomes in mind, it is not inherently equitable or socially responsible. The question remains as to whether there are measures of social impact that are universal, and essential, to all projects in order for them be considered socially sustainable.
Brita Carlson is a Project Manager, Design Specialist, and Enterprise Rose Architectural Fellow with A Community of Friends, a leading developer of permanent supportive housing in Los Angeles, CA.
Iowa State University
Yes, we can and do measure social change in the design profession. However, there are two barriers currently facing the design profession: the message we are communicating, and a lack of resources dedicated to the evaluation of our work. If we continue to promote the idea that we don’t have enough evidence-based research in design, those outside of our profession are unlikely to be inclined to collaborate on projects focused on creating social change. It is imperative that we shift the dialogue within and outside of the design profession from bemoaning what we lack to proudly acknowledging the many great studies that have been published that promote the benefits of improved environments and serve as the root for social change.
Evidence-based research examining social change is of particular interest to the environmental justice movement. If it was your home next to a landfill or you who was confined to a small lifeless room for much of your day, would you feel motivated to make positive changes to improve your health and the health of your family and community? We need to make the effects of those injustices vivid enough so as to make environmental justice an integral part of every project. We can do this by providing clear evidence of how the environment affects us, and what we gain when we place more value on our health, well-being and ability to make positive choices.
Julie Stevens is an assistant professor in Iowa State University’s department of landscape architecture where she teaches design studios and plant related courses. Along with ISU students, she developed a garden curriculum and committee structure for offenders at the Iowa Correctional Institution for Women,winning the 2015 ASLA Award of Excellence in Community Service.
Arts Institute Executive Director
Social impact from design is measurable, but because designers aren’t fluent in social scientists’ language (and vice versa), the metrics are literally less recognizable – we don’t see each others’ symbols. A disconnect in methodologies and languages, combined with the unquestioned pedestal on which many funders place science and quantitative data, rather than design and qualitative data, has caused designers to doubt the ways in which we express the positive social impact we measure.
There are hundreds of metrics that researchers around the country are using to detail the symptoms of inequity. Architects and designers depend on the health or education industries to conduct studies that ‘prove’ that the quality of our environment impacts our mental, physical, and cognitive health, and to verify, for instance, that environmental stress and poor indoor air quality can contribute to low reading proficiency. These are helpful tools, yet in Public Interest Design we already know that a healthy environment created by good design and in deep engagement with community positively mitigates dysfunctional systems and poor health behaviors. We constantly educate our clients, funders, and city councilors to respect this knowledge and experience, and yet we rely on non-designers to translate it through scientific research.
I led a project that helped to revitalize a community’s historic core and communal traditions as well as create a more beautiful, culturally authentic environment. Walking there outside at sunset, one can hear children playing and smell dinners cooking in a community where both activities had previously been absent. No statistics, data, or evidence-based methodology will make any of that truer, more valuable, or more impactful.
Jamie Blosser, Executive Director of the Santa Fe Art Institute, works to creatively address equity in the built environment through critical community dialogue. She is the founder of Sustainable Native Communities Collaborative, a forum for peer networking, advocacy, and education supporting culturally appropriate and sustainable development in American Indian communities.
Taking into consideration that social impact in developing countries is predominantly being measured on per-capita economic growth, it is clear why there is an unhealthy obsession with numbers by organisations working in this space, as it directly reflects donor demands and power dynamics. As we move into a different era, organisational architectures and mindsets require a more balanced and blended methodological approach to defining human need and measuring impact in order to remain sustainable – for both these organisations and those people they seek to serve.
The current nature of funding, dominated as it is by linear frameworks, static outputs and quantitative measures, understandably drives donors to invest in development initiatives which generate predictable and tangible returns on the dollar. Although the current way of operating is argued by some to be working, it is not working as well as it could. For some time now, there have been growing concerns about the relevance of indicators used to measure social impact in developing countries, in particular narrow measures of economic progress based on GDP figures.
No doubt having numerical and defensible measures of success is critical for the continued legitimacy of these initiatives, but what if an over-emphasis on this approach is coming at the expense of human dignity and wellbeing? The conclusion of the 2009 Stiglitz-Sen-Fitoussi Commission Report proposed that those organisations concerned with genuine human-centered development need to shift their focus from narrow measures of economic progress to broader measures of human well being.
Ledia Andrawes is program director at ThinkPlace Foundation, where she has spearheaded ThinkPlace’s international development work in sub-Saharan Africa through the creation of innovative products, services and strategies in healthcare, agriculture, renewable energy and various other fields.