June's Topic: Outputs vs Outcomes vs Impact
After coming to understand that evaluation is an ecosystem of many moving subsystems and parts, the next foundational concept that nonprofits must wrangle with is the distinction between outputs, outcomes, and impact. After all, how can an organization measure outcomes and impacts (or even create a strategic plan) if it doens’t understand what these things are?
Simply put:
Outputs are measures of amount of services/activities. “100 people attended classes,” “250 volunteers recruited,” “$2.1M raised,” “800 people received free lunches.”
Outcomes are the CHANGE wrought on individual service recipients/constituents by the services provided. Outcomes almost always fall into one of five categories: a change in skills, knowledge, behavior, beliefs, or values. Examples of outcomes might be “50 people increased their understanding of what constitutes a nutritious meal,” “1,200 people fulfilled a pledge to stop smoking for 30 days,” “200 people increased their emergency savings by 50% during the program,” and even “90% of participants continued saving 50% more than prior to program participation for 1 year after the program ended.” Outcome are usually the achievement of strategic planning objectives and/or the organization’s mission.
Impact is the cumulative change in the community/constituency being served as a result of the organization’s services. Impact is things like “the neighborhood unemployment rate went down by 10%,” “the city’s median household income went up by 25%,” “calls for services for stray/feral cats to the county animal control officer went down by 90%, saving the county $400,000.” Impact is usually the fulfillment of the organization’s vision.
All too often, organizations talk about “outcomes” and “impact” and what they are really referring to is outputs (or worse, objectives!). When talking about results, it’s easy to identify an ouput versus an outcome or impact by simply asking “what changed for people (outcome) / the community (impact) [as a result of our services]?” Note that here “community” can mean a group of people or it can mean a physical space, depending on your service target.
If you provide free, hot breakfast for 100 school-aged children - is that an output or an outcome or an impact?
Well, what changed as a result of that breakfast? Generally, providing the breakfast is so that kids are not hungry; specifically, that they are not hungry during the school day as such hunger will lead to lower school performance due to fatigue, stomach pain, inability to concentrate, poor memory, anxiety, and stress. So what we are saying is that the reason for providing the breakfast to children is to increase their school performance. Increased school performance is the OUTCOME - it’s the change we are hoping to achieve. Providing free, hot breakfast to 100 kids is not a change. It’s an activity/service, and, as such, the “100 kids received free hot breakfast” is an OUTPUT. It’s volume of services provided.
In this example, what would be the impact then? Well, impact would be the overall change in the population/community as a result of the services. Since the outcome is to increase educational performance/attainment by children suffering from food insecurity / hunger, the impact would be if the overall performance of the school district went up - increase in on-time advancement and graduation rates, increase in mandatory assessment scores for the school/district, reduction in disciplinary rates or behavioral issues or percent of children with an IEP/requiring additional supports in school. Think of outcomes as the benefit to individuals (note: individuals do not have to be people; individuals can be animals, a physical space, or inanimate objects) and impact as the benefit to society (and, again, society doesn’t necessarily mean human beings).
Now, what makes this all the muddier is that there are goals, objectives, strategies, and activities brought into the strategic planning mix. All too often, I see strategic plans with “outcomes” that are really objectives. Objectives are mini-steps to achieving goals. If the goal is to reduce the stress and trauma of childhood hunger and food insecurity, objectives are how you are going to achieve that goal - it’s the action plan. One action (objective) we might take, then, would be to provide free, hot breakfast to school-aged children. But, alternatively (or additionally!), we might provide:
free, hot lunch,
a family food pantry,
a soup kitchen,
supplemental food and nutrition cash benefits to families,
nutrition counseling classes to parents, and/or
increase the amount of fresh and organic produce and meat available in the community.
I’m sure you can think of even more alternative solutions! Basically, there are many strategies we could use to achieve our goal, and the specific strategies and the breakdown of those strategies are objectives.
Often, when I work with organizations around crafting outcomes, they are tripped up by how easy/simple/straightforward the outcome is. Outcomes don’t have to be complicated, complex, or hard. If your goal is to reduce the feral cat population because feral cats have short, unhappy lives then the expected outcome of your services is to increase the quality of life of homeless cats and/or to increase the average age/lifespan of homeless/feral cats. If the goal is to reduce the feral cat population because they are costing the town money in terms of calls for service to the animal control officer, then the outcome is that calls for service go down (as a result of educating citizens not to call animal control about the feral cats or as a result of changing people’s attitudes about the value of feral cats or as a result of reducing the number of nuisance issues caused by feral cats) and the # of dollars saved by the reduction in such calls. There should be a direct 1:1 correlation between the goal, action taken, and the result (outcome), and, as such, the outcome should be pretty obvious and straightforward. All too often nonprofits psyche themselves out accepting the obvious result as an outcome, thinking that it’s too simplistic.
This is, of course, just the tip of the iceberg when it comes to understanding outputs vs outcomes vs impact. We’ll be diving deeper into this topic throughout the month with my weekly “Ten Minute Nonprofit Evaluation Tips” videos, released every Tuesday (all of which are a bit over 10 minutes this month because there’s a lot to talk about!). Make sure you’re subscribed to this blog to recieve notice of new videos.
I’m kicking the supplemental video series off this week with “When is an ouput also an outcome?”
And feel free to join the conversation - leave a comment or question, share a best practice or lesson learned, and interact with other nonprofit evaluators in the comments section below.
Happy evaluating!
—Terri