Designing Process and Outcome Evaluations for Non-Profit Programs

By: Monika Lalezarzadeh

Presenter:

This year, New York Law School hosted the event “From Poverty to Opportunity”, a symposium marking the 50th anniversary of the Economic Opportunity Act. In collaboration with the Department of Youth and Community Development (DYCD), this symposium was part of an ongoing series of statewide public events, training programs and education seminars presented by the New York State Community Action Association (NYSCAA) to discuss anti-poverty strategies and best practices from the field.

One of the many workshops and panels that were conducted during the day- long symposium was, “Designing Process and Outcome Evaluations for Programs”. This session introduced participants to a framework for thinking about and developing outcome evaluation in ways that demonstrate impact, and contribute to continuous organizational improvement within and amongst non-profit organizations.

Speaker Randall Quan, Senior Partner at Community Resource Exchange (CRE), discussed overarching issues that plague some non-profit organizations: What does a successful outcome for a non-profit organization look like, and how do the non-profit organizations achieve such successful outcomes? Randall Quan began answering this broad question by discussing dysfunctional practices of some non-profit organizations. First, because of the charitable nature of their work, some non-profit organizations have trouble saying “no” to certain missions that are asked of them. It is also difficult for some organizations to stay focused on their target population and mission. It is important for each organization to know its target “niche” and attain successful outcomes with its target mission, rather than divert its resources towards causes and populations that do not fulfill its mission.

According to Quan, another dysfunctional practice of some non-profit organizations is that there is too much of a focus on service and not enough of a focus on change. For example, many non-profit organizations (such as soup kitchens or pantry organizations) offer food to those who are hungry and cannot afford this basic need. However, some of these organizations are not figuring out why these people are hungry or why they are coming to this soup kitchen in the first place. It speaks to the age-old saying, “give a man a fish and feed him for a day, but teach a man to fish and feed him for a lifetime”. Are these non-profit organizations looking to provide services (such as food) or change (such as providing ways to combat hunger)? If an organization is too service-oriented then it may just be providing an ephemeral solution to a long-lasting and more complex issue.

In addition, too often non-profit organizations may neglect data, be it data collected internally about their own operations or external data research from third party sources. Internal and external data research helps non-profits target issues and solve them more efficiently and effectively. The workshop continued by pointing out that non-profit organizations are often managed by their contracts and grants rather than being managed by their missions and purposes.

What is the best way to overcome these dysfunctions? Randall Quan suggests adopting the “Theory of Change”. The “Theory of Change” provides a framework for non-profit organizations to combat organizational dysfunction, and ensure successful outcomes in accordance with their mission. This theory is laid out in five critical points:

1) Organizations must define their purpose or mission. Here, the organization must address why it exists in the first place by answering questions such as, “What is our organization’s reason to be?” and “What is our urgent social need that drives the organization?” Once such questions are answered, then the organization has a clear vision as to what they are looking to change in society.

2) Organizations must define a target population for whom their organization exists. There are two types of populations for whom non- profit organizations usually provide services. One is the target population and the other is the service population. The target population should be the organization’s first priority and the people for whom the organization was created. The service population should consist of second (and third) priorities, in which working with them would not take away from work that must be done with the target population. Still, service must be rendered to those in this second priority due to funder demands and political necessity. Non-profit organizations must remember to not redirect resources from their target population when working with the service population. This could lead the organization astray from its mission and purpose.

3) Organizations must define the change they aim to make. Organizations must specifically define the outcomes they wish to attain from the organization’s efforts. What is the promise that the organization is making to everyone in their target population? These outcomes can be observed, measured, and monitored. There are three levels of outcomes that every organization must outline. There is the long-term outcome, which is the organization’s “ideal goal”, such as aiming to end world hunger. There is an intermediate outcome, which is what the organization is directly responsible for and what the organization holds itself accountable for, such as opening up and managing soup kitchens. And then there are short-term outcomes, which is what the organization is actually working on at the moment.

4) Organizations must define the programs or services (strategies) that achieve such changes. Organizations must ask what are the best programs or services to attain the given outcomes they have defined.

5) Organizations must manage for effectiveness in getting to outcomes. An organization must use performance data and other data needed to manage its performance against its outcomes (or desired outcomes). This allows for organizations to learn from and make charges to their work in order to improve quality and outcomes.

For more information about CRE’s work please visit: www.crenyc.org.