This Pew Fund for Health and Human Services program is helping grantees assess their impact

Impact evaluation is the process by which nonprofits measure the effectiveness of their programs.

But many nonprofits don’t work in simple numbers — an art program, for instance, may aim to reduce loneliness in seniors, but that doesn’t necessarily translate to an increase in government savings that’s easy to communicate to funders.

The Pew Charitable Trusts’ Pew Fund for Health and Human Services in Philadelphia recently launched an evaluation capacity-building initiative for grantees to help them assess those hard-to-measure outcomes.

Thirty social services nonprofits joined the 15-month program in May to receive both group instruction and individualized coaching from the Fels Institute of Government’s ImpactED and McClanahan Associates Inc. on data tracking and evaluation best practices.

Frazierita Klasen, VP of Pew’s Philadelphia program, said the program was born of a recognition that although many of its grantees were doing impressive work, they weren’t always clear on what specific goals they were trying to accomplish, or how to best assess whether they were reaching the goals they had identified.

How do you measure something like social capital and relationship building?

“We’ve seen over the years that there’s a growing demand for accountability in the social sector and proof that the programs [being offered] are producing strong results,” she said; funders want to know their money is being used effectively. Plus, advances in technology have made data collection more accessible for smaller organizations.

“Pew has long emphasized outcomes,” Klasen said, but recognizes that the organizations it funds “are in different places of capacity to articulate the results of their services. We recognized, if they were going to do better in that, we needed to provide those resources.”

Pew grantees applied to join the program, and the size, scope and missions of the inaugural cohort’s participants vary greatly: ElderNet of Lower Merion and Narberth, for instance, is a tiny, volunteer-driven nonprofit serving low-income seniors, while the citywide Philabundance saw a 2017 operating budget of $52 million. That’s intentional, according to Klasen.

At the start of the program, Pew surveyed the grantees on their existing evaluation processes and skills, then grouped them into tiers according to capacity for improvement. These cohorts-within-a-cohort allow for more productive collaboration and lessons sharing, Klasen said. About 10 participants will also receive grants to support the purchase of new management systems and other technology.

Broad Street Ministry’s (BSM) director of its Hospitality Collaborative and evaluations, Brenna McGinnis, said her social services nonprofit applied to participate because it was in need of an updated evaluation program, including a theory of change (how well a program’s activities and its goals match up) and logic model.

Broad Street Ministry’s clothing closet. (Photo by Getty Images/Spencer Platt courtesy of Broad Street Ministry)

“We worked with the Scattergood Foundation to create an evaluation program in 2013, which was the first time we did anything like that,” she said. Since then, though, BSM, which served about 6,700 guests last year, has undergone a “huge transition.” Whereas at the time, it offered three mealtimes per week, it now offers five and has added services such as case management, workforce development and reentry help.

The program offers her an opportunity to learn how to communicate BSM’s impact beyond tangible outcomes such as number of meals served. There’s no clear metric for the organization’s ultimate goal for its guests, which is “helping people create relationships with one another” and build “social capital,” McGinnis said. “How do you measure something like that?”

McGinnis said she expects the Pew program to help BSM improve guests’ experience, improve its grant reporting, ensure that its mission is being met and increase its donor base. It’s all about “making data part of the culture here,” she said.

Whether there will be future cohorts depends on the program’s self-evaluation of its effectiveness, Klasen said: “This is very Pew-like. We want to learn from our own experience and learn what’s working.”

Here’s the full list of the first cohort’s participating nonprofits:

• Broad Street Ministry
• Career Wardrobe
• Center for Advocacy for the Rights and Interests of the Elderly (CARIE)
• Central Behavioral Health
• Community Learning Center
• Drueding Center
• Ed Snider Youth Hockey Foundation
• ElderNet of Lower Merion and Narberth
• Esperanza Health Center
• Face to Face
• Family Service Association of Bucks County
• Intercommunity Action
• Intercultural Family Services, Inc
• Joseph J. Peters Institute
• Lutheran Settlement House
• Maternal and Child Health Consortium
• Mural Arts Philadelphia
• NOVA
• Philabundance
• Philadelphia LISC
• Project HOME
• Senior Community Services
• SquashSmarts
• Support Center for Child Advocates
• Supportive Older Women’s Network
• Surrey Services for Seniors
• The Garage Community & Youth Center
• Women Against Abuse
• Women’s Opportunities Resource Center
• YouthBuild Philadelphia Charter School