Philanthropy can be made to measure

By Gara LaMarche

The philanthropic world, poked and prodded by a wave of new donors fresh from success in the business world, is grappling with the issue of evaluation. How do we know that grants or, as they are now often called, reflecting the influence of the profit-making sector, “investments” are making an impact?

Evaluation in some form is something every organisation has to think about and plan. Whether it’s reaching customers, losing weight or teaching kids to read, we accept the fact that you don’t know how you are doing if you don’t have measurement. Why should foundations and charities be any different?

I’ve worked in philanthropy for a dozen years, first at George Soros’s Open Society Institute, which took a more intuitive approach to assessment, and now at The Atlantic Philanthropies, which is more data-driven indeed, it is one of the relatively few foundations with an in-house evaluation unit.

These experiences, following many years of leading non-profits and acting as a grant-seeker, prompt me to offer advice to people who are considering starting or expanding their philanthropy, but are unsure of how to get the most bang for their buck. Donors successful in business want answers about impact, and while traditional philanthropy needs to help them understand the ways in which social initiatives are often quite different from profit-making ventures, it needs to be responsive to these legitimate concerns. Traditional philanthropy also needs to raise its own game in this respect.

• Evaluation should be based on a shared understanding of what is important to measure and learn. The organisation seeking funding should be asked to set forth what it thinks constitutes success, how it plans to measure it, and what it needs to get it done. Evaluation is a learning tool for the organisation and the funder, not a stick with which to beat grantees. This holds true for almost all measurement efforts, from after-school programmes to your co-worker’s performance review. If evaluation is coupled with punishment, fear will overwhelm learning.

• Doing this correctly takes money. When groups have to choose between providing services and devoting resources to assessment, they often give the latter short shrift. Funders should recognise and support their grantees in their efforts to learn what works.

• Evaluation should measure only what is important. Data should never be collected for the sake of it. The “metrics” obsession that has overtaken some funders has not always recognised this. Funders should never make grantees jump through hoops, distracting them from their core mission and costing valuable staff time, for reporting on trivial things. And there is nothing more demoralising, from the grantee’s perspective, than doing all this paperwork only to have it ignored.

• Evaluation should never be used to outsource judgment. Use it to inform your judgment, and then stand behind it. Evaluation is a tool in the quest for effectiveness in everything from helping kids make a successful transition from middle school to stopping the Bush administration’s embrace of torture. But it is not a Magic 8-Ball that will tell you what to do.

• Both funders and the organisations they support need more humility about cause and effect. Organisations working for social or policy change should understand that no significant change was brought about by one organisation working alone. Funders should acknowledge that the tendency of organisations to claim disproportionate credit for some policy advance has a lot to do with their need to impress funders. And funders, as well, are too often quick to claim credit for things that others had a hand in. When a neighbourhood turns round or climate change is reversed, your $50,000 grant will have played a role, but probably not a pivotal role that can be isolated with scientific precision.

• Donors should think about engagement in policy and advocacy work as well as investments in service organisations. All the private funding for after-school and arts programmes, community development, legal and medical clinics, is dwarfed by the potential of democratic government to invest in broad community needs. Advocacy can help shape the flow of those investments, supporting programmes and practices that government can learn from.

• Finally, the most important thing: start with what you believe. If you have a passion about ending the death penalty or the isolation of older people whatever it is find a way to advance it first and worry about how to measure it second.