When Ken Berger spoke at the Valuing Impact conference in London last May, he caused quite a stir by suggesting that a one-dimensional focus on non-profit finances, if not supplemented by other information, can lead a donor/funder to make the wrong decision about which organization to support and that an overemphasis on overheads is misleading. The surprise was due to the fact that many have criticized Charity Navigator in the past for precisely that. Alliance invited Ken Berger and Robert Penna to talk about how they are overhauling Charity Navigator’s ratings system to reflect accountability and – most importantly – outcomes as well as financial performance.
We have always believed in the importance of these things, as the narrative of a number of our website pages shows. We now realize that many were not reading the narrative. Charity Navigator is the largest evaluator of non-profits in the USA and rates roughly 5,500 non-profits on its website, on a scale of zero to four stars based on financial ratios ranging from administrative expense to working capital and programme expense growth.
However, we are changing this system so that, even if you do not read the narrative, you will be much more likely to make a well-informed social investment decision.
Three dimensions, not one
As we have been saying for some time now, we believe three dimensions must be considered by social investors:
- Financial health Does the organization have the financial strength to survive in good times and bad? Is the overhead not at the extreme end of the continuum?
- Accountability Does the organization have ethical practices, good governance and transparency? Is it accountable to its constituents?
- Outcomes Does it have outcome indicators that measure what matters, and does it strive to achieve meaningful and lasting change in the lives of the people it serves?
To answer these questions, the zero to four star system will change in two ways. First, it will take account of all three dimensions, with outcomes getting the greatest emphasis. Second, while the system currently benchmarks non-profits according to how they compare to industry standard (zero stars means exceptionally poor, four stars exceptionally good), in the new rating system investment risk is likely to be the defining concept. In other words, a zero-star non-profit would be a very high-risk social investment and a four-star would be low risk. For example, if in the accountability review it is discovered that a non-profit is under investigation by a state agency for Medicaid fraud, the risk would increase.
Top priority for outcomes
As for outcomes, this dimension of our new rating system will probably be weighted the most heavily because it is – or should be – the purpose for which the organization exists.
What does risk mean in this connection? The risk is less that a charity might fail to achieve an audacious set of outcomes but rather that it might have no outcomes at all (and is therefore almost exclusively activity-based in its thinking and approach), that it has no real measurement of progress against the outcomes it has established, or that its chosen outcome indicators don’t reflect significant changes in its target group. In addition, if an organization consistently fails to make significant progress towards its outcome targets, we believe that this reflects either a problem in setting those targets or in its approach.
None the less, there are social investments where the potential gain is worth running a higher risk. For example, an organization working with repeat offenders has an inherently high element of risk because of the target population’s failure to break the pattern of behaviour that led it into trouble. However, the chance to help make these men and women functioning, productive members of society can be worth that risk for those with higher risk tolerance.
Making the changes
Since the May conference we have begun the process of change and convened an advisory panel of experts and non-profit leaders to help us. They all had to agree to three things to join the panel: (1) that a star system that rates non-profits on a macro scale like ours is an acceptable approach; (2) that the three dimensions noted are the appropriate components of that star system; and (3) (our favourite) that if they have a criticism of something we are doing they MUST suggest a concrete solution that meets (1) and (2)!
We hope to have prototypes for the three components ready at various stages of this year, with a view to getting the entire new rating system up and running by spring 2011. Some feel that we are setting ourselves an impossible task, given the complexity and scope of the non-profit sector, but we believe there are scalable tools that can cover 5,500 non-profits per year and at the same time distil the level of risk a social investor will bear.
Further, we believe it must be done. We know, just from the number of enquiries we now receive (over 3 million people a year come to our site), that the donor community is looking for some reliable way to inform its social investments. Incidents such as the Madoff scandal have underlined the critical need for openness and accountability regarding all potential investments, including charitable ones. With the growing focus on outcomes, we believe that our potential ability to help make sense of the claims of the largest of the roughly 1.9 million US non-profits will be a valuable contribution to all parties concerned.
Why this must be done
Many of those attending the Valuing Impact conference were excited by our plans and appreciated the fact that we had acknowledged the weaknesses of our existing system. What we have learned over our years in the field is that all the sector’s actors need to be transparent and accountable for their actions. This is one of the reasons why Charity Navigator recently signed up to a press release, along with GuideStar, Great Nonprofits, Philanthropedia and GiveWell, which was critical of overhead ratios.[1] While we don’t agree with everything the press release says (we think overhead does have a place in rating charities, but agree it should not be overstressed or the paramount consideration), we do concur with the fundamental truth that the most critical dimension in evaluating a non-profit has to do with achieving meaningful results.
Charity Navigator will not just talk the talk, we will walk the walk. We are about to go through our own theory of change process and develop outcome indicators for our own work, so we will be rating ourselves, too. And if we don’t get it right this time, we will keep trying until we do.
Finally, we want to explain why we are so passionate about all of this. We believe that this is a critical battle for the very soul of the non-profit sector. We must get past the idea that non-profits are too complex or unique to be measured. The non-profit sector must get its act together and make sure it is really helping to provide meaningful change in communities and in people’s lives. Whether we are effective or not is life or death for many of those we serve.
1 http://www.philanthropyaction.com/documents/Worst_Way_to_Pick_A_Charity_Dec_1_2009.pdf
Ken Berger is President and CEO of Charity Navigator, Email kberger@CharityNavigator.org
Dr Robert Penna is an independent outcomes consultant and advisor to Charity Navigator. Email rmpc52@aol.com
For more information
http://www.CharityNavigator.org
Comment Tris Lumley
It’s great to see Ken Berger and Robert Penna at Charity Navigator continuing their openness about the journey they’re on, developing a better approach for analysing non-profits. They get right to the heart of it when they say that ‘We must get past the idea that non-profits are too complex or unique to be measured’. That argument is often used to dismiss the potential value of analysing non-profits, as an excuse not even to begin to try. But if you accept that non-profits vary immensely in their effectiveness and outcomes, you have to get past this idea.
Charity Navigator’s approach is shifting from focusing on the financials to covering outcomes, accountability and finances, with outcomes the clear priority. This has to be right − outcomes are hardest to rate, but they’re what all of our work is about.
Even if you have a different take on analysing non-profits, you can engage with what they’re doing because they’re sharing their thinking. We’ll be doing exactly the same with the changes we’re planning to Intelligent Giving’s ratings approach, which will also have a much stronger focus on outcomes through a lens of transparency.
I think there are some real questions for Charity Navigator to tackle. Who will pay for the extra work required to analyse these dimensions of non-profits? Will donors value the increased rigour and depth of analysis? Will it change the way they think about giving? How do we work with non-profits and funders to help them build their own capacity for analysis and self-assessment?
Charity Navigator and NPC are both part of a growing field of organizations trying to answer these questions, work out how to analyse non-profits and provide practical information and advice − for donors, non-profits themselves, governments and society. In fact, the conference Ken was speaking at discussed a proposed new network of social impact analysts, which would help those analysing non-profits to share approaches, agree principles, learn from each other and lobby collectively.
One of the basic principles of this network would be, I think, that we don’t have the answers yet. But we’re working on them. And we’re going to share our workings so that others can contribute, scrutinize and build on them. The more we do this, the more opponents of analysing non-profits will find it hard to say that non-profits are too complex or unique to be measured.
Tris Lumley is head of strategy at New Philanthropy Capital. Email TLumley@philanthropycapital.org
Comments (1)
Awesome and interesting article. Great things you've always shared with us.