‘If the call of the last decade of the 20th century was accountability, the mantra of the new millennium is impact,’ said Barry Gaberman, former Senior Vice President of the Ford Foundation, in his closing address at the EFC meeting in May.

Impact is the new vogue, and in response an ‘industry’ is springing up. We have systems of evaluation – Six Sigma, Results Based Accountability, etc – new centres to help organizations become more effective and a growth in consultancy to fill the new market niche. Why can’t we do it ourselves? Might we not learn more from the process and have greater ownership of the outcomes? These were questions that faced a group of ten foundations that met in St Petersburg in July as part of the Ford Foundation’s International Initiative to Strengthen Philanthropy (IISP).

Despite its apparent diversity (participating foundations came from India, Bangladesh, Israel, Russia, Mexico, Brazil, Ghana and Kenya), the group had a number of things in common – unsympathetic political and economic climates, and a relatively short philanthropic tradition in their home countries. Only one of the participating foundations was older than 20 years.

By the time of the St Petersburg meeting, the relationships between group members were already strong. This is one of the features of the IISP process that offers a unique peer learning opportunity, bringing together this diverse group once or twice a year to learn from what distinguishes them as well as from what unites them. Participants were thus well placed to act as critical friends, helping one another to develop their impact evaluation strategies.

In order to find the starting point for impact measurement, we are basically trying to answer the following questions: What are we seeking to accomplish? Are we doing the right thing? How good are we at it? How can impact be measured when it’s so difficult to attribute responsibility for any change?

The St Petersburg group began by using the outcome mapping methodology of the International Development Research Centre. Each foundation was asked to identify ‘partners’ – other actors contributing to the complex processes of social change – and to describe the changes in their behaviour that would be expected and desirable if the goals the foundations had set themselves were to be achieved. Foundations then presented their own self-analysis to their peers, who shared feedback on whether the ideas were clear and convincing, and whether the key partners and outcome challenges chosen seemed appropriate in light of the foundation’s vision and mission.

Next came consideration of what constitutes a learning organization. How does a foundation perceive itself, its adaptability and its capability for continuing effective action? Here, Kolb’s learning cycle was used. The cycle has four stages: reflecting, planning, acting and observing, and participants were asked to identify what systems and procedures would need to be in place at each of the four stages to ensure learning is continuous. Once a common set of criteria was agreed, each foundation ranked itself against them. Again, foundations presented the results to their ‘critical friends’, who raised pertinent questions about their self-analysis.

The findings of these exercises were then used to revise the case studies that each foundation had submitted in advance of the meeting, and these were presented to the whole group on the final day. It was perhaps the high point of the meeting and produced some practical points for immediate action, as well as a list of themes for longer reflection (such as devising key performance indicators and using other methods of self-appraisal). For one foundation it meant a major shift in thinking – the participants tore up their previous organizational strategy and opted for a new approach to embrace impact measurement.

So what did we learn? Each foundation naturally drew its own distinct lessons, but generally the following points emerged:

• It is vital to be clear about the precise difference you want to make in your society and how your activities help to create it.
• Key performance indicators are crucial here, but also the hardest thing to develop.
• Time, space and resources need to be dedicated to the task of performance management.
• Finally, the foundation needs to ‘own’ the process of impact assessment and not leave it to an outside ‘expert’ – no matter how tempting this appears.

Inga Pagava is Director for Development & Consulting at Charities Aid Foundation Russia, one of the foundations participating in IISP. Email ipagava@cafrussia.ru