After six years, with an annual budget that has grown from $25,000 to $1.5 million and an increase from five grantee partners in Nicaragua to 140 in six countries, the Central American Women’s Fund (Fondo Centroamericano de Mujeres – FCAM) is now embarking on its first major impact evaluation. Like many foundations, we have been using different methods and tools, knocking our heads against the wall trying to find the best way to evaluate both our own work as a fund and the impact of our grantee-partners’ work. What have we come up with?

First, there’s no such thing as a ‘magic bullet’. There are already a good variety of useful tools that we can select and combine according to our needs. Second, the most important things are the kinds of questions that we want to answer with this evaluation. We need to know not only if we did the right things but also, if we did the right things, why and how did change happen or fail to happen? We also need to ask ourselves questions and select tools that are sensitive to the nature of gender inequalities while at the same time allowing us to track backlashes and resistance to change.

According to our evaluation adviser, Virginia Lacayo (who specializes in complexity-based approaches to planning and evaluation), traditional monitoring and evaluation models, theory of change models (regressive mapping), logical frames (cause-effects chains) and cause-effects maps (cause-effect relationships) may work well when we are working on issues with more or less knowable outcomes or assessing products/results, but they are not as effective for measuring complex programmes and social change impact.[1]

Most of our grantee-partners are grassroots groups of marginalized Central American women such as indigenous, rural young women, sweatshop and domestic workers, HIV-positive women and migrants. Along with many other organizations in the region, they are working both on immediate issues and on larger, longer-term ones such as the eradication of violence against women or women’s economic and social empowerment.

Processes involving movements don’t respond to linear and predictable logics; they cannot be controlled or planned in the same way we can plan the building of a school or a vaccination campaign. When the causes and effects are distant in time and space and many different actors are involved, attribution of cause and effect is impossible. We may only be able to find evidence of our contribution to these results.

A systemic vision can encourage us to see different things or to see things in a different way, allowing us to identify key actors and contexts that explain the situation we are evaluating. Instead of beginning with indicators to ‘prove’ our theory of change, we look for trends, patterns and connections that will create new key questions and give us some clues for future actions while our theory of change keeps evolving. In this way, our evaluation process is developing alongside what we want to evaluate, with new means emerging as the process unfolds.

Finally, the more complex the phenomenon, the simpler the design of the evaluation should be. This is particularly true in our case, where our grantee-partners are grassroots groups with little ‘institutional’ experience. We are trying to develop a highly participatory approach where the assessment questions will be defined jointly because questions will arise from the observation itself, on the fly, an approach that can help us break down the dichotomies between donors and grantees and between evaluators and evaluated. Our intention is to facilitate both the observation of unexpected results and the learning process.

We are at the first stage of the process, which is the documentation of FCAM’s philanthropic model through the lens of complexity science theory. A second stage will be the design of an impact evaluation model based on feminist principles and with a system thinking approach.

One of the challenges for FCAM has been that we don’t know of any other organizations using this combined approach for impact evaluation of social justice work. So we’re hoping to develop a new model that could be used not only by FCAM but also by other interested women’s funds and social justice philanthropy institutions.

We may not succeed in developing a whole new model but we’re hoping that at least we’ll get closer to developing a conceptual and practical framework for understanding our impact in the many dimensions we’re working on. We look forward to sharing our successes and our failures with you in the future. Stay tuned!

1 For more details about system thinking and complexity-based approaches, see Virginia Lacayo, ‘What complexity theory teaches us about social change’, in Mazi 10 (February 2007), available online at http://www.communicationforsocialchange.org

Ana Criquillion is founder of FCAM and a current board member. Email anacawf@gmail.com

For more information
http://www.fcmujeres.org