Evidence-based philanthropy is more talked about than done. Philanthropy support organisations can help to remedy this.

Though evidence-based giving and data-driven philanthropy have become buzzwords in the sector, they have yet to be widely practised by private philanthropists and foundations. Data¹ collected is often unhelpful, nor does it significantly influence a funder’s strategy. Many of the reasons for this could be addressed by philanthropy infrastructure organisations. This article looks at why smaller grantmaking organisations have yet to fully embrace data-driven philanthropy, what the potential benefits of data are and how it should be used, and how philanthropy infrastructure organisations could help.

The challenges of data collection and use

Photo: Janneke Staaks (CC BY-NC 2.0)

Ultimately, the impact (defined here as lasting and sustainable positive change that you are seeking to achieve) philanthropists can have is limited by: the area they focus on; the interventions and organisations they fund; and the outcomes (both positive and negative) of that organisations work. Data can be used at all three stages. Unfortunately, there are a number of challenges that prevent this evidence-based approach from really taking off among smaller grantmaking institutions.

Capacity

The first is capacity. Many smaller foundations do not have the capacity to do extensive analysis of impact reports on different interventions, nor do they have the in-house expertise or funding to support charities in collecting monitoring and evaluation (M&E) data. As a 2014 UN report on the use of data in sustainable development notes: ‘too many people, organisations, and governments are excluded from this new world of data because of a lack of knowledge, capacity, or resources.’

Motivation

The second is motivation. Many philanthropists and smaller private foundations are more inclined to fund causes or organisations that they feel a personal connection to, not because evidence suggests they are effective. It is not the case that donors do not care about impact, or ensuring that their money is making a difference, rather that a different set of heuristics are employed to try and ensure this is the case. Anecdotal evidence suggests that donors often rely on their proximity to, or knowledge of, the area where the programme is being implemented, or the size of the charity requesting funds to determine whether an organisation is effective or not. Despite feeling comfortable and intuitive, these measures rarely give any indication of a charity’s effectiveness.

Complexity

Photo: P. De La Cruz (CC BY 2.0)

The third is focus area. A common refrain is that problems that are complex, address the future, or involve multiple stakeholders are nearly impossible to collect accurate monitoring and evaluation data or prepare comprehensive impact assessments about. Donors who take this view often further argue that the growing importance of data and evidence is having the counterproductive effect of discouraging philanthropists from taking risks. In reality, neither proposition is true. The Open Philanthropy Project, for example, applies a ‘hits-based’ approach to giving, funding projects that could be 90 per cent likely to fail provided that they also have a very high expected value; the Hewlett Foundation, which places an absolute premium on outcomes despite the variety of cause areas they support, including hard-to-measure areas such as policy; and the Centre for the Study of Existential Risk at Cambridge University where a team of researchers and academics is actively working to build the knowledge base around biological, nuclear and environmental threats and the best way to address these challenges.

Availability of high-quality data

The fourth challenge is availability of data. Good quality data on successful interventions is still not widely published. As a 2014 report by the Conference Board argued, foundations rarely collect information about their constituents, or ultimate beneficiaries including their unmet needs, mechanisms for meeting those needs, or the work of similar organisations. Even today, four years later, the most common type of information shared by foundations is grantmaking data – how much money was given to which organisation, for what programme, and for how long, which teaches us nothing about the impact that these grants have had. It is also not legally mandatory for grantmaking organisations to share this information so the extent to which foundations, particularly smaller foundations, participate is limited.

Ease of collection

The fifth is ease. Collecting data on social issues is difficult. When the outcomes that charities are trying to affect are so varied, room for standardisation on metrics and on what data to collect is minimal. Further, to date, there has been more of a focus on outputs, which are easier to measure, though less explicit about change achieved than outcomes. Unfortunately, outputs do not really give a picture of the lasting social change philanthropists hope to effect. The lack of capacity, noted above, compounds these problems.

In many cases sound M&E data or impact reports do not exist, particularly for more neglected, or smaller-scale social problems. Philanthropy infrastructure organisations could help to fill this ‘evidence gap’ by supporting and funding randomised controlled trials or, if that method is not appropriate, other methods for assessing outcomes of an intervention such as linear regressions and statistical matching.

The general effects of these challenges are, first, charitable funds are not always directed to the most neglected problems and geographies because grantmakers do not really know what these are. Second, grantmakers rarely know what is working and what is not. Third, even if sound data has been collected, or reports on the impact and outcomes of a programme have been published, too often there are no mechanisms in place for sharing this information with grantmakers. As a result, the sector is missing out on a lot of knowledge and there is substantial waste in time, money and resources through the duplication of effort.

How philanthropy infrastructure organisations can help

Philanthropy infrastructure organisations such as WINGS (led by Benjamin Bellegy, a guest editor of a forthcoming Alliance special feature on philanthropy infrastructure) are ideally placed to address this issue.

They could support both the writing and publishing of impact reports that synthesise evidence about the value of particular interventions, and could also support grantees to develop robust M&E systems. Having this type of information more widely available could help to inform grantmakers and grantees of what has been tested and seems to work in specific contexts. At Founders Pledge, we are supporting the first item by building an online research library that will make available all of our analysis and reports into charities and cause areas to date. Over time, we will also be publishing a number of methodology documents which we use ourselves to parse and interpret data, which we hope will also be of use of other grantmaking organisations.

The June edition of Alliance magazine will focus on ‘Philanthropy’s Developers’. Click on the cover to listen to a preview.

Identifying a cause area to focus on is arguably the most important step that donors will take in informing their giving, at least as far as their ability to be as cost-effective as possible is concerned. Nevertheless, remarkably little aggregated data on this is available at a global level. Philanthropy infrastructure organisations would be ideally placed to produce and publicise such aggregated global data. This would give a clearer picture of where the majority of philanthropic funding is going, and which areas remain relatively underfunded relative to the size of the problem. The OECD Center on Philanthropy is producing research on the contribution private philanthropy is making toward development.

In many cases sound M&E data or impact reports do not exist, particularly for more neglected, or smaller-scale social problems. Philanthropy infrastructure organisations could help to fill this ‘evidence gap’ by supporting and funding randomised controlled trials or, if that method is not appropriate, other methods for assessing outcomes of an intervention such as linear regressions and statistical matching.

An example of an organisation already working on this is J-Pal. Based at MIT and focused on supporting governments in low-income countries establish evidence-based policy, J-Pal is not a philanthropy infrastructure organisation in the traditional sense. Nevertheless, to date, J-Pal have 902 ongoing and completed randomised controlled trials across 79 countries and are also actively building the infrastructure for philanthropic RCTs. Results of all completed trials are available on their website.

For donors who may wish to work directly with the grantees to collect and assess data, or develop their in-house capacity to collect and assess their own grantee data, philanthropy infrastructure organisations could also have a role to play. As mentioned, collecting and processing data on social issues is difficult. Philanthropy infrastructure organisations could help donors to navigate this by training them on how to collect M&E data from programmes, processing it and using it to inform giving strategies. The US-based Feedback Labs, a membership organisation with the aim of ensuring charities and grantmakers are responsible to their ultimate beneficiaries, enumerates five steps necessary to ensuring that data is accurately collected and incorporated into grantmaking process: design, collect, analyse, dialogue and course-correct.

The sector’s responsibility

Ultimately, the key point to remember in any discussion about why data is important is that donors and their grantees should not just collect data for the sake of it. Instead, it should be an important part of the sector’s responsibility to ensure it is working in the most proven way possible. As this article has illustrated, there are a number of organisations working toward these goals, however, what is needed at this stage is a larger, sector-wide effort to bring this information together and make it easily searchable and useful for donors of all kinds. Philanthropy infrastructure organisations are in the ideal place to do this.

Aurelia Kassatly leads deployment and consulting at Founders Pledge, which supports giving in the tech sector.

Coming up: The June 2018 issue of Alliance magazine, Philanthropy’s Developers, will focus on the development of philanthropy infrastructure worldwide.

¹ Two types of ‘data’ are referred to in this article: ‘monitoring and evaluation data’ which is collected by grantees to assess the performance of an intervention against the outcomes that it is trying to affect. The second are external studies of an intervention which attempt to determine its effectiveness, and sometimes take the form of ‘impact reports’.