Buzzwords are to aid what sound bites are to the media – an attempt to capture a complex idea in the span of a breath. Indeed, development practitioners regularly arm themselves with terms such as ‘capacity building’, ‘gender empowerment’, ‘bottom of the pyramid’, ‘good governance’ and ‘sustainable development’ to successfully navigate the myriad of meetings, donor reports and presentations of the aid world.
A re-emerging buzzword appears to be ‘Value for Money’ (VFM) in aid programs. And it has a nice ring to it. Taxpayers demand value for money, donors need to ensure value for money and beneficiaries deserve value for money.
So what is VFM with respect to aid programs? The New Zealand aid program defines it as “achieving the best possible development outcomes over the life of an activity relative to the total cost of managing and resourcing that activity and ensuring that resources are used effectively, economically, and without waste”. DFID defines it as “maximising the impact of each pound spent to improve poor people’s lives”. The 2011 Independent Aid Review of AusAID suggested that VFM be a ‘fundamental operational principle’ of the aid program. However, AusAID does not appear to have publicly released a VFM definition or framework for its aid programs (although the 2012 AusAID Multilateral Assessment represents a form of VFM and AusAID does consider VFM in its procurement activities).
To measure VFM, DFID focuses on the three ‘Es’ of economy, efficiency and effectiveness and cost-effectiveness analysis. Economy involves ensuring inputs (such as human resources and capital) are of sufficient quality at appropriate cost while efficiency is ‘how well’ inputs are converted into outputs. Effectiveness is ‘how well’ outputs achieve a desired outcome while cost effectiveness measures inputs relative to impacts.
The terms woven through DFID’s VFM framework are similar to those in a typical evaluation log frame; ‘inputs’, ‘outputs’, ‘outcomes’ and ‘impacts’. However, while an evaluation may focus on measuring outputs, outcomes and impacts based on initial objectives (such as SMART objectives), VFM focuses on demonstrating that outputs, outcomes and impacts are maximised for minimal inputs (without compromising quality). Hence while evaluation largely involves analysis of outputs, outcomes and impacts relative to objectives, VFM largely involves an analysis of outputs, outcomes and impacts relative to inputs, and outcomes relative to outputs.
This is great in theory but in practice, similar issues plaguing evaluation measurements remain. Given its increasing interest, earlier this month the Development Policy Centre hosted a forum on VFM. Two speakers were invited, Cathy Shutt, from the Institute of Development Studies and University of Sussex, and Russell McKay from GRM’s Effectiveness Development Group (a podcast and slides are available).
One thing that was clear from the presentations is that the approaches to measure VFM are still unclear. Cathy mentioned that NGOs should be able to measure ‘value’ based on what’s important to their programs. Given the heterogeneous nature of aid programs, the issues involved in attribution and contextual differences, this appears reasonable. This echo’s DFID approach, which involves providing examples of best practice indicators but also acknowledging that there is no one size fits all approach, especially when it comes to measuring the value of programs with benefits that are not easily quantifiable (such as gender empowerment programs).
However, this approach also means that ‘value’ becomes subjective which in turn makes it tough for small NGOs or even evaluation specialists to disentangle how to measure VFM to the satisfaction of donors.
I understand AusAID is working on their approach to VFM. So given the inherent difficulties and subjectivity in VFM measures, what could AusAID do to make it easier for implementing partners? The following are my suggestions:
- Issuing a draft set of VFM guidelines for feedback prior to enforcing it as policy.
- Create appropriate guidelines based on the experiences of organisations such as DFID and implementing partners that have been living and breathing VFM over the last few years.
- Distributing ‘best practice’ examples of VFM reports across different sectoral areas.
- Undertake training sessions on how to implementing VFM and/or recommend whether specialised ‘VFM’ consultants are required for VFM design.
- Roll out VFM gradually to address teething problems that are sure to arise.
An alternative approach is to simply provide some guiding principles for partners to use when considering VFM. However I fear this may result in partners ‘muddling through’ the process and may create greater uncertainty and confusion.
Regardless of AusAID’s approach to VFM, it should be integrated with AusAID’s monitoring and evaluation frameworks (even if these have to be adapted). This will hopefully reduce the bureaucratic burden on program partners; the last thing we want is another ‘aid industry’ that simply spawns a new generation of VFM consultants.
Ultimately, VFM will increase in prominence, especially if the Coalition is elected in September. While difficult (think for example AusAID’s unenviable task of transferring VFM reports into quantitative formats for program comparison purposes), the VFM agenda is responsible and prudent given the need to be politically accountable for taxpayer dollars.
Dinuk Jayasuriya is a Research Fellow at ANU. He was previously a Monitoring and Evaluation Officer with the World Bank Group and has provided evaluation advice to ADB, AusAID and International Red Cross Societies.