Shareholders hold the private sector to account – success, or perceived future success, drives share prices higher and management benefits. The reverse is also true; lower profits pressure managers into action. Bottom line, the private sector is driven by a profit incentive.
What about aid? The incentive imbalances have been widely acknowledged (see for example, “An overview of aid effectiveness determinants and strategies” by Stephen Howes) and are pretty intuitive – taxpayers in donor countries have collectively strong political input into the aid process but weak interest in the outcomes while poor beneficiaries have little political influence but stronger interest.
One way to counter these imbalances is through independent evaluations. Moreover, if development outcomes are to the aid constituency what ‘profits’ are to private firms, then one of the roles of evaluations is to act like ‘audits’ for the development world.
How many organisations then make their independent evaluations public? I was planning on presenting a table full of facts and figures but there’s no need; very few of the 33 AusAID accredited NGOs have links to evaluations of their programs on their websites. One refreshing exception is CARE which has a whole website dedicated to their evaluations.
I worked as an independent evaluation consultant for various Red Cross Societies so I know organisations undertake independent evaluations that are not reported publically. If these reports remain confidential and are used to improve future programs then that’s a positive. However too often, evaluations are a corporate requirement – a necessary evil – rather than an important part of the learning process. Inevitably many reports remain unused or are lost in the electronic ether. This is difficult to change from the outside.
What is not as difficult to change is to increase transparency. To its credit AusAID has taken a step in the right direction by creating a Transparency Charter and requiring independent evaluations of projects, even though many of these evaluations are yet to see the public glare (see an earlier blog).
Ultimately we support NGOs directly through donations or indirectly through our taxes. Just as shareholders of private companies have a right to view an independent audit of a company’s accounts to ensure they are ‘true and fair’, the public should have the right to view an independent evaluation, or multiple independent evaluations, on the effectiveness of a development organisation’s programs.
We deserve rigorous analysis on the development “bang for our buck” – and we deserve for that analysis to be made public. Is it hard to do? NGO accreditation from AusAID brings with it important benefits, including tax-deductibility and eligibility for government (i.e. taxpayer) funding. These numbers aren’t small; the Independent Aid Review puts them at just over a quarter of a billion dollars for the 2010 financial year and they are only going to increase over time. Clearly AusAID holds at least some of the NGO purse strings – and they can make publication of independent evaluations a requirement of accreditation (and therefore funding). This may not guarantee better aid, but it certainly guarantees better transparency.
Dinuk Jayasuriya is a Post-Doctoral Fellow at the Development Policy Centre. He was most recently a Monitoring and Evaluation Officer with the IFC at the World Bank Group.
Thank you Chris and Garth for your comments.
I agree with your comments that ideally, the poor in our communities should have a say relating to the effectiveness of our aid program. Such actions would reduce the incentive imbalances between the aid agencies and the poor. My suggestion, that we make public disclosure of evaluations a pre-requisite for NGO accreditation (and therefore funding), is one of the first steps that can be made to improve transparency. This is unlikely to be too difficult for AusAID to implement and administer. A next step could be ensuring that publically available NGO evaluations are up to a particular standard and that those NGOs receiving core funding from AusAID are evaluated by someone independent of both AusAID and the NGOs. These actions would allow any interested Australian to view whether their taxes are being spent appropriately – as arguably the Australian taxpayer is the most important (albeit on average the least interested) stakeholder. But these are not the only steps – other steps to improve transparency are as you suggest by “implementing, sharing and publicising innovations” and making activities accountable to the poor.
The issue of publishing evaluation is who goes first. Dinuk seems to suggest the NGO should start and publish all of their evaluations (the definition of ‘all’ becomes interesting here), but many do publish program overviews ( more than only CARE) which are meta studies of all their evaluations and key lessons learnt. Partner program evaluations are also published on the web, for example the We Can End Violence against Women program in South Asia (supported by multiple donors)
AusAID does not publish all of it evaluations (it is fairly selective), nor does the World Bank. In line with Chris and Garth comments most useful evaluation are those to the partner community, country, or NGO, depending on who the partner is.
I have done a Third Party Assessment for the partner government of a large decade long World Bank program. I suspect neither the partner government of the World Bank would like it published, but it was useful to the partner government as it gave a clear steer on how future loans may be directed in that sector, and how it might direct is own investment.
Likewise evaluations I undertake for NGOs are largely directed to the partner as they are the ones that it is most useful to guide them in the negotiations with donor NGOs.
I am not sure AusAID encourages recipient government to review the AusAID programs but that would be a much greater step forward in transparency than mere publication. Most published evaluations focus too much on outputs than outcomes, and the authors are very mindful of who is paying the bills.
Perhaps greater attention could be to academic studies of development effectiveness at local level which look at these issues particularly from the recipient point of view, as these programs invariably have multiple donors. For example I have looked as effectiveness in a sector across 15 small and medium NGO in India. Published studies like these may yield more that published evaluations with their focus on short term results.
I think Chris has the right emphasis here. Evaluations are important (especially if we can get people to learn from them and apply the lessons – not often achieved), but shortening and strengthening the lines of accountablility to poor communities is more important. Dinuk cites profit as a critical motivator in the commercial area. Profit is not generally generated by the assessments of expert evaluators of the commercial transaction, but by the powerful choices of people on the ground.
While there have been many improvements in AusAID processes recently, making activities accountable to the poor, seems to me, to need a lot more focus and work by AusAID management.
I certainly I agree with Dinuk that greater transparency amongst International NGOs is important. As the ‘Promoting Voice and Choice’ report (http://bit.ly/x3rUvT) – which I authored for ACFID in 2010 – noted there have been some improvements in this area but they were not sufficient. Much of the innovation in this area has focused as much on NGOs becoming more accountable and transparent to the people and communities they work with, as on transparency to the Australian public. However these experiences are inadequately shared and often done in a piecemeal fashion.
A truly strategic approach to transparency and accountability would see more collective efforts to not only publish reviews and evaluations – which is starting to happen and is important – but also have a focus on implementing, sharing and publicising innovations which allow people living in poverty to be holding the powerful – including International NGOs – to account