Comments

Satish and Tess, Thanks for your comments on my blog. On Tess's point, ODE does a good job in terms of being transparent about which questions it is asking. For some of its evaluations (I noticed rural development and civil society) you can see their terms of reference up on the ODE website. On Satish's point, its not the case that ODE was modelled on the Bank's Evaluation Department (now called the Independent Evaluation Group or IEG). IEG reports to the Bank's Board, not to its President, and so is able to maintain its independence. There is no such protection for ODE, since it reports to the head of AusAID. I don't see the Independent Evaluation Committee as another layer of bureaucracy, but as a mechanism for ensuring independence and timeliness. Which take's me back to Tess's question. The IEG produces a lot more output then ODE, but then it's a much bigger outfit. In fact, it does a brief review of every completed Bank project, not to mention a number of country evaluations, thematic reports, and an annual aid effectiveness review. A more relevant comparison is on timeliness. I don't have the details, but, as far as I can tell, IEG manages to avoid long delays around the release of its reports. It certainly manages to get its annual review out every year, something ODE has struggled with.

I can't help feeling that evaluating the delivery of aid is a bit like the old story of tourists asking for directions in Ireland to be told "Well I wouldn't be starting from here if I were you". It is often hard to be sure that the fundamental questions have been framed properly before the "busy work" of evaluating has begun. It would be good to know how ODE compares with similar outfits such as the World Bank's Evaluation Department referenced by Satish in terms of output, timeliness, etc.

Thanks to Stephen for seeking explanations for the silence of ODE. I mused at the creation of ODE, as I do now, on whether this function should be housed within AusAID rather than in the Australian National Audit Office (ANAO). The merits of placing ODE within AusAID were to allow access to in-house data. ODE’s independence was to be protected through and organisational structure modelled on the World Bank's Evaluation Department. With Stephen's thoughts now, is it time to fold ODE into ANAO? ANAO, after all, is independent, reports directly to parliament, and is mandated to provide: "independent assessment of selected areas of public administration, and assurance about public sector financial reporting, administration, and accountability". This may be a superior strategy to the creation of yet another layer of bureaucracy, in the form of an Independent Evaluations Committee, to oversee the functions of ODE.

Development work is often focused on unanswered questions, something very well known by the religious. To be an effective aid worker, is in and of itself a leap of faith. See: http://www.how-matters.org/2011/09/28/a-matter-of-faith/

Thanks for this piece which I found very interesting and hopefully there will be more to come by way of research in this area. I wonder whether there are any significant differences between the established players (e.g. Catholic and Anglican derived entities) and those who are newer on the scene (e.g. LDS & SDA based groupings). My examples are based on the Pacific experience but I imagine that elsewhere in the world there are similar divides between FBOs and FBDOs that have been around for a long time and others that are new on the block.

Thanks for that Sharon. It was only 'officially' opened this month, I will change that in the text.

I think the USAID office in PNG has been open for 6 months.

Thank you for your comment Tess. I very much agree with your sentiment that it is important for key players to be honest with themselves and stakeholders — I think aid and development sector jargon is another barrier to broader understanding in the community of what humanitarian assistance and aid really does and what the process involves. Putting a simple message forward to the public to get support, but then using jargon/technical language in reports and other documents and basically making information impenetrable to laymen, is a problem. I think the NGO sector does a much better job at publishing clear, readable reports (or at least more accessible executive summaries) that cater to a wider range of readers than government agencies or multilaterals. I feel there needs to be more communication materials in this middle ground... not overly simplified but not overly technical.

I think your penultimate point is particularly valid - if agencies make this information available and include it in their communications then there is hopefully less risk of it being held up as some sort of smoking gun by someone else. And of course as media develops agencies are becoming more adept at tailoring messages according to the intended audience and the means by which it is to be transmitted. But in order to build transparency and, more importantly, sustainability, it is really important that the key players are honest with themselves and their stakeholders about the reality of what they are doing and the environment in which they are doing it.

Dear John, Thank you for sharing a provocative post on funding to multilateral development banks. I think if you re-read my blog again you will see that it is very clearly about core contributions to the IDA16 replenishment round. I don't dispute that Australia gives earmarked funding to the multilateral banks, and this reported on the AusAID website. The issue is that picking projects and programs, and then earmarking funding for these could be thought of as bilateral aid--with the multilateral banks being used as an agent. There has been a proliferation of this kind of funding in recent years, not just from Australia but also other donors. And I'd argue that this undermines, rather than strengthens, multilateral banks. I look forward to discussing this and other concerns with the balance of your report during the panel discussion tomorrow. Best regards, Matt

I find it interesting that Korea and Japan did so well in the ActionAID analysis but so poorly on the CGD's Quality of ODA assessment. Any thoughts on why that may be?

The whole report is now available online at the link contained here. I agree that it is really important that assessment (which may include criticisms) of aid effectiveness and efficiency has to be done with a great deal of rigour and appropriate criteria need to be applied. It is essential that recipient countries' leaders are well informed in order to enter into meaningful dialogue with (potential) development partners and in terms of the domestic situation for donor countries, aid should not be at risk of becoming a political football on the back of poorly constructed critiques.