Comments

For those interested, <a href="https://dfat.gov.au/aid/how-we-measure-performance/ode/Documents/impact-evaluation-discussion-paper.pdf" rel="nofollow">here is a discussion paper</a> on impact evaluations from AusAID / ODE. The paper is from 2012 and shows that greater use of impact evaluations was on the radar. Perhaps the embers of the Impact Evaluation Working Group are still warm?

Thanks, Ash, for this great policy brief and blog series. As far as I can tell, DFAT (and other aid agencies) have almost nothing to lose and everything to gain from communicating more, and more openly, about what Australian aid does and the difference that it makes in people's lives. I think your recommendation on increasing engagement mainstream media is a particularly important one; I imagine many of the people who go to the aid pages of the DFAT website or follow it on Twitter are already well-informed about Australian aid -- the challenge is reaching the broader public.

Here in Oxfam's water governance programme in the Mekong region we are putting the Augusto Boal 'Theatre of the Oppressed' into action to achieve development goals. We are working on some of the most conflict ridden and politically sensitive issues of natural resource grabbing and have found theatre is a valuable way for communities to examine and identify their own solutions. Skills are required to safely facilitate these processes and we are training 8 young people from the region in these methodologies. They will go on to work with hundreds of community members. government officials and private developers. You can learn about them <a href="https://drive.google.com/folderview?id=0BzneKwk1NKwmSFJCVDVIcGtaZE0&usp=sharing" rel="nofollow">here</a>. If you would like more information please contact me.

Thanks Patrick, I agree good panel data would be better than the cross section we currently have. Still the cross section is, I think, better than individual data points, and makes for a good complement to qualitative work. I agree with you that the nature of the issues of the day could well be important (we say as much in the blog and the paper). Avoiding the cases of any individual NGOs, in a broad sense I completely agree understanding issues such as values and how they might be associated with the variables we've studied makes a lot of sense and is the stuff of interesting future work (I think we discuss this possibility in the paper). Terence

Terence I think it may have something to do with values c.f Save the Children of recent times. NGOs have alway spoken out on values issues and the evidence seems to support the funding effect (in aggregate) seems to me at least, to be relatively weak. In the 1980s when they were more outspoken they even got more funds. Perverse incentives maybe

I just watched 'they only drink it in the Congo' in London, written by an aid industry outsider, Adam Brace. But it is excellent - touches on the white saviour complex, the realities of conflict and violence in diaspora communities and at home in the DRC, and the terrible disappointment of trying and failing to make things a bit better. Shame I can't see your work - sounds great!

Hi Patrick, So, as best I understand your argument, you are claiming that: because the NGOs in question are less heavily dependent on DFAT for the salaries of their Australian staff, and are only actually heavily dependent on DFAT funding for their work in the field (more heavily than my initial numbers would suggest), there is no reason to anticipate them being wary of doing stuff that might displease the government? I'm sure there is a good reason for why the pattern we identified does not conform with theoretical expectations. But I'm equally sure the explanation you've just provided is not it. Enjoy the rest of your week.

How fabulous Tom - great contribution to the narratives about development dilemmas. Theatre and other art forms really under-utlised in development interventions and M&E. Ironic really given the power and place of art and culture in many of the countries we all work in.

Terence even thought the number is large (>50%) it may not impact running costs as these DFAT funds are required to be quarantined (contract wise) so the impact may not be as much as if it is general revenue. Thus the problem of drawing too much from these aggregate type analyses..

Thanks Patrick, I know of the book, but wasn't clear if it was the work you were referring to or not. I look forwards to reading chapter 8. With respect to how much money NGOs get from DFAT. From 2014 data (the data used in our study): Of the top 50 ACFID members (recall our study looked at the largest NGOs), looking at DFAT funding/total revenue for international development work, the mean NGO received 24% of its funding from DFAT. The median 26%. More than 20% of NGOs receive over 50%. Such significant shares of revenue aren't pocket change. I would have thought that losing such a large share of revenue (or money spent in the field) would cause pause for thought. Intriguingly, within the constraints of our study, it does not appear to have.

Sorry Terence I thought you were aware of it. The reference is Kilby, P. 2015. NGOs and Political Change; a History of the Australian Council for International Development. Canberra. ANU Press. The key chapter on this is chapter 8 and p.157 has the case I refer to. The dependency argument has been peddled for over twenty years now with little evidence of it having much effect at all. The fall in numbers of accredited agencies points to Government funding being a nice bonus for many (mostly small) agencies but not enough for them to change practice to meet accreditation criteria. Most DFAT contracts are time bound and so agencies do lay off staff etc when they come to an end. Nowadays most agencies are careful not to risk a sharp drop in income hence they keep the level of government funding at lower level than it was in the 1980s, when they were more trusting and got their fingers burnt.

Hi Patrick, Thanks for the comment. As I said to Garth, I think combining qualitative with quantitative work is ultimately the best way to undertake this sort of study, so I am interested to hear of the book (which book? the history of ACFID?). I think anyone seeking to take this work further would benefit from case histories. No disagreement there. What we've done with our work is something not done to date: contribute to the quantitative component of potential work in this area. Learning about specific cases tells you one very useful thing; learning about overall patterns reveals another. Also, you wrote: "One is scale and that is the larger NGOs can afford to devote resources to awareness raising and they coincedently get a more of the government pie for obvious reasons." Indeed, if you read our original blog post you will see: "In each regression we included two control variables (whether an NGO was religious or not, and the NGO’s size in terms of revenue), alongside the main independent variable of interest..." We controlled for NGO size in our regressions. Then, you wrote, "A statistical association does not necessarily suggest causation." We're quite aware of that. Reverse causality is not likely to be an issue, but omitted variable bias may be. Which is why we wrote in our blog post that, "Ideally, we would have been able to add more control variables into the equation..." Having said that it is hard to think of obvious confounding factors that aren't in the existing regression model. Also, you wrote, "The other issue is the focus on internet campaigns, may not be the full picture. The church agencies in particular use church based campaigns for example, and these can be on quite a big scale." We, control for religious or not in our regression, which ought to address this specific trait. However, as we mention in our blog post, "The relationship might plausibly have been different had we studied other media. Because of these caveats..." So once again: we are aware of the boundaries of our investigation. We wrote about them in the blog post. Nevertheless, we think the media terrain we did cover is a very important one for NGOs. And you wrote, "The use of the term ‘dependency’ here is loose. Either one is dependent or one is not (like pregnancy). One cannot be a little bit dependent." I disagree entirely. As an NGO changes from getting 5 to 25 to 45 to 65 to 85 per cent of its funding from the government, the costs of it losing government funding for the work it cares become incrementally more severe. There may be a point where the costs of withdrawal of government funding become particularly severe (i.e. end of NGO), but long before that there will be sufficiently major costs -- programmes closed and staff laid off -- to change the nature of the relationship in theory. Also you wrote, "I refer in Ch 8 to when the governemnt suggested gag clauses about 10 years ago and the big agencies simply said in response they would withdraw from the funding scheme, thus ending that discussion." This is interesting to know about. Thank you.