Development twitter erupted in chirping this week. The cause was a Guardian op-ed from a group of development notables decrying foreign aid. Prominent aid commentator Duncan Green tweeted that the piece was a “must read”. Green’s tweet alone was re-tweeted nearly 100 times.
Aid, especially government aid, needs criticism. I’ve spent the last eight years highlighting problems with New Zealand aid. But criticism is more useful if it is cogent and accurate. This op-ed was anything but.
To be fair to the authors, the first error presumably isn’t their fault. The Guardian describes them as, “Fifteen leading economists, including three Nobel winners”. At least one of the fifteen “economists” – Jason Hickel – is actually an anthropologist. Oh well.
The authors add their own errors quick enough. Their first paragraph:
“Global poverty remains intractable: more than 4 billion people live on less than the equivalent of $5 (£3.80) a day, and the number of people going hungry has been rising. Important gains have been made in some areas, but many of the objectives set by the millennium development goals [sic] – to be reached by 2015 – remain unfulfilled. And this despite hundreds of billions of dollars of aid.”
Global poverty is not intractable. The share of the globe’s population living in extreme poverty is falling. This is true if you use the standard global poverty lines, or the authors’ preferred $5 a day line. (For a good discussion see here. You can play round with the numbers yourself here). As for hunger, the absolute number and percentage of undernourished people rose globally between 2015 and 2016 (the most recent available data; see page 5 here), but this is in the wake of steep declines over the previous decade.
And while not all of the Millennium Development Goals were met, in most areas trends were in the right direction.
Meanwhile, “hundreds of billions of dollars of aid”, sounds like a lot of money, but it’s not, when put in perspective. In 2017, government aid from OECD DAC donors summed to 0.31% of donors’ combined Gross National Income (GNI) (data here.) A trivial effort.
And in 2016 (the most recent year with data), government aid was 0.42% of the combined GNI of aid recipient countries (data here.) In the scheme of things, “hundreds of billions” is a trickle.
Next the op-ed complains about the evaluation of aid projects to learn what’s working, taking a particular focus on Randomised Control Trials (RCTs). It points out that RCTs are imperfect and expensive. And states that RCTs are: “fraught with ethical challenges – especially when it comes to health-related interventions. (Who gets the treatment and who doesn’t?)”. (By ‘treatment’ they mean aid-funded intervention.)
RCTs are expensive and they certainly aren’t the right tool for all evaluations. But RCTs aren’t fraught with ethical challenges. “Who gets the treatment and who doesn’t?” is determined by chance (the giveaway is in the word ‘randomised’). Very few aid projects have truly universal coverage. Is random allocation less ethical than any other means of choosing where to work? And is it less ethical than failing to learn? Less ethical than wasting time and money on something that doesn’t help?
Finally, over the next paragraphs the op-ed makes what seems to be its main point. Focusing on aid and aid effectiveness will distract us from the real issues: learning about better policies in developing countries, and more action from wealthier countries on issues such as tax evasion and climate change.
Better policies in developing countries would be great. Past research in this area hasn’t met with much success, so there’s certainly scope for learning more. But policy change in developing countries is rarely a simple intellectual exercise. Policies – everywhere on earth – emerge from political economies, and improvements usually stem from less than ideal compromises that are shaped by context rather than research. I’m in favour of trying, but this may not be an area where aid donors can do much.
On the other hand, wealthier countries can and must take more action on issues such as climate change. Yet is there actually any evidence that aid and RCTs are preventing this action? Some aid NGOs such as Oxfam are major advocates on climate change. And I’ve yet to hear a politician say, “we’re slashing carbon taxes so we can focus on Randomised Control Trials”. The real impediments to action on climate change are political ideology and the fossil fuel industry. Not aid.
Aid will never solve all the world’s development problems. Aid can be improved. Aid can and should be criticised. But when criticism is incoherent and incorrect, it is worse than no criticism at all.
Wood criticises the claim that global poverty and hunger have been increasing. This claim seems to be based on research by Jason Hickel, who is one of the signatories on the letter in question. Perhaps Wood would find it useful to read the underlying research before he weighs in? Here is an example: https://www.academia.edu/21593862/The_True_Extent_of_Global_Poverty_and_Hunger_Questioning_the_Good_News_Narrative_of_the_Millennium_Development_Goals
Hi Chris,
Thanks for your comment. I think you’re likely correct and that the claims about poverty and hunger come from Hickel. I’m familiar with his research and more general claims.
The critiques he makes of the MDGs (and the broader argument that some seem to have been retrofitted for success) sound plausible. But then again so do William Easterly’s claims that the MDGs were constructed in a such a way as to make improvements in many African countries seem like failure: https://www.brookings.edu/research/how-the-millennium-development-goals-are-unfair-to-africa/
Global goals are imperfect at best.
More importantly, in most of the areas of human development captured by the goals, progress was made in most parts of the world. This is true regardless of flaws in poverty lines chosen or yardsticks of progress, or whatever. In the case of poverty and hunger, data indicate real improvements. You can check this out yourself from the links in my blog post.
One final point: Hickel often makes his arguments in terms of absolute numbers. (i.e. the absolute number of poor in the world). But given the world’s population is growing, speaking in terms of percentages (% in poverty) makes a lot more sense if you want to track development progress.
Terence
I wish I could agree with Garth’s comments that it is ‘amazing what has been achieved especially in areas that are well understood such as … education’ and also ‘the main impediment to further progress is lack of funding rather than lack of knowledge’.
Sadly, evidence suggests the opposite may well be the case – for both. For example, Gerard Guthrie, who has already commented here, has written two important books ‘The Progressive Education Fallacy’ and ‘Classroom Change in Developing Countries’ that show how a lack of understanding of the important place of traditional epistemologies in developing counties such as PNG and elsewhere has seriously hampered educational development.
My own work on the sustainability of benefits from educational development assistance to Indonesia shows how little of the considerable effort and resources over 40 years has yielded longer term benefits. In both cases, my view is that we in the development community have not applied educational and local knowledge to the extent necessary, favouring instead imported, top-down, western economic and managerial approaches.
More funding in areas of education that are not well understood will likely lead to further waste and disappointment. More positively, the adoption of analytical approaches such as Problem Driven Iterative Adaptation advocated by Harvard’s Center for International Development and currently implemented by DFAT in Indonesia, may yield a significantly stronger base of understanding to achieve more in education and in other areas of development.
I’m not sure I agree with Garth’s comment about ‘well-understood’, at least in regard to education. The main benefit of aid to education has been to increase enrolment levels, especially at primary level (Riddell and Nino-Zarazua 2016). However, quality-focused aid trying to introduce Anglo-American (and Australian) curriculum philosophies, teacher education and classroom methods has been a widespread failure. Accumulating evidence of the lack of success in fundamentally changing formalistic classroom teaching has been ignored or downplayed. In PNG, this evidence goes back to the 1970s. I’ve also found evidence from over 600 research and evaluation studies in at least 31 other ‘developing’ countries that shows similar findings. Often projects successfully generate material and professional inputs, but I have not uncovered a single methodologically sound example of success in generating paradigm shift to ‘progressive’ styles, nor impact on increasing student achievement from trying to change teaching styles.
Actually, the evidence of failure is there and fundamental cultural reasons are understood, at least in the broad, but the evidence just happens to be ignored. Why?
Thanks Gerard,
Interesting comment. Education isn’t my specialty but my understanding is similar to yours. Aid – in aggregate – has been better at producing outputs than outcomes. That said there are good cases of aid-funded education work delivering education outcomes too, so it’s clearly possible.
Terence
Thanks for the comment, Terence. I’m always interested to see any examples of successful educational outcomes – what cases did you have in mind?
Hi Gerard,
This is an interesting read, although not all aid funded. Also note the external validity limitations of rcts discussed below.
https://www.povertyactionlab.org/policy-insight/improving-learning-increasing-motivation-targeting-instruction-and-addressing-school
Terence
Hi Terence
This is an interesting summary of school effectiveness research. Like most such research, the studies focus on managerial findings rather than classroom practice. While quite large compared to most developing country studies, the improvements indicated mostly fall below the 0.4 of a standard deviation recommended by Hattie as a cutoff for cost-effective investment. Encouraging but not convincing.
In any case, little of the research reported deals with the classroom teaching style and professional issues that I raised. The one area that does relate is teaching to the test. The assumption that this is a bad thing is culturally biased. In societies where poverty is endemic, public examinations are a high stakes opportunity that might provide a rare, and hopefully corruption free, opportunity for upward mobility, escape from poverty, and capacity to support family: one of the many things not well-understood in the western literature!
Gerard
Hi Gerard,
Final comment from me.
It seems that your position is, in a sense, not that far from Garth’s after all. You appear to believe, as best I can tell from your comments on pedagogy, that we know what to do, we just don’t do it.
Minor technical points from me:
1. Cost benefit will be a function of two things: (1) benefit & (2) cost. Small benefits may well be worthwhile if the economic cost is itself small.
2. The standard deviation is itself a function of variance around the mean. If variance is high, it is possible to achieve a non-trivial absolute benefit, on average, and still have a gain, in terms of standard deviations, that is not especially high. Under some circumstances (amidst a diverse population, say) I would be interested in knowing the substantive absolute improvement, more than some figure expressed in terms of standard deviations.
Anyhow, that’s mostly by the by. It’s been interesting to hear your thoughts on education.
Terence
Really sensible response, thanks Terence!
Thanks Sally
I take issue with randomised controls not having ethical issues – apart from who gets to select which village (just one example) has access to a particular programme that conditions are such that its not possible to compare outcomes, which is what RCTs aim to do. The notion that positivist methods can be applied to social research is questionable at best
Hi Suzanne,
Good to hear from you. Sorry we didn’t get to chat in Wellington.
With all aid projects some selection has to be made: which country to work in, which region to work in and so on. This is as true with projects that are evaluated using RCTs as any other method of evaluation.
With RCTs, once that selection has been made, the actual allocation of people, or villages, or appropriate units, to the treatment is done randomly. What would be a more ethical approach?
With respect to internal validity, with random allocation, so long as sample size is large enough, the group of people (or villages, or whatever) that receive the treatment will be effectively identical to the group that does not receive the treatment. (It is also possible to test for this, and control for this, at least on observed traits using regressions). That means it is very much possible to compare outcomes in this sense.
With respect to external validity, it is not guaranteed that a treatment that worked in one location will work in others. This is a serious limitation. Although: (a) it does not invalidate utility for individual evaluations and (b) can be overcome *to some extent* with meta-analysis.
There are many other problems with RCTs — there is much that matters in both aid work and development that they can tell us nothing about, they don’t answer how/why questions well — and so on. But their strengths are real enough. They eliminate confounding factors, they reduce subjective assessment (albeit not entirely in all instances).
As for positivism: it seems to me that the null-hypothesis significance testing, which forms an integral part of how people read the result of RCTs, isn’t really positivism at all, rather it’s Popperian or post-positivist.
But as to whether this broader family of approaches can be applied to social science…sure it’s applicability can be questioned, but so can that of post-structuralism, critical theory, interpretivism, and so on. Debates amongst philosophers will continue for a long time to come. I really enjoy listening to podcasts about this stuff in the weekends. For now though, at the very least I think it’s fair to claim that broadly empirical approaches to social science produce enough useful knowledge to warrant their continued use.
I’m very happy to concede that this should be done with humility about what we can and can’t know. This should be the case with everything in aid work.
Have a good week.
Terence
I agree with their emphasis on changing the rules of the international economic system but can’t we do this at the same time as providing useful development assistance? Like so many people the authors appear to have an inflated view of the scale of aid both in terms of donor and recipient income and energy. Given how little effort is put into aid it is amazing what has been achieved especially in areas that are well understood such as basic health services, water, sanitation, education and agriculture. The progress to date shows that the main impediment to further progress in all of these areas is lack of funding rather than lack of knowledge or international economic arrangements.
Thanks Garth, I think you make a really important point with the following:
“Like so many people the authors appear to have an inflated view of the scale of aid both in terms of donor and recipient income and energy. Given how little effort is put into aid it is amazing what has been achieved especially in areas that are well understood such as basic health services, water, sanitation, education and agriculture.”