You should read “The Why and How of Foreign Aid“, published in the Journal of Economic Perspectives, partly because of who wrote it. Professor Dean Karlan was chief economist at the United States Agency for International Development (USAID) from November 2022 until he resigned in February 2025. His reflections are worth hearing and some of the paper’s best parts describe reforms written into USAID funding calls that were cancelled as the agency was dismantled.
The framework is straightforward. The “why” of aid is allocative efficiency: which problems, which countries and who benefits (recipient, donor or everyone, through global public goods). Cost-benefit analysis is the tool.
Productive efficiency is the “how”: for a given problem, how do we do the most per dollar? Cost-effectiveness analysis is the tool and evidence is central, using it where it exists and producing it where it does not. A program can be cost-effective and still fail a cost-benefit test if it efficiently solves the less important problem.
Sound familiar? It should. The Development Policy Centre’s first discussion paper, by Stephen Howes in 2011, argued that we cannot definitively answer whether aid works and that the more useful question is how to make it work better. Karlan’s version of this is that “Does aid work?” is no more informative than “Does medicine work?” We can test whether a particular drug treats a particular condition, and similarly whether a particular aid program achieves a particular change in outcome, but not whether “medicine” or “aid” works.
Howes organised effectiveness around recipient governments, donor agencies and their interaction; Karlan ends the paper urging donors to “look in the mirror” at their own capacity, Howes’ second determinant. Australia’s 2011 Independent Review of Aid Effectiveness made value for money a “fundamental operational principle” and said country allocations should be based on poverty, national interest and a capacity to make a difference: Karlan’s “why”, feasibility included.
Old wine in new bottles? Maybe a little, but Karlan adds at least three things Australian aid has lacked, at least in any serious form.
First, a candid account of why evidence struggles inside aid bureaucracies (doubly so, I’ll add, after mergers with other arms of the bureaucracy with often-conflicting objectives). Cost-benefit analysis asks a health or education team to justify its favourite program; cost-effectiveness accepts its goals and helps it do more. Perhaps this explains why agencies keep refining programs rather than reallocating between them, and the “Christmas tree” problem, where every aid constituency hangs its favourite ornament on a program design.
Second, a rule for how much to spend on evidence. Evaluation budgets are often set as a share of program cost, which Karlan says is a mistake. What should matter instead is how much future spending the evidence could influence and how little is already known: the larger each of those, the more it is worth spending on evaluation. And spend less where findings are unlikely to be used because the politics are already settled.
The returns can be large: Karlan cites a World Bank estimate that investing under 1% of project costs in research design and data systems can increase impact by more than 50%. Yet new analysis from the Center for Global Development estimates that rigorous evidence generation accounted for less than 0.6% of disbursements across nine major donors from 2010–25. Of the two biggest funders, USAID is gone and the United Kingdom’s Foreign, Commonwealth and Development Office faces major cuts.
Third, a shift from evaluation as process and accountability to credible evidence as a public good, built with recipient governments. Karlan, like many others, argues that more leverage lies in improving how recipient governments spend their own, larger budgets.
This logic is weaker where aid is a large share of budgets, as in parts of the Pacific, but another applies: with many donors and small administrations, donors consume much of governments’ limited bandwidth. Evidence capability built inside government, on its own priorities, might give some back, though if done poorly it is just another demand on agencies that are often small or weighed down by wider governance challenges.
Karlan points to embedded evidence labs set up inside the Central Bank of Nigeria’s consumer protection department, Peru’s environmental regulator and Rwanda’s education ministry. There is nothing comparable in the Pacific yet, and where we did do this well we pulled back: one of Karlan’s success stories, Indonesia’s shift from in-kind rice to electronic vouchers for almost 19 million households, drew on evidence Australia funded from 2012, first through AusAID and then through the Department of Foreign Affairs and Trade (DFAT), which absorbed the aid agency in 2013.
On evaluation and evidence, Australia’s record is weak. Its performance systems have long centred on accountability, and project ratings became less rosy once independently reviewed. The 2011 Review found little use of evaluations in decision making and said almost nothing about impact evaluation.
DFAT’s Development Program Committee is now meant to “drive evaluation quality and … ensure that evaluation findings inform policy and program decision making”. Managers report using almost all evaluations, 45 out of 46 in a 2024 review, but that same review found that perception of quality “appears to be linked to usage more than actual (independently assessed) quality”.
The last full assessment rated 70% of the 40 evaluations completed in 2022 as adequate or better, roughly unchanged in a decade (71% in 2017 and 74% in 2012). The most recent review rated only eight of 20 evaluation plans (40%) as adequate, and the term “impact evaluation” does not appear once across those three reviews. Poor evaluations, whatever the method, are of little value and at worst harmful. The aim should be better evidence, not more evaluations.
The Office of Development Effectiveness, whose evaluations the then Secretary Frances Adamson called “robust”, was abolished in 2020 and replaced by a unit with about half the staff, likely leaving fewer experts in-house to do the work and mentor others. Rigorous impact evaluation remains exceptionally rare: under the 2023 policy, DFAT committed to it for only a small number of significant investments, and none appears to have been published.
As the Pacific’s largest donor, Australia is unusually well placed, funding similar programs (for example social protection, health, labour mobility) across small countries where data are often sparse. Karlan’s case for multi-site evaluations, which share fixed costs and allow meta-analysis, is even stronger where small populations make single-country trials hard.
He asks evaluators to set out exactly what it took to deliver a program: how many staff, with what skills, with what equipment and at what cost. Without those details, a result cannot be priced or repeated somewhere else. We should be asking this of all our major investments, especially given the variability in prices and distances and increased market concentration in aid procurement. Embedded evidence teams in partner finance, education, labour and other ministries might serve Australia’s strategic “why”, but more importantly the development one, as knowledge is one of the few things aid can buy that benefits everyone.
The principles of aid effectiveness were largely settled here in 2011: allocate by poverty, national interest and capacity to make a difference, insist on value for money, and look just as hard at the donor’s performance as the recipient’s. But we never built the machinery to give them any bite: an aid program that funds high-quality evidence because it improves future decisions, and helps partner governments do the same.
On evidence, as distinct from accountability, we have gone backwards and fed the growth of a monitoring, evaluation and learning complex of frameworks, indicators and reviews that consume budgets, time and bandwidth and rarely tell anyone what works. The Minister for International Development Anne Aly says we must “make every development dollar work harder, reach further and deliver the greatest possible impact”: hard, if not impossible, without knowing which dollars work.
I’d be remiss not to acknowledge the ambition of Assistant Minister Andrew Leigh and the Australian Centre for Evaluation (ACE) to rekindle this fire. But aid is the big gap in this work. The 2023 development policy said DFAT would work closely with the newly established centre, yet the seven randomised trials ACE has supported so far cover employment, service delivery and regulation, with four more underway.
None, as far as I can tell, relates to aid or development policy. When I worked on impact evaluation policy in government well over a decade ago, AusAID clearly led the public service on evaluation. Perhaps ACE can help Australian aid get back there. Karlan’s paper offers a helpful starting point, if we choose to use it.
Dean Karlan is presenting on a hybrid panel at the 2026 Australasian AID Conference, ‘The future of aid: influence, not investment?’ on Friday 4 December. Purchase your tickets before conference registrations close on 25 November.