A recent Devpolicy Blog post, “The science of scale: what works to implement effective education programs?” (hereafter, “the post”) discusses a session on scaling education interventions held at the 2025 Australasian Aid and International Development Conference. As factors critical for achieving scale, the article emphasises the importance of evidence, leadership, evaluation and collaboration among policymakers, implementers and evaluators.
What is scale? The core idea of scale in international development is bringing benefits to more people. Scale is defined here as the geographical spread of benefits to more districts, schools, teachers and learners. The OECD’s Glossary of Key Terms in Evaluation and Results-Based Management for Sustainable Development does not define the term scale. Its absence is consistent with the Glossary’s editorial approach to only include OECD countries’ agreed technical terms in common use.
That absence of agreement is revealing. The post to which I am responding, for example, does not define scale and that reflects the muddled state of affairs with scale where most development projects neglect to define it. Scale is linked to the idea of the sustainability of benefits, which is defined in the OECD Glossary as the extent to which the benefits of a change continue. However, sustainability and scale are different concepts: sustainability is the time dimension of change; scale is the spatial dimension.
Is scale grounded in science? Clear definitions are foundational in science but are often missing in the development and educational literature, and in professional practice. Definitions establish the boundaries of the phenomenon being studied, making claims about it measurable and falsifiable. Measurement for evaluation and research cannot produce reliable and valid data without clarity about what is being measured.
The post’s assertion that theory alone is rarely sufficient in complex systems is true. However, theory is the bedrock of science. From the limited research on scale in education, Cynthia Coburn’s work provides an example of theory-building. She argues that only defining scale as reaching more beneficiaries overlooks the challenge for project design and implementation activities to achieve deep and lasting change. Coburn conceptualises scale as having four interrelated dimensions — depth of change, sustainability, spread and a shift in reform ownership toward local actors. Scale depends on sustainability and is only significant if change is sustained in both the original and subsequent intervention sites.
The post’s focus on scale is timely. Donors’ current policy settings are far from adequate to address global development challenges. For example, Australia’s current International Development Policy does not present a policy position on achieving sustainability or scale, and the Department of Foreign Affairs and Trade (DFAT)’s International Programming Guide provides no guidance on either. Yet the development policy is based on its “grand-scale” objective of a peaceful, stable and prosperous Indo-Pacific, and on the Minister’s commitment to the Sustainable Development Goals (SDGs) that begin with the goal of ending poverty in all its forms — “everywhere”.
However, in an example of policy incoherence, DFAT’s Design and Monitoring, Evaluation and Learning Standards does set out standards for sustainability, but not for scale. Sustainability and scale are conspicuously absent in Australia’s International Development Performance and Delivery Framework, where quality criteria are limited to effectiveness, efficiency, gender equality and disability equity. Effectiveness, the extent to which a development intervention achieves its objectives and results, is rarely a relevant criterion for evaluating sustainability and scale unless they are clearly specified as project objectives.
Why sustainability and scale are so often ignored is puzzling. Both are central themes in addressing the global development emergency, according to the United Nations (UN) Secretary-General in his foreword to the 2025 Sustainable Development Goals Report. Yet the reality is that they have not been central in development projects. That omission raises the question: if sustainability and scale are central to addressing global development challenges, why are they so rarely addressed?
Evidence from Indonesia’s education sector illustrates the significance of that question. Between 1971 and 2023, Indonesia received donor support through 114 education projects worth more than US$6 billion.
Analysis of donors’ reports from this support confirms the post’s opening assertion that “… success at pilot level does not guarantee success at scale”. Donor completion reports judged 82% of their projects to be successful. However, success at project completion did not translate into sustainability or the scaling of benefits.
Only 23 of the 114 projects (20%) were evaluated at least two years after project completion for valid evidence of the actual sustainability of project benefits. Twelve of those 23 projects (52%) showed evidence of actual sustainability. Comparisons of donors’ attention to actual sustainability show that this dimension was evaluated by the Asian Development Bank in 11 of its 27 projects and by the World Bank in four of 44 projects. Australia evaluated one of its 25 projects for actual sustainability. The evidence for scale was far weaker, with only four of the 114 projects (3%) evaluated in this dimension.
The important place of political leadership in implementation is rightly stressed in the post. But political leadership can be unreliable due to short-term instability. Donors’ reports emphasise in-country educational leadership and local ownership of change to achieve sustainable benefits at scale. Educational leadership is critical to drive depth of change. Without practices informed by a depth of understanding of educational principles and local cultural values, there is a shockingly high risk of failing to achieve desired change, as Gerard Guthrie has demonstrated in his thorough analysis from all 142 developing countries.
Is scale grounded in science? The idea of a “science of scale” remains more aspirational than realised. The post’s conclusion that “ … the science of implementation and scale is finally taking off” is not supported by the evidence presented. Francis Fukuyama cautions about being “scientific” and ignoring culture in development in his book, State Building:
The effort to be more “scientific” than the underlying subject matter permits carries a real cost in blinding us to the real complexities of public administration as it is practiced in different societies (p. 123).
If educational development projects are to move beyond successful implementation to achieve lasting, sustained change at scale, they will need coherent donor policies that require, support and reward the professional work necessary for cultural relevance, clearer definitions, stronger theoretical and evidential foundations and excellence in project implementation and evaluation, including the evaluation of sustainability and scale after project completion.
And that matters. As the UN Secretary-General has pointed out, 800 million people still live in extreme poverty. Lifting people out of poverty requires focused interventions supporting benefits at scale from sustained access to essentials like food, clean water, healthcare, education and economic opportunities. The development policy drift away from a strong focus on the sustainability and scaling of benefits from development interventions places that poverty alleviation goal at risk.
I appreciate your post, Robert. It reminded me of this recent SSIR article, ‘Scale Really Matters’ – https://ssir.org/articles/entry/scale-really-matters. I liked this quote: “In this vision of scale, the role of the NGO—and, hence, of philanthropy—is to develop solutions that can be scaled via governments, prove and refine those solutions, and then work with government to achieve ownership and adoption.” It is a slightly different lens on the same topic, but I think scale and sustainability are critical to ponder. Thanks for prompting us to do so!
I appreciate your comment and had a look at the SSIR article you cited. It provides much food for thought. In international development there are special challenges that must be addressed: there has to be donor policy demands for sustainability and scale in the first place and these demands are diminishing. Why, I wonder. Second, assuming there is a supportive policy framework, we need sound project design and excellent implementation, and then sustained benefits from change before scale is possible. And I think trying to tease out general principles of scale must always be sensitive to specific disciplinary qualities and to the culture and belief systems where development and scaled benefits are intended to occur. It is rewarding to see that a discussion on these matters has emerged here – thank you!
This is a valuable piece. Thanks Robert!
It’s striking how little attention is given to learning from cases of long-term sustainable scaling.
I think there is another connection worth mentioning – sustainable scaling involves low-cost, large-scale systems change which scales because collective action which is culturally congruent and inclusive (and therefore locally owned) produces sustained flows of benefits by enlarging the common good.
Thank you for your positive feedback, Bill. In the case of donor-supported educational development projects, I fully agree with you about how little attention has been given to learning among the major donors from cases of long-term sustainable scaling and would go further and argue that there has been limited learning about long-term outcomes of success and failure, especially over the past two decades. Previous learning, in the case of Australian international development, encapsulated in the very clear and practical AusGuides, has been largely lost. Further, the point you make about action which is culturally congruent, inclusive, and locally owned is critical and one that I have drawn attention to in an April 2021 DevPolicyBlog reviewing the very important work of Gerard Guthrie on this theme.
Thanks for the interesting post. One thing I found striking was the timeline: going to 1971, this covers the INPRES school building era, which was entirely funded with domestic funds from oil revenues, as far as I recall. And there is a lot of serious evidence on the impacts of that program at serious scale with respect to time, location, and spillovers. What changed since then? Was it just the nature of how the money was spent, and that the supply side was the lowest-hanging fruit?
I was also surprised to not see any reference to John List’s writing on this topic or the work that has been going on for many years at Yale (through their Y-RISE initiative). Prof Imran Rasul’s recent piece in Science provides a good synthesis, in my view: https://www.science.org/doi/10.1126/science.aef8482
Best,
Ryan
Thank you for adding to this discussion, Ryan. The early INPRES experience is illuminating as it has parallels with the more recent large-scale government intervention during decentralisation of implementing school based management. The evidence there shows general success although it varies considerably across schools and is often fragile. In a similar way, INPRES schools, notwithstanding their benefits, were also ‘fragile’ in the sense that field observation demonstrated the deleterious impact of a lack of maintenance so common in infrastructure development. Thank you also for drawing attention the List’s work.