The federal budget has now been presented. Analyses of Australia’s foreign aid commitments are currently popular topics for discussion in the development community. Curiously, even though sustainable growth is a major purpose of Australian aid, there is comparatively little discussion about the evidence for the sustainability of benefits as a justification for further aid.

Understanding sustainability

To assist in understanding sustainability it is helpful to begin by sharpening our focus. First, we need to focus on the concept of sustainability. Many will think immediately of environmental sustainability, however, the conception of sustainability explored in this post is different. It is encapsulated in the clear definition [pdf] used by the former AusAID: “the continuation of benefits after major assistance from a donor has been completed”.

Second, making generalisations about sustainability is risky, given the kaleidoscope of countries involved, different cultures, development sectors, and the variety of assistance provided. Focusing on a specific sector, as has been done so well for education, and further narrowing our focus on one country may help our understanding of the dynamics of sustainability. The focus here is on Indonesian education.

But pause for a moment and look at the photograph above. The bridge on the right is an example of the kind of sustainability being considered here. The bridge was constructed during the Australian-funded Indonesia Steel Bridges Project. Over 2,000 bridges were built from 1980 to 1992 in 24 provinces. Overshadowed by the gleaming newer structure, the older bridge contributed to the circumstances for the creation of that new bridge by supporting a two-lane highway in Sulawesi.

Here we have strong evidence of the sustainability of benefits from the bridges project. First, the older bridge is clearly still there and the cars indicate it is doing its intended job. Second, the existence of the new bridge is an indicator that the older bridge has made a contribution to development in the region. Finally, an AusAID evaluation supports the sustainability of benefits from that project.

The sustainability of benefits from education projects

This is positive news from an infrastructure project. But is there comparable evidence of the sustainability of benefits from assistance to Indonesian education? There is actually very little and much of it is in donor reports, typically Independent Completion Reports and Project Completion Reports. However, these reports are not independent. They are commissioned by the donor, reviewed by the donor, and often further reviewed by development partners.

To assess the evidence of sustainability from these sources, I located and analysed 48 donor reports of education projects from 1973 to the present. These reports represent the work of 10 different donors over more than 40 years.

Key findings from my analysis are:

• 64 per cent of projects were rated as likely sustainable; the remaining projects were either rated as uncertain (23 per cent) or negative (14 per cent).
• Most evaluations were conducted at or close to project completion. In other words, there is only an estimate of potential sustainability.
• The actual sustainability of benefits has not been assessed for most projects. Of the 13 projects that were evaluated two or more years after completion, only half were considered to be actually sustainable.

These findings of potential sustainability are similar to the findings of a larger and broader study for the Independent Review of Aid Effectiveness in 2011, but a 2010 ADB post-completion evaluation was more positive with 63 per cent successful or highly successful.

Is achieving actually sustainable benefits from half the projects reviewed good enough? How do findings compare with government programs and projects in Australia, for example? We are not really able to answer such questions. However, in its latest education project design for Indonesia, Inovasi [pdf] the Department of Foreign Affairs and Trade (DFAT) signals its own bleak assessment of sustainability. One of the Inovasi design’s chapter headings bluntly asks: ‘Challenge: Millions More Attending School But Not Learning – Why Haven’t Reforms Taken Root?’

With more than 40 years of Indonesian development experience, it is reasonable to enquire why we do not have stronger evidence of the actual sustainability of benefits from aid to education. Why have the ten donors in the analysis, who have provided over USD 4.5 billion in assistance since 1973, been so incurious and for so long about sustainability arising from this massive investment to support Indonesian education?

What might be done to assist in sustaining the benefits from current and future educational development assistance?

I suggest five strategies for consideration until such time that we have a stronger base of evidence for policy and practice, something that DFAT’s Inovasi could provide in the future.

1. Approach educational development with a better balance of cultural, political and technical realities in both design and implementation. Recent Indonesian theses, one a study [pdf] of the Australia Indonesia Basic Education Program and the other [pdf] of the sustainability of active learning in North Maluku, call attention to the need for this approach. Failure from ignoring culture is also demonstrated in a 2005 study of schools in Java.

2. Build strategies from a deeper understanding of local and institutional cultures, of what we know works, where it works, and why strategies work. The outstanding example of the need for this approach in education is illustrated in Gerard Guthrie’s well-researched book, The progressive education fallacy in developing countries, about the challenges of educational development in Papua New Guinea.

3. Explore ways of democratising sustainability by inculcating the idea of shared responsibilities among all beneficiaries to move away from a dependency mentality where beneficiaries assume all responsibility rests with government and donors.

4. Consider the implementation of culturally suitable concepts drawn from education. Some concepts offer the potential for learners, teachers and others to act as sustainable agents of change themselves:

• Lifelong learning: The idea of moving away from a focus on teaching towards sustaining learning throughout life.
• Sustainable assessment: The recent idea that assessment can be designed to contribute to meeting future learning needs.
• Feedback: Learning is not sustainable if it requires continuing support and feedback from teachers and development experts. Recent work on feedback is focusing on the contribution of others to lifelong learning through sustainable assessment.

5. Pay attention to sustainability at all stages of the project cycle and ensure it is tightly coupled with the deepening of change.

There is presently a lack of congruence between aid funding concerns, and donors’ soaring policy rhetoric about sustainability and poverty reduction on the one hand, and the neglect of rigorous inquiry about the actual sustainability of benefits on the other. This curious situation also raises concerns about the effectiveness of educational development. AusAID began to address such concerns in 2012 with its enquiry into educational development. Sadly, that positive initiative seems to have gone nowhere.

Robert Cannon is an educational evaluation adviser to USAID in Indonesia.