Comments

It is gratifying to see this open debate about aid to schools in Indonesia. Discussion can lead to better outcomes, particularly for the ultimate beneficiaries of our aid, the children and their families in Indonesia. I would like to respond to both Marc Purcell and Matthew Morris and begin by assuring Marc that I am as much committed as he is to ensuring that the 'Dewi's of Indonesia' can enrol in schools, complete their education in a challenging, caring, pleasant and non-threatening environment, and with access to such basic facilities as toilets. I believe that constructing and maintaining schools should, by now, be a routine responsibility of the Indonesian government. It is because I am convinced that the present AusAID plans do not reflect current good practice in basic education development, and the evidence of effectiveness that we now have about basic education in Indonesia (Matthew's issue), that I wrote the article for The Australian. Restrictions of newspaper space and editing meant that the evidence could not be presented in that article. This is regrettable but reflects the reality of editorial policy. What is the evidence? First, it is a matter of concern that very little independent research has been published on Indonesian education upon which to make good planning decisions. Second, much of the evidence that we do have comes from the 'grey' literature, the studies and evaluation reports commissioned by donors but which have not been subjected to independent peer review. Nevertheless, the cumulative evidence from these studies provides reasonable clear evidence that: (1) centrally managed, large infrastructure projects are less effective than smaller bottom-up projects, based on educational principles, that focus on schools, teachers and communities, and (2) the Government of Indonesia's record of managing development projects is poor. I can provide a copy of a study I did for the World Bank and AusAID on this: email me at cannon@indo.net.id. Marc, you pick up some of these themes when you write: "The evidence shows that funding and empowering Indonesian civil society, like parent's groups, to monitor education services and hold their government to account is the best way counter corruption". Third, information that enables us to make evidence-based decisions about improving the quality of education is now more freely available than at any time. A very recent example of this is the large international study prepared by McKinsey and Company on improving school systems. The study was released in November 2010. The McKinsey study shows that many school systems, not showered with extra money, actually did better than others such as the UK and the USA where standards have slipped. What does make schools better is strikingly similar across the 20 different education systems that McKinsey studied. Successful strategies depend very much on a system’s current stage of development but, unsurprisingly, educational interventions such as curriculum revision, teachers’ skills, and the assessment of learning figure largely. In this finding, the McKinsey study echoes other large studies such as the 2007 International Project to Frame the Transformation of Schools. This Project concluded that no single strategy or collection of strategies could be successful unless there is an alignment of change efforts and the focus is on the student. To paraphrase Riddell's question: “How can aid to [Indonesian basic education] be made more effective?” (p. 257), the answer, based on my experience in Indonesian basic education and from my own research and evaluation studies, is to work from the bottom-up (teachers, schools, Districts); empower and support local networks/communities of practice for dissemination of good practices in educational development; avoid one-shot or limited, decontextualised training programs of the kind now proposed by AusAID; and work on the 'community of practice' principle including whole school and whole District development. This requires capacity development; not only the capacity of individuals, but also of their organisations and the institutions of educational administration in Indonesia. And when we talk of consultants (highly paid or not) let us think mostly about the very talented Indonesian consultants, most of whom are actually practicing teachers, principals or academics with relevant front-line implementation experience. Perhaps 'facilitator' is a more appropriate term. I am emphatically not encouraging more jobs for expatriates. I need convincing that AusAID is not being driven by the ideologies of development embedded in the Paris Declaration and its related Jakarta Commitment. I also suspect the political need to spend increasing sums of money on aid is also a strong factor in what we now have. $500 million is not only a lot of money; it is also a suspiciously 'round' figure! That is, we have supply-driven development rather than demand-led (which does not mean the dominant, centralist demands of Jakarta-based management and political elites). I really wish I could believe Marc's views about disaster proof school construction, provision of facilities for the disabled, provision of toilets and the freeing up of resources for teacher training. For a view about construction issues, a reading of the Independent Completion Report of the Australia Indonesia Basic Education Program (available on AusAID's web site) presents worrying information about lack of earthquake resistance, poor site selection, limited provision of toilets, lack of availability of enough engineers to supervise and assist with construction, and contracting limitations. Frankly, I think it a credit to those contractors and their national and international staff involved in AIBEP that they achieved what they did in such a complex environment. The same can be said for the ICR team that has worked under incredible time constraints to produce their evaluative report. But even when schools have been well-built and corruption of processes eliminated what then? Is there a firm guarantee that the Indonesian Government will maintain these new facilities? No. Will we see, as I observed in 2008, schools we had rehabilitated under the ADB Decentralized Basic Education project, being rebuilt or in urgent need of refurbishment only 4 years later because of no or poor maintenance? This is not the kind of “sustainable” development AusAID says it is committed to. Finally, does AusAID have a guarantee that the Government will use its funds freed-up by Australian aid for financial support for teacher training as intended, or is the view of some Indonesian Government officials correct that the education budget provided by their Government will actually be reduced by an amount equivalent to that of Australian support? There is much to discuss and much to remain concerned about in the current AusAID proposals.

I enjoy reading public commentary on aid issues, including Robert Cannon's op-ed for The Australian that questions Australia's approach to supporting education in Indonesia, and Marc Purcell's persuasive rebuttal. Mr Cannon argues that 'the evidenced-based, educational perspective on schools and what happens in them is missing.' I read Robert Cannon's op-ed several times and agree with this statement - I could not find much evidence for many of Mr Cannon's assertions. If Indonesia scores poorly on Transparency International's perceptions of corruption, is this evidence that there is corruption in this particular education program? If Australia's domestic programs are implemented poorly, and this is debatable, is this evidence that international aid programs will be? If technical assistance is the solution, then where is the evidence that this will be more effective than the current strategy? If Mr Cannon would like to have an evidence-based debate on the effectiveness of Australia's support for education in Indonesia, then a good place to start would be to look at the experience of the existing project over the last four years. AusAID have, commendably, published the Independent Completion Report for the first phase of the project, which sheds some light on some of the concerns that Mr Cannon raises. http://www.ausaid.gov.au/publications/pdf/2010indoaibepicr.pdf The evaluation report does support Mr Cannon's concern that there is insufficient funding for school maintenance, with most Indonesian government funding going to teacher salaries. (Though the report is less sanguine about the effectiveness of technical assistance in helping the Indonesian government to remedy this - something for AusAID to think about for Phase 2) This evaluation report does not identify any specific cases of corruption, but it does commend the safeguards put in place to manage fiduciary risks in the project. The evaluation concludes that the project has been 'implemented effectively and efficiently, and is a successful program.' This included building more than 2,000 schools and creating over 330,000 school places. The project does include technical assistance, including on how the Indonesian government can fund education and maintenance, and the evaluation report comments are generally favourable on the value of this component of support. Yet on the issue of technical assistance, it is also worth noting the broader evidence on the efficacy of this instrument - for a good summary see Roger Riddell's 'Does Foreign Aid Really Work?' - which suggests that it is not the panacea that Mr Cannon suggests. We need more discussion on aid in Australia, and like Mr Cannon I agree that it should be grounded in good evidence!

Paul raises an interesting point, which touches on an important policy issue for PNG - which is “ what should the future role of government be in delivery of essential social services”. The government in PNG has traditionally been responsible for policy, regulation, standard setting, funding and service provision (although much of the rural health services have been, de facto, sub-contracted to churches to provide). In several countries, the funding, policy and regulatory roles have been separated from service provision and the provision of health care services is contracted out with external (private or public) institutions. The role of government changes from one of managing health care provision into one of purchasing health care services from different providers. The government retains certain powers, including, controlling market entry ( e.g. clinical authorisation licenses ); reviewing prices and setting minimum quality standards. The PNG government struggles to deliver services to its rural population. It already, in effect, contracts out services to churches, and in other areas, mining companies and plantations informally step into the breach, and provide services to populations in their immediate impact areas. There is already some policy dialogue on the issue of Public Private Partnerships. The potential for a much more pluralistic approach to health service provision is worth investigation. This could include churches, NGOs, mining and resource companies, and private health providers. However, it is not without risk – as cost escalation can be an unintended consequence if payment mechanisms and incentives are not carefully thought through. In addition government capacity to design, implement and monitor contracts would be needed.

Thank you for an insightful piece Peter. 'Developmental risks' indeed. More boldness and better analysis of inequality and insecurity for the poorest people in developing countries is required in Australia's aid program. This report was particularly important as the sector considers the role of ODE in the Aid Effectiveness Review. An interesting development that might take some of Peter's arguments forward is a new report from ODE, still in draft form, about whether AusAID should have a strategic framework for dealing with civil society. The draft report is entitled 'Working with the State is Not Enough', and in many respects it is the missing piece in the puzzle - http://www.ode.ausaid.gov.au/publications/index.html#progress. Taking civil society seriously in the aid program is overdue. How do weak governments improve? By technical assistance from other governments, or by demands for better governance by empowered citizens? Surely by both, but without pressure from the domestic constituency, reforms will falter. The latest thinking from AusAID on governance recognises this ('Power to the People' - http://www.ausaid.gov.au/publications/pubout.cfm?ID=9742_5949_4436_8120_49&Type=PubKAG), but still fails to think of ways to promote bottom-up governance by engaging civil society. Presumably because such a focus is messy and human, but that is the game we are in.

This is a very interesting and timely idea. I wonder if and how we could translate it in a humanitarian context? Perhaps not at the onset but later in the emergency cycle? The sooner the better to avoid the wait and see attitude of beneficiaries, what they call "attentisme" in Democratic Republic of Congo.

Good to see the responses to my posting. I am just back from a week in Dili, Timor-Leste, conducting interviews for a workshop on a related issue - how to provide demand-focused technical training. Informal feedback from colleagues I spoke with is that big changes are still needed there in how aid is delivered. The use of a hybrid system of COD in a post-conflict setting will require careful analysis and innovation. Both parties will also need to negotiate changes if it is not working as expected. Better and more independent evaluation tools will be needed to provide feedback. However, the longer-term benefit must always be kept in mind - the full acceptance by the recipient government of the funding and implementation of the intervention. This is a key test of aid effectiveness. The change in the mode of aid delivery that COD encompasses is akin to the fundamental reforms to labour exchange services in Australia over a decade ago. Australia led the world in these reforms, reviewed favourably by the OECD in its 2001 report Innovations in Labour Market Policies: the Australian Way. The switch from the direct delivery of services to job seekers through the Commonwealth Employment Service (CES) to outsourcing these services to a network of for-profit and non-profit organisations was a huge change. The new focus was on payment for outcomes - jobseekers in jobs, training or education more quickly. Switching from a centralised funder-driven model of service delivery to a ‘steering not rowing’ focus was difficult. It took a number of attempts to finetune the new arms-length system. Above all, it required within the funding department a change of mindset from service deliverer to service designer based on high-level conceptual skills It also took alot of ‘learning by doing’ about what worked and what did not to design tenders that worked more effectively over time. These reforms were an outgrowth of a period of experimentation in public policy in Australia. Is the Australian Public Service up to the challenge in the current era?

Migration to urban areas is a normal phase of development and so is the use of land for more productive uses. I agree with you that GDP should not be the only measure of development, but even Bhutan, which came up with the happiness index , is now heeding traditional national accounts metrics.

Mauri Theo, Thanks very much for this wonderful article. Kiribati has just entered into a Tourism Partnership Initiative with the Maldives and a team of tourism experts have just left us after assessing our tourism potential in resort development. A group of investors will follow up in January 2011. Your report is very timely and useful in our dealings with the Maldivians. Wishing you a Merry Christmas, Tara

COD Aid deserves a go! Coffee connoisseurs can get their cappuccino, café latte, or the like in nearly every little town on the planet. And while the price of a café latte differs considerably across locations, the quality – if my tastebuds are any guide - does not. The same can hardly be said of a classroom-experience in the local primary school. And the reasons may rest with incentives – the café thrives on serving good coffee; the primary school may not. Lots of funds have been spent training teachers, equipping schools, and improving the curriculum to improve literacy and numeracy in the developing world. This, using my café latte example, is akin to increasing the quantity of beans, the size of the coffee grinder, and possibly the dispenser to improve the quality of coffee produced. Absurd you might think. But this is exactly the strategy being pursued by donors such as Australia in Papua New Guinea. A lot is made of the poor quality of teachers for the poor educational outcomes in PNG. However, I find it hard to believe that the level of skills required for delivering a simple lesson in arithmetic or English is any more demanding than those for making a good café latte. Sure – they are very different skills, but which of the two is more demanding is debatable. Good coffee gets served cash on delivery. (BTW, no one has yet given me coffee on credit – but it may just be me!) So why not try the same for primary schooling? I have suggested a small pilot in PNG to test the effectiveness of COD in increasing the quantity and quality of throughput from primary schools. And it is only through carefully designed experiments such as these that we will know if COD works. It is time to roll up our sleeves and give COD a go.

Success story? Rapid urban growth and loss of customary lands to privatization ( = less land available for food plantations...implications for food security!) as contributors to expanding squatter settlements...is this success? I thought we were moving away from a GDP focus to measure development?

The Center for Global Development have a really good FAQ on COD Aid that provides some answers to the questions raised in the previous comment. The full FAQ can be found at http://www.cgdev.org/section/initiatives/_active/codaid/faq#9 <blockquote>How can the country be expected to make progress if it does not receive payments until it has already achieved some of the goals? Countries that enter the COD Aid contract are not starting from a blank slate. They already have their own programs financed with domestic and foreign resources. The COD Aid funds provide an incentive to make existing technical assistance and educational investments “pay off” in improved outcomes. Once the COD Aid payments start flowing, these funds can be used to sustain and further accelerate progress. Wouldn’t there be a big lag between investment and the payoff to the country? This depends on the appropriate time frame. Most development programs have to be thinking of 5 to 10-year horizons, and any program aimed at making substantial changes in institutions or behaviors also has to consider long time frames. The lag in payments is likely to make the contract seem unattractive to short-sighted leaders, but very attractive to those who have longer-term visions. This is, then, another attraction of this approach since it is most likely to be taken up by the same leaders who are most likely to use it well. What if a country doesn’t meet the goal? The COD Aid contract is not an “all-or-nothing” form of assistance. A country would be paid a set amount for each additional unit of progress. So, rather than “passing” or “failing,” the recipient is rewarded in proportion to the amount of progress achieved. Governments only control a fraction of the factors that affect outcomes. How does COD Aid account for this? While it is true that governments do not have full control of all the factors that affect outcomes, there would be no reason to provide aid if funders didn’t believe that governments can influence those outcomes. The outcomes chosen for COD Aid agreements should be outcomes that funders care about and over which governments can have significant impact. The funder may end up paying for progress that is not directly attributable to government action but we view this ‘risk’ as more acceptable than the ‘risk’ of paying for inputs that may have no effect on outcomes at all. What if a drought or other unforeseen shock keeps the government from achieving progress? Sharp changes in international interest rates or terms of trade and natural disasters such as droughts can interfere with development programs. The COD Aid contract should contain contingencies for adverse shocks that interfere with measuring and verifying the outcome indicator. However, COD Aid funds should never be disbursed except against progress measures. If emergency programs are necessary to assist a government, they should use a different aid mechanism rather than compromise the credibility of the COD Aid agreement. Is lack of money really the problem? What about insufficient demand, political obstacles, or weak technical capacity and coordination? Lack of money is not necessarily the problem. The funds disbursed under COD Aid provide an incentive for the country to undertake the right diagnosis and design effective strategies, whether or not those strategies require additional funding. For example, if the only obstacle is political will to enact a particular law or enforce specific management practices, COD Aid would provide leaders with an incentive to take those measures. In such a case, the funds that the government receives for successfully improving educational outcomes could then be applied to other public priorities. In most cases, however, some funding is still necessary though not sufficient for improving outcomes – whether those funds go to additional training, more staff, infrastructure, supplies, research, information systems, administrative reforms, supply incentives, or demand incentives.</blockquote>

Interesting idea, appears to raise a number of implementation issues, though I haven't studied in depth and may be missing something 1. Cash flow. Results aren't instant. There are often long time frames to achievement of results and the recipient is likely to be at an inherent disadvantage in obtaining funds. Possibly this would need further loan aid funding simply to cover the cash flow, more agreements, more administrative complexity. And the longer the time frames the greater this cost. 2. Impact of failure to achieve results. What happens when the aid recipient fails to achieve outcomes at all? Examples are legion, and the more difficult to achieve results, the greater the likelihood of failure. Using the example above, does the recipient default on the loan? Insure against failure (a further overhead)? [Also that nice issue about what is results tied in, do we measure something hard that resembles what we're really looking for, or something easy to measure (and achieve) that isn't] 3. Horse before cart? Kinda related to point 1 above. Where aid is intended to build capacity this may generate change, which may improve performance, with risks in both connections. Donors hate this, they want to see aid deliver results in terms of performance. But commonly the capacity fix (e.g. the old chestnut training) is needed as the hoped for catalyst for change (e.g. new techniques) to improve performance outcomes. But if the aid funding is tied to performance, the process can't start.