Comments

I think I'll have to agree with Laurence here, UNDP has to bear some of the blame for inaccurate statistics. I run a multimedia company in PNG and we have been invovled with SMS marketing campaigns for the last 6 years, even before Digicel came into the market. Last year we recently approached UNDP here in Port Moresby about using SMS for awareness and statistics for health data. Specifically in relation to health statistics we proposed a wheel chart that could show a specific code in relation to a health statistic being reported which could be sent as a text to a specific given short code. You can have a look at the proposal here at page 4: http://www.masalai.com.pg/projects/pdfs/sms_health_communications_undp_10061.pdf However after sending in the proposal last year we never got a response about it from the person responsible. We found out just last week that she has now left for further studies so I am trying to work out again who to speak to at UNDP since her position now needs to be readvertised. To be honest its not totally UNDP's fault if an employee decides to resign, I guess my point is more that we have the technology to make data more accurate and I would like to know if anyone readung this could maybe point us in the right direction to see if we can make this work. On a technical level we work with www.mobimedia.com.au who are the SMS platform suppliers to Digicel for all their countries in the South Pacific. So we have the presence and technology to implement this SMS Data Collection System in nearly every network in the South Pacific. John Burton, could you please have a read of my document and advise on your thoughts. Regards, Emmanuel Narokobi Masalai Communications

Recent reports of a massive shortfall in teachers in PNG. <blockquote>There are vacancies for 11,000 teachers nationwide and, with only 1,200 leaving teachers colleges to join the service, there would be a massive shortage of teaching staff, the PNG Teachers Association said yesterday. Association general secretary Ugwalubu Mowana said that there had also been a huge flight of teachers from the service, with 1,700 teachers resigning last year. He said rural-based schools would be the hardest hit as teachers preferred to teach in urban schools.</blockquote> http://pidp.eastwestcenter.org/pireport/2011/January/01-20-10.htm

UN statistics are notoriously flaky, for reasons unknown to me. The FAO figures on land use and land use intensity in PNG are just plain wrong, yet year after year they are reproduced. Below is an extract from Food and Agriculture in PNG, Part 1, People, Land and Environment by Bryant Allen and Mike Bourke pp35-46. <blockquote>Significant differences exist between statistics presented by the Food and Agriculture Organization of the United Nations (FAO) on land use in PNG and those that can be derived from PNGRIS. The FAO statistics require comment because they are easily available from the FAO website and because they are frequently quoted in PNG Government documents and consultants’ reports. In 1999, the FAO figures estimated ‘agricultural land’ in PNG to be 760 km2 and ‘arable land’ as 60 km2, compared with the PNGRIS estimate of 117 858 km2 ‘cultivated land’. This is a difference of several orders of magnitude. Some of the difference arises because the FAO figures refer to ‘agricultural land’ and the PNGRIS figures refer to ‘cultivated land’. FAO defines ‘agricultural land’ as the sum of the areas of ‘arable land’, ‘permanent crops’ and ‘permanent pasture’. ‘Arable land’ is defined by the FAO as ‘land under temporary crops, temporary meadows for mowing or pasture, land under market and kitchen gardens, and land temporarily under fallow (less than five years)’. The FAO category of ‘agricultural land’ excludes the large amount of land in PNG that is under fallow for periods of more than five years. In contrast, the PNGRIS class of ‘cultivated land’ includes all land in fallow. If the PNG population is brought into the analysis, serious problems that go beyond definitions are revealed in the FAO figures. For example, if around 0.08 hectares of land is cultivated for food production for every person in PNG (a reasonable estimate), and with a rural village population of 4.2 million in 2000, it follows that there had to have been at least 336 000 ha, or 3360 km2, of ‘arable land’ in PNG in 2000,2 not 60 km2, or even 760 km2. Furthermore, if the area of ‘agricultural land’ in PNG was only 760 km2, as the FAO figures state, and the rural village population was 4.2 million in 2000, then the average population density on ‘agricultural land’ in all PNG would have been around 5526 persons per square kilometre. This population density is not reached anywhere in PNG, nor in many rural places in the world.</blockquote>

I said that "it is awfully difficult to get empirical data from local reports into international ones" and I stand by this. You may well have put your finger on it in respect of UNDP procedures, but it remains the case that half the battle lies within people's minds. A returned post-grad reacting to the latest airing of a report on poverty in PNG told the Post-Courier in 2008 "I have just returned from ANU (Australian National University) after completing my studies. These so-called experts on PNG affairs continue to amaze me." Well obviously not enough, we might feel. The late Susy Bonnell told of an instance when she worked in Welfare in PNG years ago. The annual figure for infant mortality, I think, was handed to the secretary to be included in statistics to be forwarded to UNDP or a like agency. "It's gone up since last year, change it" was the reply. The PNG media are not much help, as most journalists appear number blind. (One wrote a story about a 50 metre high dwarf last year.) Whowever else has a hand in this, it's intriguing that the line on the UNDP spaghetti graph is monotonic whereas the actual (made up) data (Fig. 2) at least wobbles up and down. UNDP is clearly at fault here for not ensuring that the smoothing doesn't occur within its own reports.

Let me start with a disclosure. I have known Robert Cannon for many years and hold him in high regard. There are surely few expatriates who can match Robert’s practical, on-the-ground, experience of Indonesian schools or his passion for good development practice. He knows whereof he speaks. The argument that spending $222 million building schools in Indonesia frees up GOI development funds for other purposes is spurious. We could as easily spend it on paying for monthly rice supplements for teachers, or blackboard chalk or any number of other routine expenditures. If that is the basis of our contribution, let’s call it what it is, budget support, and stop trying to justify it as anything else. If however, we are concerned about aid-effectiveness surely we must concentrate on those things that make the most difference and in which we have a comparative advantage. It seems highly improbable to me that this lies in school construction. Indeed, I am struck by the contrast between the Bill and Melinda Gates Foundation approach to education which has the elegantly simple objective ‘an excellent teacher in every classroom’ and an ‘education’ project which scarcely mentions teachers or for that matter students. Robert is right, there is copious evidence of what works in education and top of the list is teacher development. This is something that Australia has a long track record of successfully supporting in Indonesia and many other countries. Not to focus on our comparative advantages and experience is to short-change the Dewi’s of this world.

You're right that any mention of poverty in PNG is typically met with hostility but I'm unconvinced that this explains the failure for the HDR to report on the little empirical data that exist. I know of no instances of PNG government staff vetoing the inclusion of data from the report (though I'd be interested to hear of any, if you do). The fault instead lies with the UNDP, whose data operations are overly centralized at headquarters in NYC, and whose country staff lack adequate knowledge of country-level data and are thus ill-equipped to report to headquarters on what data sources should be used in the report.

It is gratifying to see this open debate about aid to schools in Indonesia. Discussion can lead to better outcomes, particularly for the ultimate beneficiaries of our aid, the children and their families in Indonesia. I would like to respond to both Marc Purcell and Matthew Morris and begin by assuring Marc that I am as much committed as he is to ensuring that the 'Dewi's of Indonesia' can enrol in schools, complete their education in a challenging, caring, pleasant and non-threatening environment, and with access to such basic facilities as toilets. I believe that constructing and maintaining schools should, by now, be a routine responsibility of the Indonesian government. It is because I am convinced that the present AusAID plans do not reflect current good practice in basic education development, and the evidence of effectiveness that we now have about basic education in Indonesia (Matthew's issue), that I wrote the article for The Australian. Restrictions of newspaper space and editing meant that the evidence could not be presented in that article. This is regrettable but reflects the reality of editorial policy. What is the evidence? First, it is a matter of concern that very little independent research has been published on Indonesian education upon which to make good planning decisions. Second, much of the evidence that we do have comes from the 'grey' literature, the studies and evaluation reports commissioned by donors but which have not been subjected to independent peer review. Nevertheless, the cumulative evidence from these studies provides reasonable clear evidence that: (1) centrally managed, large infrastructure projects are less effective than smaller bottom-up projects, based on educational principles, that focus on schools, teachers and communities, and (2) the Government of Indonesia's record of managing development projects is poor. I can provide a copy of a study I did for the World Bank and AusAID on this: email me at cannon@indo.net.id. Marc, you pick up some of these themes when you write: "The evidence shows that funding and empowering Indonesian civil society, like parent's groups, to monitor education services and hold their government to account is the best way counter corruption". Third, information that enables us to make evidence-based decisions about improving the quality of education is now more freely available than at any time. A very recent example of this is the large international study prepared by McKinsey and Company on improving school systems. The study was released in November 2010. The McKinsey study shows that many school systems, not showered with extra money, actually did better than others such as the UK and the USA where standards have slipped. What does make schools better is strikingly similar across the 20 different education systems that McKinsey studied. Successful strategies depend very much on a system’s current stage of development but, unsurprisingly, educational interventions such as curriculum revision, teachers’ skills, and the assessment of learning figure largely. In this finding, the McKinsey study echoes other large studies such as the 2007 International Project to Frame the Transformation of Schools. This Project concluded that no single strategy or collection of strategies could be successful unless there is an alignment of change efforts and the focus is on the student. To paraphrase Riddell's question: “How can aid to [Indonesian basic education] be made more effective?” (p. 257), the answer, based on my experience in Indonesian basic education and from my own research and evaluation studies, is to work from the bottom-up (teachers, schools, Districts); empower and support local networks/communities of practice for dissemination of good practices in educational development; avoid one-shot or limited, decontextualised training programs of the kind now proposed by AusAID; and work on the 'community of practice' principle including whole school and whole District development. This requires capacity development; not only the capacity of individuals, but also of their organisations and the institutions of educational administration in Indonesia. And when we talk of consultants (highly paid or not) let us think mostly about the very talented Indonesian consultants, most of whom are actually practicing teachers, principals or academics with relevant front-line implementation experience. Perhaps 'facilitator' is a more appropriate term. I am emphatically not encouraging more jobs for expatriates. I need convincing that AusAID is not being driven by the ideologies of development embedded in the Paris Declaration and its related Jakarta Commitment. I also suspect the political need to spend increasing sums of money on aid is also a strong factor in what we now have. $500 million is not only a lot of money; it is also a suspiciously 'round' figure! That is, we have supply-driven development rather than demand-led (which does not mean the dominant, centralist demands of Jakarta-based management and political elites). I really wish I could believe Marc's views about disaster proof school construction, provision of facilities for the disabled, provision of toilets and the freeing up of resources for teacher training. For a view about construction issues, a reading of the Independent Completion Report of the Australia Indonesia Basic Education Program (available on AusAID's web site) presents worrying information about lack of earthquake resistance, poor site selection, limited provision of toilets, lack of availability of enough engineers to supervise and assist with construction, and contracting limitations. Frankly, I think it a credit to those contractors and their national and international staff involved in AIBEP that they achieved what they did in such a complex environment. The same can be said for the ICR team that has worked under incredible time constraints to produce their evaluative report. But even when schools have been well-built and corruption of processes eliminated what then? Is there a firm guarantee that the Indonesian Government will maintain these new facilities? No. Will we see, as I observed in 2008, schools we had rehabilitated under the ADB Decentralized Basic Education project, being rebuilt or in urgent need of refurbishment only 4 years later because of no or poor maintenance? This is not the kind of “sustainable” development AusAID says it is committed to. Finally, does AusAID have a guarantee that the Government will use its funds freed-up by Australian aid for financial support for teacher training as intended, or is the view of some Indonesian Government officials correct that the education budget provided by their Government will actually be reduced by an amount equivalent to that of Australian support? There is much to discuss and much to remain concerned about in the current AusAID proposals.

I enjoy reading public commentary on aid issues, including Robert Cannon's op-ed for The Australian that questions Australia's approach to supporting education in Indonesia, and Marc Purcell's persuasive rebuttal. Mr Cannon argues that 'the evidenced-based, educational perspective on schools and what happens in them is missing.' I read Robert Cannon's op-ed several times and agree with this statement - I could not find much evidence for many of Mr Cannon's assertions. If Indonesia scores poorly on Transparency International's perceptions of corruption, is this evidence that there is corruption in this particular education program? If Australia's domestic programs are implemented poorly, and this is debatable, is this evidence that international aid programs will be? If technical assistance is the solution, then where is the evidence that this will be more effective than the current strategy? If Mr Cannon would like to have an evidence-based debate on the effectiveness of Australia's support for education in Indonesia, then a good place to start would be to look at the experience of the existing project over the last four years. AusAID have, commendably, published the Independent Completion Report for the first phase of the project, which sheds some light on some of the concerns that Mr Cannon raises. http://www.ausaid.gov.au/publications/pdf/2010indoaibepicr.pdf The evaluation report does support Mr Cannon's concern that there is insufficient funding for school maintenance, with most Indonesian government funding going to teacher salaries. (Though the report is less sanguine about the effectiveness of technical assistance in helping the Indonesian government to remedy this - something for AusAID to think about for Phase 2) This evaluation report does not identify any specific cases of corruption, but it does commend the safeguards put in place to manage fiduciary risks in the project. The evaluation concludes that the project has been 'implemented effectively and efficiently, and is a successful program.' This included building more than 2,000 schools and creating over 330,000 school places. The project does include technical assistance, including on how the Indonesian government can fund education and maintenance, and the evaluation report comments are generally favourable on the value of this component of support. Yet on the issue of technical assistance, it is also worth noting the broader evidence on the efficacy of this instrument - for a good summary see Roger Riddell's 'Does Foreign Aid Really Work?' - which suggests that it is not the panacea that Mr Cannon suggests. We need more discussion on aid in Australia, and like Mr Cannon I agree that it should be grounded in good evidence!

Paul raises an interesting point, which touches on an important policy issue for PNG - which is “ what should the future role of government be in delivery of essential social services”. The government in PNG has traditionally been responsible for policy, regulation, standard setting, funding and service provision (although much of the rural health services have been, de facto, sub-contracted to churches to provide). In several countries, the funding, policy and regulatory roles have been separated from service provision and the provision of health care services is contracted out with external (private or public) institutions. The role of government changes from one of managing health care provision into one of purchasing health care services from different providers. The government retains certain powers, including, controlling market entry ( e.g. clinical authorisation licenses ); reviewing prices and setting minimum quality standards. The PNG government struggles to deliver services to its rural population. It already, in effect, contracts out services to churches, and in other areas, mining companies and plantations informally step into the breach, and provide services to populations in their immediate impact areas. There is already some policy dialogue on the issue of Public Private Partnerships. The potential for a much more pluralistic approach to health service provision is worth investigation. This could include churches, NGOs, mining and resource companies, and private health providers. However, it is not without risk – as cost escalation can be an unintended consequence if payment mechanisms and incentives are not carefully thought through. In addition government capacity to design, implement and monitor contracts would be needed.

Thank you for an insightful piece Peter. 'Developmental risks' indeed. More boldness and better analysis of inequality and insecurity for the poorest people in developing countries is required in Australia's aid program. This report was particularly important as the sector considers the role of ODE in the Aid Effectiveness Review. An interesting development that might take some of Peter's arguments forward is a new report from ODE, still in draft form, about whether AusAID should have a strategic framework for dealing with civil society. The draft report is entitled 'Working with the State is Not Enough', and in many respects it is the missing piece in the puzzle - http://www.ode.ausaid.gov.au/publications/index.html#progress. Taking civil society seriously in the aid program is overdue. How do weak governments improve? By technical assistance from other governments, or by demands for better governance by empowered citizens? Surely by both, but without pressure from the domestic constituency, reforms will falter. The latest thinking from AusAID on governance recognises this ('Power to the People' - http://www.ausaid.gov.au/publications/pubout.cfm?ID=9742_5949_4436_8120_49&Type=PubKAG), but still fails to think of ways to promote bottom-up governance by engaging civil society. Presumably because such a focus is messy and human, but that is the game we are in.

This is a very interesting and timely idea. I wonder if and how we could translate it in a humanitarian context? Perhaps not at the onset but later in the emergency cycle? The sooner the better to avoid the wait and see attitude of beneficiaries, what they call "attentisme" in Democratic Republic of Congo.

Good to see the responses to my posting. I am just back from a week in Dili, Timor-Leste, conducting interviews for a workshop on a related issue - how to provide demand-focused technical training. Informal feedback from colleagues I spoke with is that big changes are still needed there in how aid is delivered. The use of a hybrid system of COD in a post-conflict setting will require careful analysis and innovation. Both parties will also need to negotiate changes if it is not working as expected. Better and more independent evaluation tools will be needed to provide feedback. However, the longer-term benefit must always be kept in mind - the full acceptance by the recipient government of the funding and implementation of the intervention. This is a key test of aid effectiveness. The change in the mode of aid delivery that COD encompasses is akin to the fundamental reforms to labour exchange services in Australia over a decade ago. Australia led the world in these reforms, reviewed favourably by the OECD in its 2001 report Innovations in Labour Market Policies: the Australian Way. The switch from the direct delivery of services to job seekers through the Commonwealth Employment Service (CES) to outsourcing these services to a network of for-profit and non-profit organisations was a huge change. The new focus was on payment for outcomes - jobseekers in jobs, training or education more quickly. Switching from a centralised funder-driven model of service delivery to a ‘steering not rowing’ focus was difficult. It took a number of attempts to finetune the new arms-length system. Above all, it required within the funding department a change of mindset from service deliverer to service designer based on high-level conceptual skills It also took alot of ‘learning by doing’ about what worked and what did not to design tenders that worked more effectively over time. These reforms were an outgrowth of a period of experimentation in public policy in Australia. Is the Australian Public Service up to the challenge in the current era?