Comments

Christopher, thanks for your insider’s assessment of the state of the ‘reporting culture’ within AusAID. You note that the Office of Development Effectiveness (ODE) has poor quality data and systems to work with. An important issue is the timing of the release of the country annual program performance reports (APPR). I have recently travelled to a number of countries in the Pacific to present a workshop. I sought information from the AusAID website for each country, only to find APP reports were up to three years out-of-date. Tonga’s report for the financial year 2007-2008, was released in November 2008, but the reports for Samoa and Nauru was for 2008-09, released in June & July 2009 respectively. For Kiribati, the APPR is for the year 2008, released in November 2009. For Solomon Islands, no APPR was on the website, only an undated document on the two countries Partnership for Development. No report was available for Fiji although reports on three specific program dated 2011 were available. Information has to be timely to be effective. It may be partly to do with the uneven capacity of its staff in its country posts, as you highlight (‘most officers are still lacking the skills to really make the assessment work’). It may be to do with the extended internal processes AusAID has vetting these reports before handing them to an external consultant to validate. You also note that AusAID’s self-assessment system is ‘riddled with faults and limitations’. ODE needs to provide this feedback, and report publicly on the extent of improvements in staff capacity and system performance. AusAID and ODE need to appreciate that there is an audience out there who want more than superficial assessments of aid activities. We want up-to-date information on results achieved. We also are looking for a good situation analysis of each country, based on honest assessments of limitations of program design and delivery, host government shortcomings and the lessons learned. Richard Curtain

Hello Robert, I see your argument has shifted from regulation to trade. Trade can, indeed, help with development. However, it seems exceptionally unlikely to me that the linkages created by the small number of visits promoted under the scheme will have an influence on poverty that is significant in any way when compared do that which can be done through market forces alone in the trade arena. With regards to NZ growth rates. The reforms commenced in the mid 1980s. Economic growth stagnated for about 10 years followed by a brief period of improvement (mid-late 1990s), followed by further recession. Recovery really kicked off in the new millennium under the rule of a government that was decried as socialist by its opponents. Even then growth rates achieved have been less than those of many Asian economies. Including ones that score much lower on ease of business rankings. Moreover, our economy while performing alright hasn't done any better than it did in the 1960s (for example) under a much more regulated regime. This isn't to say that regulation doesn't matter - but clearly it isn't the only thing that does. Otherwise the New Zealand story would be quite different. Cheers Terence

The term 'blue economy' is not only used to describe a 'marine version of the green economy': Gunter Pauli, (2010) The Blue Economy-10 Years, 100 Innovations, 100 Million Jobs http://www.clubofrome.org/?p=1625 For those who are time challenged or suffering from bibliophobia, 'The simple show' explains Pauli's 'blue economy': http://www.youtube.com/watch?v=1af08PSlaIs

Hello Robert, Thank you for your comment and apologies in advance for my limited reply I don't have good internet access at present. You state:"It is not clear what benefit you are asserting NZ businesses get from this program." The anticipated benefit is clear enough. It is in the quote I used in my post and on the webpage I linked to. To reiterate, the desired benefit is "the establishment of lasting and progressive trade and business relationships". This, if it works, will benefit New Zealand business. It may even benefit New Zealanders more broadly. Perhaps, even, it will be one of those relatively rare moments where the government can use its material resources in a planned manner to generate economic outcomes that are better than those produced by markets alone. Perhaps. But these are benefits first and foremost for New Zealand - they aren't aid. As for the potential for visiting business people to learn about NZ's light touch regulations and then take this information back to their own countries, subsequently enhancing economic growth which in turn helps the poor. This is unlikely for the following reasons. 1. It does not seem clear that the visiting business people will be taught anything about the NZ regulatory environment. They are being brought over to visit businesses not government departments. 2. It is unlikely that the views of a few business people could really bring about regulatory reform - political economy is much more important here than what a few people might learn during a 10 day jaunt. 3. It is unclear that New Zealand really has a regime worth emulating. In the immediate aftermath of our adoption of it, NZ's economy stagnated for more than 10 years. Since then it's performance has been average at best (contrast this with most Asian economies, which have boomed). Cheers Terence

I think the recent debate directed at the limitations of Australian Aid effectiveness data is helpful and important. However, I also think the heavy criticism hoisted upon ODE and AusAID misses the point a little on the development effectiveness agenda within the organsation. As one of the few professional evaluators to have worked in AusAID, I think it is important to acknowledge the limitations of what ODE has to work with. The reporting culture in AusAID is traditionally and continues to be weak. Datasets are poor, good baselines are rare and the political imperative of reporting in sound bites is common. This means strong, rigorous and thorough impact reporting is virtually impossible for most programs due to the poor systems and structures built into the aid delivery model. ODE and the Operations Division of AusAID has begun to tackle this by driving a cultural change built around the 'self-reporting' performance system of the agency. This system is based largely upon a World Bank model that looks to regular reflection on how programs are tracking against standard DAC measures (relevance, effectiveness, efficiency, M&E, sustainability and gender). While this reporting is completed by AusAID officers, there is a moderation process carried out by the Operations Division and then a realism review carried out by an independent consultant through ODE. The quality and robustness of these reports varies enormously and they range from the abysmal to the impressive. What is important when we critically dissect the process is to remember that it is still new, most officers are still lacking the skills to really make the assessment work, and we are going from virtually no discussion of accountability and performance to a dialogue now informed by a call for better skills and greater rigour. The system is riddled with faults and limitations, but there are a range of committed individuals working hard to make it work. To single out one good example, the Inodnesia program in AusAID has instilled an impressive M&E regime in the last three years. Working on many fronts, they have built in protocols for evaluation report release, they insist on management responses, they have a set of M&E standards, they have a professional development program supporting staff in the Jakarta office and they have a Minister Counsellor committed to cultural change. This is impressive for a program that was publishing less than 5% of its evaluations four years ago. Having recently joined the World Bank, I continue to be stunned by the poor datasets that exist in the development aid world. I argue that part of this is driven by an obsession with data, indicators and information that does not mean anything, does not help anyone and is not utilised in interesting ways. We should remember that part of the performance mandate is about what we learn and how this is used. Yes, ODE have missed some opportunities to more strongly impose themselves on the effectiveness agenda, but I believe the analysis they conduct on the APPRs and on the annual quality at implementation (QAI) reports is actually very good. The sampling is sensible, the consultant has worked on the review for each of the four years the system has been in place, and there have been identified changes in the quality of the regular reports. Yes, the process is qualitative and yes the tracking data is still limited, but the report is a review of the existing system and at this stage I don't think it warrants constructing an in-depth range of tracking information. To do this would mean further challenges to an already stretched and inexperienced staff and the danger that they freeze like deer in headlights. I believe there are enormous opportunities for stretching the work ODE does and inviting them to be involved in an informed discussion on approaches and methodologies would be a good place to start. Believing that people, organisations and stakeholders will always make good use of data and the transparency charter is naive as the regular press articles by Steve Lewis will attest. Having a robust discussion of the best ways to make aid programs more 'effective' is a much bigger challenge than simply hoping that ODE can lift its game in its very limited mandate. Now that I am outside the public service I look forward to being more involved in this discussion. Christopher Nelson is a M&E Specialist with the World Bank based in Sydney. He previously worked in the Performance Effectiveness and Policy Division at AusAID.

I agree wholeheartedly with Stephen's first three proposals - reduced fragmentation, improved evaluation and greater aid transparency. Hopefully in each of these areas significant improvements are currently being implemented: by cutting sectoral spread in each country, through a simpler, easier to understand and outcome-focused performance framework and through implementation of the Aid Transparency Charter. Yet as Stephen points out there is much more to do in each of these areas. Aid transparency for me particularly stands out. When at least the ten biggest country programs have updated web pages with the promised new level of detail and when the IATI activity data is complete and just one quarter behind then I think we can say there has been some real movement in terms of transparency. I'll pass on Stephen's fourth and fifth priorities and instead suggest two others: ensuring accountability to affected communities and getting serious about the aid program objective of "saving lives". AusAID is working to improve accountability to Australian civil society, the Australian Cabinet and to partner developing countries - we need a similar effort to build accountability to the communities touched by our aid - in the interests of both justice and effectiveness. The first strategic goal of the aid program is "saving lives". The intrinsic importance of action in this area and its proven effectiveness demands that it be given a greater share of aid funding. Health should join education as the second flagship funding area of the program and all country strategies should ensure that health assistance is adequately covered by the donor pool before considering support for any other sector. We wouldn't neglect health in budgeting for the Australian population and we wouldn't in planning emergency aid, yet this is what is happening with Australia's long term development assistance. Aid for health needs to rise from around 14% of the program to around 20% (25% when including funding for sanitation and water) if we are to meet the commitments we have made to cut maternal and child deaths, to provide universal treatment for HIV/AIDS, TB and malaria and to provide family planning.

AUSAid is to be commended for developing and releasing this paper. John Eyers is correct in his view that the Paper is overwhelmingly positive where experience may suggest otherwise. Perhaps we are able to learn as much from our failures in this complex and difficult activity as from our successes. I do note with interest that there is some emphasis through the Paper on the importance of building state structures and working through host Government systems. Both are critical in sustaining any intervention. Also of importance is integrating a focus on public finance management including revenue and expenditure management and procurement. Why procurement? Recent experience in fragile states suggests that public procurement can account for up to 60% to 70% of all government expenditure. In Swaziland, 65.9% of Government of Swaziland expenditure (both recurrent and project) was spent through a procurement process; in Timor-Leste, the percentage was 70.19%; and in Sierra Leone and Uganda, 70%. Procurement is a critical consideration in achieving the effective and efficient delivery of public services. It involves the management of a large amount of public funds and is the largest single cause behind allegations of corruption and government inefficiency. The combined effect of high value transactions, excessive discretionary powers afforded to public officials and dependency of private firms on government contracts to survive economically provide fertile grounds for procurement corruption in fragile states. Government contracts also provide an additional source of income for both politicians and bureaucrats and a substantial source of funding for political parties, election campaigns or individual politicians. Weak procurement policies and practices constrain fragile state development and waste scarce public funds. Efficient and effective procurement is an important contributing factor to the achievement of superior development effectiveness results for fragile states. As procurement is one of the most challenging sectors to reform because it often meets major resistance from vested interests within society, understanding the political economy aspects of procurement is critical to address the underlying factors that may impede the effective implementation of reforms. Procurement reform is a highly technical and complex process that requires major changes involving a wide variety of actors with multiple, often conflicting and changing political incentives that can complement or compete against each other. At the operational level, reforms involve major procedural changes that can profoundly alter the work of procurement officers and generate massive resistance from public officials who benefit from the old ways of doing business. Any improvement in the public procurement system can have a direct and beneficial effect on the overall economic situation of a fragile state. E. John Blunt is an Institutional and Public Procurement Expert with extensive experience in leading public procurement reforms in a variety of international development environments. He has worked in fragile states in the Pacific, Asia and Africa. He is currently on assignment with the Southern African Development Community Secretariat in Botswana.

Sri Mulyandi is favored for the World Bank's President?? It is not going to happen till th end of the world. The substantial reason is that the WORLD BANK BELONGS TO THE USA. American is the bigest shareholder of the bank. the Word Bank and the IFM are the tools of the industrialized contries to control or to reconlonize the poor developing countries economically. And this has been goi9ng on for most than60 years now. According to the long standing traditions so far, as a result, the preident of the World Bank is always from the USA and the President of IMFis from Europe. A debtor country or an economiclly colonized country like Indonesia is elected president of the the World Bank? It is weird and is not going to happen.

I agree that SWAps have offered the prospect of health aid being more efficiently provided through processes that were aligned and harmonised both amongst donors, and with Government systems; offered the prospect of a more holistic approach to financing that encompassed and linked capital and recurrent budgets; promised to reduce the burden on government officials of managing the often competing and onerous demands of donors; and most importantly, sought to increase the effectiveness of health aid in improving health services and outcomes. The theory of change underpinning SWAps is premised on the aid effectiveness principle that robust, legitimate institutions are central to the sustainable development of effective health systems, and that these institutions will only develop through deeply-embedded local, political and cultural contestation and processes. However, many SWAps have failed to properly strengthen health institutions. Some Health SWAps in the Pacific, Asia and Africa have not included sufficient focus on planning, budgetary, management, procurement and supply issues; some have only partially focused attention on these issues rather focusing on technical health issues; and some have divided focus among a number of Donors leading to systems that are sometimes disjointed. Is it any wonder that health practitioners sometimes, following a significant intervention, still do not have the required pharmaceuticals, medical consumables, medical equipment and the other ‘hundred-and-one’ items necessary for the health system to work! Remember that health planning, budgeting, management, procurement and supply do not operate in isolation, but rather operate within ‘whole-of-government’ systems. Do Health SWAps always consider the ‘whole-of-government’ context and indeed the necessity of designing interventions to strengthen these systems before focusing on strengthening the health system? The recent intervention to strengthen the health system of the Government of Swaziland is a good example of the requirement to design interventions at government and ministry level to compliment required health interventions. Perhaps a more strategic view is required when planning, designing and implementing Health SWAps is required. E. John Blunt is an Institutional and Public Procurement Expert with extensive experience in leading public procurement reforms in a variety of international development environments. He has worked in the Health environment in the Pacific, Asia and Africa. He is currently on assignment with the Southern African Development Community Secretariat in Botswana.

Thanks for this interesting blog and discussion paper. A resurgence in oil prices will probably wreak havoc again on many of the small island economies, sad to say. The 2008 fuel crisis was a real wake up call for the Marshalls, as you mention. The silver lining on that cloud was that it forced leaders to focus more attention on reforms. The major electric utility was already developing a reform plan when the crisis hit, forcing it to fast-track the process (it was aleady insolvent prior to the crisis!). The crisis also resulted in RMI developing its first comprehensive national energy policy and action plan, which emphasizes energy efficiency, renewables, and other initiatives. Australia, ADB, and other partners responded with good support. Since 2008 we've seen a wide range of energy projects and initiatives get moving. But.... much more could have been done to strengthen the country's resilience, both in terms of its energy security and its fiscal position. As they say, history repeats itself... because we don't always learn our lesson the first time around!

The issue of publishing evaluation is who goes first. Dinuk seems to suggest the NGO should start and publish all of their evaluations (the definition of 'all' becomes interesting here), but many do publish program overviews ( more than only CARE) which are meta studies of all their evaluations and key lessons learnt. Partner program evaluations are also published on the web, for example the We Can End Violence against Women program in South Asia (supported by multiple donors) AusAID does not publish all of it evaluations (it is fairly selective), nor does the World Bank. In line with Chris and Garth comments most useful evaluation are those to the partner community, country, or NGO, depending on who the partner is. I have done a Third Party Assessment for the partner government of a large decade long World Bank program. I suspect neither the partner government of the World Bank would like it published, but it was useful to the partner government as it gave a clear steer on how future loans may be directed in that sector, and how it might direct is own investment. Likewise evaluations I undertake for NGOs are largely directed to the partner as they are the ones that it is most useful to guide them in the negotiations with donor NGOs. I am not sure AusAID encourages recipient government to review the AusAID programs but that would be a much greater step forward in transparency than mere publication. Most published evaluations focus too much on outputs than outcomes, and the authors are very mindful of who is paying the bills. Perhaps greater attention could be to academic studies of development effectiveness at local level which look at these issues particularly from the recipient point of view, as these programs invariably have multiple donors. For example I have looked as effectiveness in a sector across 15 small and medium NGO in India. Published studies like these may yield more that published evaluations with their focus on short term results.

The ‘Capability Trap’ papers highlight the importance of mapping the local organisation context as a precursor to selecting and lifting administrative capability. Organisations come with their own histories and operate in specific contexts. I have found that two factors impact on the goals, methodologies and markers negotiated to define measureable success. I currently work in Timor-Leste and a feature of donor literature is a tendency to define the development of civil service expertise as a post-independence phenomenon. The scorched earth response, which included mass murder, has contributed to the narrative of a ‘phoenix’ nation, rising new and untested from adversity. The Indonesian civil service decamped, taking with them managerial and technical expertise, leaving the United Nations Transitional Authority in East Timor (UNTAET), with the responsibility for establishing and ‘capacity building’ a cadre of civil servants across the entire spectrum of government with an unskilled workforce. The work continues today with a low capacity workforce...The human resources reality is however, more complex. At one stage I conducted interviews with all staff in three Directorates (63 people) and was surprised to find that 48% of people had been previously employed in similar jobs in the Indonesian civil service for an average of 6.3 years. While few had held managerial or technical roles, one could not characterise staff as being novices to administrative work. This brings into focus Pritchett and Weijer’s point about measuring system performance and developing staff ‘buy in’ to any change process by acknowledging and valuing prior experience. As stated in the article, ‘training’ should take advantage of existing staff skills and knowledge. The context of their former work also needs to be taken into account because it shapes staff perceptions regarding organisational purpose and indicators of success. In the case of Timor-Leste, Indonesian, donor and current staff capacity building/professional development have relied on significant subsidies which unfortunately foster ‘isomorphic mimicry’. I would suggest that the focus of external assistance has to be the work of the organisation and the people who perform that work. If, as advocated by Pritchett et al., that donor agencies move away from concentrating on significant organisational targets towards individuals and their ability to perform their key work tasks, then these tasks need to be clearly defined and measured. Richard Curtain has commented, more than once, that AusAID appears to be averse to actually defining and measuring capacity development in practical terms. There is a tendency for some capacity building approaches, such as the Staged Capacity Building Model (AusAID 2006) and ‘enabling environment’, to value high level outcomes or products over a systematic approach which takes into account the contribution of all members, at all levels of a department to achieving agency objectives. As the articles note, it is possible to have an apparently satisfactory outcome, but one which is so heavily reliant on outside funding, expertise and systems support that it is not only unsustainable, but undermines the ability of recipient governments to develop their own, more modest but effective systems.