Missing, one kangaroo with crossbones: ODE evaluates DFAT’s support for development research

Written by Robin Davies

imageIn her mid-2014 ‘new aid paradigm’ speech, and elsewhere, foreign minister Julie Bishop placed innovation at the centre of the aid program, saying it would drive the way Australia delivers aid. A new ‘innovation hub’ within the Department of Foreign Affairs and Trade (DFAT) would ‘trial and test development innovations’ and ‘reach out to the best and brightest’. Its emblem, she joked, might be a kangaroo with crossbones (a Steve Jobs reference).

Since innovation is fuelled by research, development research basked in the same limelight. Australia, Bishop said, would ‘do much more in [the] area of innovative health and medical research’ and funding for same would reach its highest-ever level in 2014-15. The ‘remarkable research capabilities’ of the Australian Centre for International Agricultural Research (ACIAR) would be used ‘to better effect’ and, it was implied, shielded from budget cuts.

Against that backdrop, an evaluation of the Australian aid program’s support for development-related research should have been a timely thing. But then, in December 2014, the government decided to prune the aid program so brutally from mid-2015 that DFAT’s aid managers are unlikely to be in the market for anything new, let alone innovative and discretionary, in the foreseeable future. Obviously, in light of this decision, most or all of what Bishop said about research and innovation last year must now be considered null and void.

In any case, DFAT’s Office of Development Effectiveness (ODE) has now released its ‘strategic evaluation’ of research use in the Australian aid program, ‘Research for better aid: an evaluation of DFAT’s investments’ [pdf]. If no longer timely, is this evaluation at least a useful backward look?

It depends on what you are looking for. If you want an evaluation of the impact of DFAT’s investments in development-related research, this is certainly not it. Nor is it an evaluation of the appropriateness of the development research strategy [pdf] published in mid-2012. That strategy is mentioned but not evaluated, on the basis that it was ‘not carried over into the integrated DFAT’ (p.12). Nor, despite its subtitle, is this evaluation really about the impact of DFAT’s investments in aid-related research.

What, then, have we got? The evaluation ostensibly set out to ‘investigate the degree to which research is being used to support aid effectiveness’. Hence ‘research for better aid’ rather than ‘development research’. As if this limitation did not sufficiently narrow the domain of enquiry, the evaluation further limited its scope—‘to be manageable within the resources and time available’ (p.58)—to DFAT’s ‘uptake’ of DFAT-funded research outputs. The effect of this was to rule out of consideration any form of development research that aims to yield unmediated benefits to developing country governments, other third parties or the world as a whole, such as support for the development and evaluation of public policies, or for technological advancement in medicine [pdf], agriculture and environmental management.

Also ruled out was ACIAR’s entire core budget which, at $96 million in 2014-15, dwarfs all other research funding allocations. True, ACIAR was subject to an external review in 2013 but it was a friendly one that did not consider trade-offs between agricultural research and other research priorities. Nor did that review draw comparisons between the way in which ACIAR commissions and uses research and the way in which DFAT does.

Even the tiny canvas that remained after the above limitations were imposed might have accommodated some interesting findings if the brush had been less broad. But what we have is essentially a dry, process evaluation of the manner in which research funding is managed within DFAT, based largely on a staff survey. A diligent plod through the evaluation report yields almost no sense of the nature, let alone impact, of the specific research programs funded, and therefore no sense of what it would mean for their outputs to be taken up. For example, the largest of DFAT’s top 50 research investments in 2012-13 was a $13.2 million Africa food security initiative implemented by the Commonwealth Scientific and Industrial Research Organisation (p.90)—yet nothing is said of it. The few specific investments mentioned, such as support for a ‘drivers of change’ study in Vanuatu some years ago, are despatched with one-line judgements, and still no sense of what uptake looked like or led to.

The evaluation’s overall finding is that ‘DFAT’s development research investment is largely appropriate’ (p.1), which turns out to mean that Australia is thought to spend about as much as the average OECD donor country on research, and allocates research funding broadly in line with the thematic priorities of the aid program. However, most everything else that the report finds is rather negative—that DFAT lacks a strategic framework for its investments in development research, has no central institutional capacity to guide research or ensure its outputs are well used, resorts too often to known partners rather than employing competitive resource allocation, under-utilises developing country researchers, and perhaps even—this is the view of ‘several’ survey respondents—has an anti-intellectual bias (p.37). The evaluation also shrugs off its restrictive terms of reference for a moment to observe that DFAT senior managers have little appetite for research that is primarily policy-oriented or contributes to the production of global public goods (p.2).

The evaluation rightly recommends that DFAT put in place a clear policy framework to guide its investments in development research, and rightly implies that it needs to be clearer and better implemented than the one adopted in 2012. DFAT, in the management response included in the report, claims to have accepted this recommendation, except for the part of it which would require a maintenance of funding at about current levels as a proportion of total aid. However, a careful reading of DFAT’s response does not reveal any actual commitment to the development and adoption of a new, over-arching development research strategy. Support for development research looks set to remain fragmented, largely confined to country programs and allocated ‘in response to specific program needs’ (p.6). Responses to lesser recommendations follow the same pattern.

DFAT’s ‘thanks for that’ response to the evaluation’s various recommendations might be considered unsurprising given the deep-freezing or axing, unremarked by the evaluation, of research funding vehicles such as the Australian Development Research Awards and the four knowledge hubs for health, the likely downsizing of country-specific initiatives such as the Indonesia Knowledge Sector Initiative, the seeming withering of what was the AusAID-CSIRO Research Alliance [pdf], and uncertainty about ongoing funding for health-sector product development partnerships. Nevertheless, the aid program isn’t at zero yet. Research and innovation are still avowed priorities of the government. Some kind of a basis needs to be provided for allocating, and indeed reducing, funds in this area.

The ODE evaluation was not constructed to answer any of the most interesting questions about Australia’s funding for development research, so cannot really be castigated for failing to do so. A more useful evaluation might have looked more deeply at a number of such questions, which are mostly questions of balance—the balance between research for the development of programs, policies and products, or between thematic priorities (health, agriculture, education, and so on), or between the local and the global, or between supply and demand (on the one hand, funding research providers in the hope that they will generate useful knowledge and, on the other hand, creating incentives for the solution of carefully defined development problems). The question of how far DFAT staff are willing and able to use the outputs of the research that they finance is, by comparison, a third-order one. DFAT is not Procter & Gamble.

Robin Davies is the Associate Director of the Development Policy Centre.

Robin Davies

Robin Davies is the Associate Director of the Development Policy Centre and an Honorary Professor at the Crawford School of Public Policy. He heads our program of research into global development policy. He was previously a member of AusAID’s senior executive service for a decade, both in Australia and overseas. Most recently he headed AusAID’s international programs and partnerships division. His policy and research interests include multilateral cooperation for development, development and climate change financing models and public-private partnerships for development. He is currently based in Geneva.

1 Comment

  • Thank you very much Robin. This is an excellent overview of the report and the context. I struggled through the report and couldn’t really make heads or tails of it. The context you provide is therefore invaluable.

    I remember vividly the orientation workshop that DFAT held for those of us who received ADRA scheme funding. There was a panel of senior DFAT (then AusAID) managers and one of them actually said something to the effect of “In my twenty years here, I cannot think of one occasion when evidence influenced my policy-making.” He obviously didn’t hear the words before he said them but that said a huge amount about the role of evidence/research in the organisation.

Leave a Comment