Are scholarships good aid?

27 April 2011

There needs to be a genuine discussion of scholarships as an aid instrument and how to improve their effectiveness. Markus Mannheim’s story on Development dollars are too precious to waste moves the aid debate forwards from fraud to development effectiveness, arguing that unless scholarships can be shown to reduce poverty they should not be counted as aid. This is an interesting discussion to have, but his analysis of scholarships is incomplete and like other recent stories tends to sensationalise the issues.

As argued before, the public deserves well researched analysis of the aid program and unfortunately we are short changed in the scholarships story. Instead we get yet another beat up of the aid program: we read about  ministers ‘doling the scholarships out’, developing country ‘elites’ receiving ‘lucrative awards’ etc.

It is good to see that more journalists are accessing aid reports through the Freedom of Information Act. The ‘confidential’ and ‘internal’ report on the effectiveness of scholarships (2008 and 2009) that forms the basis Mannheim’s story is now available on the AusAID website following an FoI request. (And according to the AusAID’s draft Agency Information Plan, we can expect more of this kind of information to be routinely shared–a very welcome move.)

A story about the effectiveness of scholarships would be more persuasive though if it made reference to AusAID’s most recent assessment of scholarships programs contained in this 2010 update on scholarships (which is publicly available on the AusAID website.)

It would also be more persuasive if it were based on the findings of a more recent audit or aid review. The story preempts an ongoing audit of Australian support for tertiary education that is due to be tabled in the Autumn sitting of Parliament, and the Independent Aid Review, chaired by Sandy Hollway, that is due to report back this week on how to improve the effectiveness of Australian aid, presumably including scholarships.

We don’t yet know the findings of the ANAO audit report or the Aid Review, but in the meantime, AusAID’s own 2010 update report on scholarships is worth reading to get a better understanding of the objectives of the scholarships program, patterns and trends in funding, achievements, and the challenge of improving monitoring and evaluation.

Scholarships expenditure According to AusAID’s 2010 update:

  • Funding for scholarships increased from $145.6m in 2008-09 to $153.7m in 2009-10 (see table 1), increasing the number of awards from 1874 to 2082. Scholarships were provided to 48 developing countries in 2009 and, 56 countries in 2010.
  • The last two years has seen an expansion of development scholarships into new geographic areas. The scholarships engagement for Africa has expanded, and long-term scholarships were offered to the Caribbean, Latin America and Burma for the first time in 2010. Scholarships to the Caribbean and Latin America are an important feature of the Australian aid program in these areas.
  • The Government’s 2010-11 Budget statement announced a doubling of scholarship to around 3,800 places annually in 2014.  This includes an increase in scholarships for Africa from 260 in 2010 to 1,000 per year by 2013.  At any given time there are between 2,500 and 3,500 people studying in Australia on development scholarships.
  • By 2014, it is estimated that more than 6,000 people from developing countries will be studying in Australia on scholarships funded through the aid program.  Based on current projections, it is expected that 16,000 scholarships will be provided over the next five financial years, at a total cost of around $1.7 billion.

Monitoring and evaluating the impact of scholarships is not easy. The Mannheim story refers to a 2008 and 2009 study that is available on AusAID’s website. As mentioned above we don’t yet know the findings of the audit and Aid Review, but M&E seems to be an area where AusAID has made good progress. The 1999 audit, the 2007 ARDE, 2008-09 study were all critical of AusAID’s systems for tracking the effectiveness of scholarships. Since then, notable progress has been made in strengthening systems. According to AusAID’s 2010 update:

  • A total of five tracer studies, three independent completion reports and three joint reviews (with New Zealand) of scholarships programs were completed over 2009 and 2010. These reviews provide valuable information on scholarships and have led to redesigned scholarships engagements and closer relationships with partner governments and other donors.
  • A mid term review of the Australia Leadership Awards was completed in September [2010] and a review and redesign of the leadership development program will be completed in early 2011.  Lessons from these reviews will inform further work on strengthening the leadership development opportunities presented by AusAID’s scholarships engagements.  AusAID is also trialling a new tool – Sensemaker – to assess scholarships impact.  Results of the trial will be available in March 2011.

It seems then that there is rich array of evidence and analysis for the Aid Review to draw on to assess the effectiveness of the Australia’s scholarships program, including looking at the geographical targeting and need for scholarships, their cost effectiveness and ongoing monitoring and evaluation.

We need a genuine discussion on the future of the aid program. Markus Mannheim has moved the debate forward from fraud to development effectiveness, but not delivered a balanced analysis of the scholarship program. For now I will reserve judgement and look forward to reading a more informed analysis in the ANAO and Aid Review reports.

Matt Morris is the Deputy Director of the Development Policy Centre and a Research Fellow at ANU’s Crawford School.

Author/s

Matthew Morris

Matthew Morris helped to establish the Development Policy Centre and served as the Centre’s first Deputy Director. Matt is a development economist with 25 years’ experience. He is currently an independent consultant.

Comments

  1. Thanks for the post and all the comments. It’s rather hard finding info about this topic.
    I will be reading them all since I have been thinking about this for a long time.
    I am from Thailand which used to be a developing country a while back. I have never held any aid/development scholarship since I have an ‘average’ profile and I am from a middle class of the country. I did my masters in environmental management at UQ graduated in 2006 on my family support and part-time work while there. I do not think that these no-string-attached scholarships fro developing countries can really have an impact. I know some of the recipients of several of such scholarships. Most of them do not contribute directly to the development of their hole countries, some of my friends after finished their PhDs have gone on to work abroad (even though a different country than the one giving scholarships) for private sector doing things that will only make the poor poorer and the rich richer for example researching for new drugs for a for-profit pharmaceutical.
    The main concern is that msot of these scholarship recipients only come from the ‘elite’ group of developing countries. They are the only people who can afford to study English to the level that fulfill the requirement of the scholarship let alone the higher education which is normally not free and available in such countries.
    So all in all, I would only like to conclude that this type of scholarship only serves to protect and extend the elite class in developing countries so that they can in turn benefit the giving countries when they are in high and powerful positions!!!

    Reply Comment
  2. From my perspective scholarships should be considered primarily as aid. No doubt there are ample diplomatic benefits. Just look at how the Colombo Plan led to good, long-standing relationships between Australia and the countries that the Colombo Plan students came from. Or, more recently, look at how the training of Vietnamese and Cambodian scholarship holders has led to very good relationships between now-senior members of government and research institutions in those countries with the Australian government and research institutions. No doubt too there are cases where scholarships are handed out by Australian officials and developing country officials not on the basis of merit. These instances are to be deplored.

    But scholarships should be principally about aid that is intended to lead to better living standards in those countries receiving the scholarships. However, I would like to suggest that we should think of the provision of scholarships somewhat differently than the way in which normal evaluation processes are undertaken. I think of scholarships as similar to the funding of research. Many research projects are funded but only a few are successful. However, the payoff to the few successful ones are usually so large that their benefits far outweigh the cost of all others. Take the field of economics. If we can train one indiviual in economics who can be instrumental in having needed economic reforms implemented, the result can be increases in the economic growth rate of the country that last indefinitely. A one percent permanent increase in the growth rate of a Vietnam would more than cover all of Australia’s foreign aid.

    I know of cases where such benefits have flowed from the high-level training of scholarship holders. This view of the scholarship program implies that if we are to carry out a realistic appraisal of the program, a much more individualistic assessment of the benefits has to be undertaken. Of course, this would be very difficult. But the counting of how many students return home from training, or other such measurements, is really not providing any kind of accurate account and leads us to worry about the wrong kinds of issues.

    Reply Comment
  3. Another method of providing education and enhancing skills for people from developing countries is through in-country education/training/support. As an adjunct to out-of-country scholarships, in-country approaches can offer culturally appropriate, accessible, affordable, relevant and grounded education & training.
    Our recent experiences of providing in-country training & support in the Pacific, in the field of operational research in health, have confirmed to me the value of this approach.
    We have found people are keen to learn, keen to develop as researchers, and eager to change local health systems as a result of their research.
    The local people involved would be very unlikely to ever be able to access the current scholarships or education abroad.
    Increasing scholarships, investment and aid that supports an in-country approach needs to be further considered.

    Reply Comment
  4. My own research study on cultural identity and the AusAID Scholarship program has found that key drivers for the scholarship program tend to be overly related to Australian economic and political interest. One could argue that this is indeed an overarching and explicit aim of the broader aid program, however it is also reasonable to expect an outcome that shows where the program supports poverty alleviation, promotes capacity and international linkages. This is currently unclear and not evidence-based.

    Given the spirit of improving development effectiveness and promoting transparency, it is unacceptable that the Australian public is still waiting for an external assessment of the program, in particular to understand if indeed it has met some of its key stated objectives. The assumption that the benefits of the scholarship program will ‘trickle-down’ to alleviate poverty is clearly unsubstantiated and there is cause to argue that scholarship aid may be of greater benefit to Australia, particularly its higher education industry which requires a critical mass of students to support research and teaching in Australian universities.

    For some reason, the Scholarship program has escaped the rigour of a full assessment which is strange given the international emphasis on development effectiveness and amount of resourcing spent on evaluating other Australian aid programs. Overall Australian Scholarship Award impact studies or reports on development effectiveness have been seriously lacking, with the last audit of the AusAID scholarship program conducted way back in 1999 by the Australian National Audit Organisation which then noted that performance indicators to measure the outcomes of the program were lacking, in terms of measuring how students contribute to their country’s development on return.

    The review of the aid program in its entirety and a focus on the scholarship program is certainly long overdue. In spite of the scale and longevity of the Australian scholarship program, there has been an overall lack of accountability to the Australian public in terms of understanding the program’s effectiveness. This constitutes a major, unresolved public policy issue that requires addressing ‘sooner’ rather than ‘later’. It is perhaps unfortunate that requests for information about the program have not been met in the absence of the Freedom of Information application and I too have experienced blockages to accessing broad documentation about the program as well as access to publicly held documents stored in the AusAID library, which for some reason is no longer publicly accessed.

    As a development worker and now researcher, I have personally encountered many impressive former scholarship awardees who I have worked alongside but I am also aware of program challenges. Not the least of these relates to the re-entry process, which I understand is being closely looked at by AusAID, but with the information closely guarded within the bureaucracy. Given that aid programs are inherently complex and there are many lessons to learn, with broad scope for feedback from a variety of sources, why not open the possibilities for improving the scholarship program, starting with bureaucratic will to share information and promote public transparency, in the interests of good governance and effective aid programming.

    Reply Comment
  5. I am currently working in development in Nepal with many people who have been the recipients of developed country scholarship programs. The almost universal experience of my fellow development professionals of working with these recipients is the same: they have sufficient technical skills but their work culture and lack of experience working in efficient organisations prevents them from reaching their potential. They have never worked in workplaces where administrative practices are streamlined, financial transparency is ensured, effective management is modelled and deadlines are met. It is often these characteristics, rather than technical skills, which prevent INGOs and International organisations from hiring local staff.

    The Migration Act student visa prevents those undertaking AusAID scholarships from undertaking substantial work during their study and they must leave Australia upon completion on their course. This regulation prevents them from working in Australian workplaces, which would be an equally valuable experience as attending lectures at an Australian university.

    I believe the scholarships should be reframed from a ‘study scholarship’ to a ‘study and work scholarship’ which allows recipients to work in Australia for a period of time before returning to their home country. The experience of working in Australia would vastly multiply the effects of the new technical skills.

    Reply Comment
  6. Can you point me to sources of more information about the following? “AusAID is also trialling a new tool – Sensemaker – to assess scholarships impact. Results of the trial will be available in March 2011.”

    I have written on its use elsewhere, not as an advocate or user but as an independent and interested observer. See http://mande.co.uk/2011/uncategorized/using-stories-to-increase-sales-at-pfizer/ and to a lesser extent at http://mande.co.uk/special-issues/participatory-aggregation-of-qualitative-information-paqi/#self-categorised

    Reply Comment
  7. An interesting post, Matthew. I’ve already had my say in The Canberra Times, but I must address some of the points you’ve raised, because I believe you’re being very generous to AusAID.

    For example, I’m surprised you praise the agency’s transparency on this matter. Did you know the documents about the scholarships that are now on AusAID’s website are only there because I requested them under freedom of information law? They are not new documents, as you point out. Perhaps you believe that AusAID was on the verge of publishing them, but that, for whatever reason, it never quite got around to it. I don’t.

    Indeed, I pointed out in last month’s Public Sector Informant that AusAID breached the FoI Act by delaying – for months – the release of these papers. Based on this conduct, and on the many years that passed without any serious attempt to evaluate the scholarships scheme, I feel confident in saying that AusAID (or perhaps, more correctly, the government) was not willing to discuss publicly the merits of this program. I am also willing to share with you some of the exchanges between myself and AusAID about the evaluation papers. Some of the responses were unequivocally disingenuous.

    Yes, AusAID has published a 2010 update on the scheme (what of the many other years?). I remain unimpressed. Look at those parts of the pamphlet that you have quoted in this blog. It is very basic information; the bare minimum one would expect of any government agency reporting on its expenditure. It is certainly not setting new heights in transparency, and I am surprised you are excited by it.

    Nonetheless, I believe it’s likely AusAID has improved administration of its scholarships since it received Margaret Gosling’s evaluations. I hope so. However, there is not yet evidence of such improvements (e.g. where are the tracer studies? It costs nothing to publish them online). And the agency’s past failures, documented in the newspaper, to respond to glaring concerns (come on – an interior design scholarship?) show it needed a prod. Or, to be less “sensationalist”, take this broader example, from part 3 of the Gosling evaluation: “Unfortunately, the major findings of this review are that the selection, reintegration and [monitoring and evaluation] processes of most AusAID scholarships programs are not in good shape … These problems have, for a very prolonged period, consistently defeated all attempts to demonstrate any level of effectiveness.” How was this allowed to happen?

    I have several roles as a journalist. The more altruistic of them is to expose potential maladministration and encourage public debate about important issues. Scholarship aid clearly needs this debate. Not because I say it does, but because organisations such as the OECD, ACFID and CIDA have expressed concerns in the past about its effectiveness, yet the Australian government has taken little notice. Indeed, it’s rapidly expanded its development scholarships scheme, without any evidence to back the soundness of doing so.

    Another of my roles as a journalist is more commercial: it’s to take a complex issue and simplify it, that so it captures the interest of more people than it otherwise would have. Occasionally, that means using plain English like “doling out” and “lucrative” (and the scholarships are lucrative, by any measure). I also used the accurate terms “confidential” and “internal”, because that’s exactly what the papers were, and because I always, on principle, call secrecy by its name.

    I’ve learnt, both as a public servant and a journalist, that governments sometimes only take notice of important issues when they are exposed publicly. (I wish this wasn’t the case, because governments today are far too media-centric.) I realise that aid practitioners are often uncomfortable when their sector is criticised, because it feeds the agenda of the Centre for Independent Studies and suchlike. But aid practitioners should, more than most, know the importance of transparency in discouraging waste and graft, and encouraging efficiency. Exposing poor practices helps eliminate them.

    I, too, hope the Auditor-General’s report, and the aid review, give this issue another much-needed kick along. But I don’t regard for a minute that my work was a “beat-up”, nor am I sorry to have written a word of it.

    Reply Comment
    • Dear Markus,

      Thank you for taking the time to write a thoughtful response to yesterday’s blog on scholarships. It sounds like you were put through the mill getting the Freedom of Information request from AusAID and it is good that you persevered so that we can all read the reports and have a more informed discussion on the aid program.

      You’ve clearly given the issues a lot of thought, so here are three debates that I’d like your views on:

      1) A debate about the debate. We’ve been running a series of blogs on the aid program over the last six months or so, and there are a growing number of other blogs on Australian aid issues. We also ran a conference on doubling aid in February. One issue that has struck us is that the debate among academics and aid practitioners (e.g. budget support, aid transparency, independent evaluation) is very different to the debate in the political sphere (e.g. cutting funding to indonesian schools, using aid to tackle Islamic extremism) and the mainstream media (e.g. expensive consultants, aid fraud). I liked that your article moved the debate from aid fraud to development effectiveness. Why is there a divergence of debates, and which debates should we be having?

      2) A debate about the development effectiveness of scholarships. My post isn’t a rebuttal of the points raised in your article, but rather a rebalancing to include additional information from AusAID and flag the ANAO and Aid Review reports, which will important inputs inputs into a discussion on the effectiveness of scholarships. You make some strong points on waste in the scholarships program, and like you I am skeptical, which is why I hope the ANOA report and Aid Review will zero in on this. Then there are then broader views on the scholarships program that should be considered, including from beneficiaries who have gone on to play important leadership roles in their countries, and a discussion of how scholarships fit alongside an approach to capacity building and investment in local tertiary institutions.

      3) A debate on transparency and accountability. This is where I strongly agree with you and we’ve been running blogs on this issue. The Australian public deserves much better information on how aid is spent and this is necessary for a healthy public debate and for accountability. AusAID needs to be more proactive in sharing and communicating information, which is why I am encouraged–even I don’t get ‘excited’ by such things–by their draft agency plan for the Information Publication Scheme. Australia has also signed up to the International Initiative on Aid Transparency (IATI) and Sandy Hollway has called for ‘warts and all’ transparency. If AusAID follows through on this, then it will an exciting opportunity to improve aid effectiveness.

      Thank you again for taking the time to write a thoughtful response to my blog and I hope you will continue to read our articles on aid transparency, accountability and aid effectiveness.

      Best regards,

      Matt

      Reply Comment
  8. Great post Matt. That (and the additional evidence contained within) has shifted my perspective on the matter.

    One question I’ve always had is that if we really give untied aid these days, why are scholarships so often still tied to donor country universities?

    Reply Comment

Leave a comment