Halfway into the 2030 Agenda, with progress on several Sustainable Development Goals stalling, is a good moment to ask about the effectiveness of the UN’s development support. With an annual spending of approximately USD20 billion, the UN’s development arm can, if effective, be an important tool for member states to mitigate global challenges around equality, conflict, and planetary boundaries.
Yet, the UN’s organisational performance for development effectiveness has received comparably little attention. The 2018 reform of the UN’s development system has focused on system-wide coherence, but not on the performance of individual entities.
This blog post presents insights from my ongoing research on the UN’s development work at the country level. It is based on a review of 28 UN country program evaluations of four individual entities (UNDP, UNICEF, UNFPA, UN Women). The selection of evaluations was broadly matched to where the UN allocates its development resources: the evaluations cover all five regions, but with a focus on Africa and UNDP.
An alternative to glossy and often quantified self-reporting, these evaluations offer relatively raw and direct, though perhaps incomplete, glimpses of the UN’s organisational performance for effectiveness.
Overall, a strong case emerges from the evaluations and the positive findings contained in them for the UN’s continued relevance in the area of development (which has become somewhat eclipsed by the UN’s growing focus on humanitarian assistance over the last decade).
One particular noteworthy take-away is the indispensable normative role UN entities play at the country level. Around the world, UN field offices help initiate and shape hundreds of national laws, budgets and agendas every year. Without that, national legislation on SDG-relevant issues would probably be much less aligned with global normative frameworks.
Generally speaking, the evaluations convey an image of the UN being most effective when it works coherently across sectors, applies long-term perspectives, brings together relevant stakeholders, provides a platform for civil society, and speaks directly to governments based on evidence.
However, the reality appears to be often far from this ideal. The most common and arguably worrisome deficit found in my analysis is that UN development work lacks an orientation towards sustainability and effects at national level – the kind of change that moves the needle on SDG progress.
A UNDP evaluation, for instance, notes that “only 50 percent of project documents addressed scale-up or replication approaches for innovative projects, and many through a generic paragraph”. Field offices often operate without sound theories of change that causally link support activities to national effects, and which can serve as a tool to monitor progress and engage host governments on larger issues.
Instead, field offices often pursue a multitude of small-scale work that does not add up to sustainable effects. As a finding from a UN Women evaluation notes: “Though these interventions were found successful, they are of a relatively small size and at best improve the lives of a few hundred to a few thousand women to some extent. Replication and scaling up have not been planned and large-scale replication of such pilots would be very expensive.”
Another recurrent theme is short-termism, which manifests itself in a focus on achieving quick and tangible results where a more comprehensive, long-term approach is needed. Such activities are typically driven by opportunities (donor funding, government demands). Short-termism and poor theories of change go hand in hand with a lack of integrated approaches, another frequently mentioned deficit.
This also affects the approach to partnerships. UN field offices’ partnerships were criticised in some evaluations for being driven by resource interests (who can contribute resources? who can spend them appropriately?) rather than by a consideration of the key stakeholders that need to be involved in reform coalitions.
Knowledge management constitutes another deficit. While UNDP emphasises its “thought leadership”, and while UN field offices do publish a lot on selected issues, the internal management and utilisation of knowledge is often criticised for being severely underdeveloped or entirely absent, as interventions end without lessons learned on what’s working, or for high-level policy advice.
Lastly, evaluations point out that monitoring tends to focus on activities rather than effects, which leaves field offices unable to adjust approaches. As an example, a field office will probably know how many people participated in a training, but not whether and to what extent the training achieved its purpose.
The common denominator of these deficits is “projectitis”, an organisational style characterised by decentralised, ad-hoc responses that are insufficiently integrated with, or informed by, the policies, purposes and overarching responsibilities of the organisation.
It is noteworthy that these deficits were found most frequently in UNDP evaluations. However, UNDP’s evaluations also stand out for the analytical rigour and critical attitude applied. Evaluations from the other three UN entities, and in particular UNFPA, often do little more than recount activities, oblivious to questions of sustainable effect. This inconsistent quality of evaluation raises a question as to the importance given to evaluations, or effectiveness for that matter, by UN entities and member states who govern them.
So, how can effectiveness become top priority?
Some humility is in order when judging the efforts of practitioners to promote sustainable development. Development support is a notoriously difficult thing, everywhere. Yet, some UN-specific factors that appear to hinder organisational performance for effectiveness can be pointed out.
First, the principle of strict demand orientation as set by the UN General Assembly contributes to making UN entities responsive to host governments, but not responsible with regard to mandates and mission.
Second, the donor practice of earmarked funding has a similar effect of turning UN entities into service providers, maximised for efficiency, rather than being globally inspired agents for change.
Last but not least, UN entities, competing for resources, have themselves been too willing to embrace short-termism and “projectitis”, arguing that more resources equals more effects. Better institutional safeguards against opportunistic behaviour are needed. Rather than the size and growth of the budget, performance towards mandates and effects should distinguish a UN entity.
The following country program evaluations were analysed for this review – UNDP: Armenia 2019, Botswana 2020, Guatemala 2018, Madagascar 2018, Namibia 2016, Nigeria 2021, Pakistan 2016, South Sudan 2021, Timor-Leste 2018, Vietnam 2020; UN Women: Albania 2021, Brazil 2021, Cambodia 2021, Cameroon 2017, Palestine 2017, Tanzania 2017; UNFPA: Caribbean 2021, Libya 2022, Kyrgyzstan 2016, Malawi 2018, Liberia 2017, Philippines 2018; UNICEF: Angola 2018, Ethiopia 2019, Jamaica 2019, Lebanon 2017, Philippines 2017, Serbia 2019.
Dear Mr. Baumann, thank you for these insights. While you do not provide a direct answer to the question of the effectiveness of UN’s development support, one must conclude: Not very effective. Or not effective at all.
Your focus on effectiveness is most welcome. While the UN agencies’ organizational performance did receive little attention in the 2018 UN reform wave, it had already been assessed, debated and ignored at the beginning of the millennium – for instance through the DfiD multilateral aid reviews.
We seem to agree on ills such as UN projectitis, short-termism, lack of scalable interventions, lack of lessons learned or viable theories of change, focus on obtaining resources rather than delivering on results. And I may add: very little capacity for critical self-reflection.
Your review is based on evaluations commissioned by the UN agencies themselves. Very few of these evaluations provide a remotely independent assessment of the effectiveness of the agencies’ work. The typical ‘country programme evaluation’ is designed as a formative evaluation to inform the design of the new country programme; evaluators traipsing around the ecosystem of UN officials and direct counterparts will hardly recommend to shut down the agency’s country office for the lack of results. Any study can point out some contributions made by a concerned UN agency; but this does answer neither to the question of effectiveness nor that of value for money.
Your first recommendation is a statement, which I wish to paraphrase: the current UN reform dogma has turned UN agencies into service providers accountable to host governments, at the cost of focusing on their mandate and their raison d’etre. The effect of this should be contemplated while looking at the countries at the bottom quarter of the corruption perception index.
I agree with you that performance, not the amount of funds raised, should be the measurement of success. It would be interesting to see a study assessing the development trajectory of a country, and the role played by the UN or other development agencies (if any) therein.
I disagree with your critique of earmarking. It is only natural that a donor wants to earmark funds to a well performing agency instead of throwing money into a muddled pool of underperforming organizations; it is also understandable that a donor may wish to finance a specific concern instead of a hotchpotch of issues that reflect every UN mandate under the sun. Earmarking is the market mechanisms that separates the wheat from the chaff. The ‘likeminded’ donors better start coordinating and rethinking their own bilateral aid.