Heated debates about the volume and allocation of international aid have long been conducted under conditions of poor lighting — it’s not an easy matter for interested people, in both donor and recipient countries, to know exactly what is done with aid at the level of individual countries and activities.
The OECD’s Creditor Reporting System (CRS) helps somewhat. It’s an authoritative source of activity-level data (hint for the uninitiated: click on the expenditure totals to view the underlying activities). However, CRS data enters the system with a very long lag and the system is quite narrow in scope, having been designed long before contemporary transparency desiderata took shape. The CRS is a rich repository of quality-assured data on past commitments and expenditure, but contains nothing very current and nothing qualitative beyond short activity summaries.
In some respects, lighting conditions have improved dramatically over about the past decade. Aid transparency has received much attention and now figures in indexes, rankings, individual donors’ policy frameworks and the collective doctrine on development effectiveness embodied in the Busan outcome document. Certainly the news is not all that it could be, but it’s pretty good.
The vast bulk of the attention directed to aid transparency, however, has been directed to the reporting of information, most notably the reporting of it in line with the International Aid Transparency Initiative (IATI) Standard. In general, far too little attention has been directed to ensuring that the information reported is accessible and meaningful, and that it can be combined — in practice, not just in principle — with information from other sources in order to allow useful international comparisons.
While my subject here is not Australian aid transparency, Australia’s case illustrates one half of the point just made. While Australia’s Department of Foreign Affairs and Trade (DFAT) seems to be improving its IATI reporting performance, at least with respect to quantitative information, it has constructed no portal that would allow anyone easily to access the data, assess its quality and make use of it. Other donor countries are in a similar situation, though their IATI reporting performance might in some cases be below DFAT’s (and fixing that should probably be their first priority).
A few donor countries have built IATI portals, among which the Development Tracker of the UK Department for International Development (DFID) is the most impressive. Some recipient countries have done likewise, in order to gain an almost real-time overview of donor activity in their countries, and assist with aid management (Myanmar’s portal is here). If this practice were to spread, and donors were to continue to improve their IATI reporting performance (including with respect to document transparency), that would eventually deal with the “accessible and meaningful” point.
But what about the “useful international comparisons” point? While individual donor portals are important, they don’t help with international comparisons. For that, a global IATI portal is required. In this recent blog post, I noted that various such portals appear to have started up and either failed or not advanced very far, and that at present there are three main options, none very satisfactory.
One is the “query builder” provided by the IATI Registry itself, a very basic data extraction tool that requires the user to undertake laborious manipulation of the extracted data. A second is AidData 3.0, an ambitious database that pulls in data from the IATI Registry, the CRS and other sources. In principle, the data can be filtered and presented in many ways but, in practice, this database is difficult to use and seems not to deliver comprehensive coverage of data from the IATI registry. A third global portal, currently the best option for accessing IATI data from multiple donor sources, is d-portal.org. Useful as it might be, it’s not very flexible and has limited options for customising the presentation of data. It seems also to be running on a shoestring (see the “Funding” section of this page), so it is hard to be sure that it will be well maintained, let alone upgraded.
In that same recent blog post, I noted the impending launch of IATI Studio, a suite of tools to facilitate the publishing, extraction and presentation of data published by international development organisations in line with the IATI Standard. This has been on the horizon for quite some time, and looked from a distance like it might constitute a fuller-featured global IATI portal. IATI Studio has now been officially launched, at the end of May in The Hague. The launch was a few months delayed but still looks to have been premature. Unfortunately, the capabilities on offer at this point are slender and in beta — just a quite basic chart builder and a “community feed” for sharing the charts.
I used IATI Studio to generate and publish a simple test chart showing Australian aid expenditure in Indonesia from 2010 to 2015, in the top 20 sectors. You can see the chart here and a screen shot (which illustrates rather dramatically the rise and fall of support for basic health care — the green blocks — as a share of the program) is pasted below. The total level of expenditure each year, it should be noted, doesn’t much resemble reality, but presumably it does reflect what information is and is not contained in the relevant IATI records.
IATI Studio: Australian aid expenditure in Indonesia by selected sectors, 2010-2015
In making the chart above and generally poking around, I found the IATI Studio chart builder to be less than friendly. It can in principle do some reasonably complex things — see this example — but using it is laborious and confusing. Among other things, if you select a parameter that it doesn’t like, it just goes blank on you. For example, if you select “native currency” rather than US dollars for Australian expenditure, you get “no data available”. It’s unclear why that would be: national currency expenditure information is in the IATI registry. Moreover, since the IATI Studio chart builder is no more than its name suggests, you can’t click through to look at the underlying activity information. You can hover over a bar and find out how many activities contributed to the expenditure total; that’s all.
As noted, IATI Studio’s principal competitor right now is d-portal.org. While d-portal doesn’t have much chart-building capacity, it is otherwise a much more useful window into the IATI data registry. For example, you can ask for a chart showing Australian expenditure in Indonesia, in Australian dollars, by top sectors (it knows what these are, whereas in the IATI Studio chart builder you have to identify them by trial and error), and get the following:
D-portal: Australian aid expenditure in Indonesia by top sectors, 2015
You can then click through to view all sectors, and then click through any individual sector to get detailed activity information for 2015 as well as previous years. Among other things, this allows you to see if there are publisher errors in the data, such as the inclusion of Myanmar projects among Australia’s aid activities in Indonesia (which I did find).
It’s hard to avoid the impression that IATI Studio is struggling to get going, though the Amsterdam-based data visualisation company behind it, Zimmerman & Zimmerman, has previously built several good donor-specific IATI portals, including for the Dutch Ministry of Foreign Affairs and UNESCO. It was also contracted by the UK Department for International Development (DFID) to build the “dynamic data engine” that funnels data into its Development Tracker portal.
The next products in the IATI Studio suite, currently scheduled for launch later in 2016, might be more useful — and, not unrelated to that, they will cost money (specific pricing information is not yet available). These products are are an “IATI Publisher” and a “Microsite Builder”. The former is intended to reduce the headaches involved in publishing valid data; the latter aims to allow organisations to make their own IATI portals even if they lack the internal technical capacity to build them from the ground up. No doubt the Microsite Builder will also be handy for watchdog organisations wanting to make sense of publishers’ data.
If the IATI Studio team can in fact deliver a portal-builder that does not require fiddly and expensive customisation to meet the needs of individual publishers, developing country governments and aid watchdogs, they will have provided a much-needed service. It’s a service that some bilateral donors, including Australia’s Department of Foreign Affairs and Trade, which pour raw data into the IATI registry but have nothing resembling the UK or Dutch portals, might consider consuming. This would akin to buying a Mini Cooper though. Most OECD donors are big enough organisations to aspire to the DFID Rolls-Royce—and the source code is free.
Whatever emerges next from the IATI Studio project, there is still a problem about international comparisons and still a need for a decent global IATI portal. At this point, d-portal.org looks the best candidate to become one, but only with a very substantial investment of resources. I would think that no such investment is likely to be made unless a major multilateral organisation, such as the World Bank or the OECD, were to be given custody of it. There is in fact something to be said for the idea of progressively developing the OECD’s existing CRS database in the direction of the IATI Registry, so that it might contain a quality-assured collection of older data and a more fluid collection of current data, and so that it might allow additional information to be attached to records, including links to project documents. A look at this argument from 2009 for regarding the CRS and IATI as “complementary” mechanisms serves only to illustrate that there is no very good argument.
Robin Davies is the Associate Director of the Development Policy Centre.
Hi Robin, thanks for taking the time to write this piece, I found it interesting and informative to read your perspectives.
As someone who has spent ten years developing data management and interpretation tools in the private sector, I have often wondered about the opportunities for such work in the aid/NGO space. From reading your article it seems like there are two main problems at present – one is the usability & usefulness of the tools to date, the other is the quality of the data available on which they can operate.
What are your thoughts on a system that aims to address these two issues at the same time – that is, a system that actually enhances an organisation’s ability to manage its day-to-day operational data, while at the same time aggregating that data across participants and making it available via a suite of tools in a way that generates real intelligence and understanding of what is happening (both historically and in real-time) across a series of organisations, countries or sectors? The appeal to aid groups of using the system would be the removal of the burden of separately reporting on their activities, while at the same time collaborating with and learning from other organisations. For example, an organisation that plans to setup operations in a specific region could look at the data already aggregated for that area and make informed decisions about the most suitable vehicles, materials and so on, based on the experience of others already there. The same system also lets that organisation fulfil its reporting obligations to donors and the public, but importantly they haven’t had to duplicate their efforts. The data that they’ve recorded is just as useful to them for internal management as it is to external users. Obviously there would need to be rules in place about what level of detail can be shared, but the principle would be clear – write once and use in many ways, with collaboration & reporting built into the system itself.
Thanks again for your article, I’d love to hear your thoughts on this idea.
I suspect everybody who has worked in a donor agency, myself included, has at some time made the same wish, while cursing the waste of aid involved in chopping and changing between activity management systems, or jury-rigging solutions to emerging problems with sticky tape and string.
However, donor agencies’ activity management systems have to serve a number of purposes simultaneously, with the main elements being project-cycle management, payment management and information management. In all these areas, each individual donor will generally have a quite specific set of requirements to meet owing to government-wide policies on financial management, project approval, procurement and results measurement and reporting. Usually they will also have additional information collection and reporting requirements to meet that flow from their governments’ results frameworks for aid.
In these circumstances, I think the best that can be hoped for is agreement on a common international reporting standard for activity information (which we have), agreement to achieve certain levels of compliance with that standard by certain times (which we sort of have, though the agreement has mainly been honoured in the breach) and agreement to invest resources in a global mechanism that would give everybody and anybody the ability to access and manipulate the information provided (which we lack).
A better capacity to see what is being reported by donors should improve the quantity and quality of what is reported, and make it increasingly likely that donor’s individual activity management systems, whatever their differences, will fully and promptly dump activity information and related documents into the global pool.
Siem and Thomas — Many thanks for responding to the post. I’ve had a direct message from somebody at D-Portal along similar lines. I think we all agree that not enough resources are currently being allocated to the development of tools for manipulating and presenting IATI data, including data drawn from multiple donors for the purposes of comparison. I think we also agree that such resources should come from the official sector, since such tools are public goods. I hope it is clear that my aim in this post was to reinforce exactly these points, even where discussing the limitations of existing tools. There seems to be a real gap here, and it’s my view that the OECD’s Development Assistance Committee should be looking to fill it.
Siem — A quick further response on your two specific points. First, it is certainly clear now that D-Portal and the IATI Studio should not be directly compared; my point, as noted, was that from a distance, based on the information available last year, one might have assumed that the IATI Studio suite would include something like a ‘souped up’ D-Portal. And second, I now see that one has to choose both ‘Use native currency’ and ‘Convert all values to AUD’ for the chart builder to do what I wanted it to do, but I would still assert that this is confusing given that the former choice would seem to entail the latter! A very minor quibble, of course.
From a structural point of view OECD would make sense, but after talking to a number of people who work around these committees I am not convinced that would work. They don’t really seem understand the technology and the use of technology, and are not prepared to let those that do define and build the next generation of tools.
This is a general shortcoming across these relatively bureaucratic organisations. They don’t have a good CIO/CTO and a sustainable technology strategy and they think they can do without, which is really not feasible in todays world.
It is easy to criticise efforts that are in an early stage, like the IATI Studio or D-Portal, but as Siem correctly points out there is very limited funding available for this type of work. Building a comprehensive data solution that allows us to compare IATI data from different countries and organisations effectively is actually a fairly substantial task.
An even bigger task is actually to get all the organisations and governments to actually report meaningful IATI data. Something which you correctly point out is slow in forthcoming.
The organisations which are working on these type of data systems, including the one I am working with, Akvo Foundation, have been talking to quite a few funders and UN organisations about this. But there is little coordinated effort to create a source of funding where systems like this could be supported.
Siem and his team is doing really good work with limited resources. I know this, having worked quite a lot with them early on. What the governments and organisations who have committed to publishing IATI data should do, is get together and provide funding for efforts like IATI Studio, d-portal and others, so we can get from working with data as a nice concept to actually drawing the benefits from the data.
Co-director, Akvo Foundation
Thanks so much for you elaborate review of IATI Studio. You are the first to provide one like it. Upon reading the review, I have read some valid points on which I would like to provide feedback, hoping this makes for a better understanding of the functional rationale behind some of the Chart Builder features and how they work. We more than welcome feedback and will take yours into consideration as well.
First one: switching to a native currency. Your example chart is converted to USD and you get ‘no data available’ if you switch to native currency and leaving USD active, while having the context “Australian Government – Department of Foreign Affairs and Trade” as reporting organisation. Your are not supposed to see anything, seeing the Australian Government reports in AUD and not USD. If you would however change the active native currency to AUD, you will obviously see your data being plotted again. Give it a run, I just duplicated your chart, changed to native currency to AUD and that worked as expected. So using native currency requires the user to make it explicit. Makes sense, right?
With regards to the Chart Builder being basic, that has been a choice of design. This will be enhanced by providing the user with context of data in the chart (transactions, reporting orgs, receiving org. etc) so a more complete data story can be made, rather than just the one off chart.
A final note on competition: we do not perceive D-Portal as a competitor. Comparing IATI Studio to D-Portal is a bit of comparing apples and oranges to us. A complete Studio would entail a set of different tools, with a strong data infrastructure powering those tools. And as you rightfully state: the majority of funds to IATI Studio are invested by ourselves, no donor currently supports large efforts into this data initiative or any other for that matter. Very odd, seeing how we are supposed to revolutionise data for the SDG’s.
Do contact me if you have any questions.
CEO IATI Studio