Comments

This pattern of project ratings is typical across DFAT, the World Bank, and the Asian Development Bank. The WB and ADB have both published various studies examining this issue. I’m not aware of any similar publications by DFAT but I would be surprised if they hadn’t undertaken a similar analysis. To my way of thinking the key issue is “How is DFAT using this information about project success/failure to drive continuous improvement in the Australian aid program?”.

Hi Ed, Thanks for your comments. On your first point, we do address this in our report: https://devpolicy.org/publications/reports/Howes-etal_Recent-trends-in-Australian-aid-performance_May2023.pdf The disconnect between self-rated ICRs and more independent IEG assessments of completed World Bank projects is 15-18 percentage points in recent years (see footnote 18 of the report). Our estimate of the same disconnect in DFAT is 13 percentage points (p.21). The optimism bias is even greater at the Bank because the IEG assessments are (even) more independent than the DFAT final assessments - that's our interpretation.

Thank you everyone for some great comments. Just quickly on the empirical side: Edward you wrote: "In addition to the correlation between the abolition of ODE and the decline in ratings, I wonder whether there is also a correlation with initiatives that commenced shortly after the merger between AusAID and DFAT. If we assume that many Australian-funded initiatives are ~4 years, is it possible that initiatives commencing in 2014 or 2015, and getting rated from 2018 or 2019 onwards are also behind the decline in ratings? I’d love to see some analysis of that." Thank you, good suggestion, I will have a look when I get time. I suspect that the subset of projects that meet the criteria will be too small to allow for definitive analysis. But I could be wrong (I hope I am). It's worth looking into. Terence

I'll send some contacts to you in an email, Christine. Mike

Some of us want to tap into the Agriculture Sector but there is no contacts where we are able to contact people who might help us. What would be the correct number and email to contact DAL?

Scott Bayley raises an interesting point, about the intentions of the proposed Evaluator-General - which will be in Treasury. This probably reflects the personal enthusiasms of the Assistant Treasurer, Dr Andrew Leigh. But it will not be located in the Department of Finance, which wrote the APS Guidelines for undertaking evaluation (Resource Management Guide 130). The key question for all APS Secretaries will be how to cooperate with the Evaluator-General and really take "evaluation" seriously. Especially as DFAT didn't, in the critical area of our aid programs by abolishing the Office of Development Effectiveness. Even if devolving it to the line program areas. The secondary question is - what does DFAT management "do" with such a series of "large and growing disconnect between the assessments of ongoing and completed investments?" Under Section 15(1)(b) of the PGPA Act, Secretaries must govern their Departments in a way that "promotes the achievement of the purposes of the entity". Has the "promotion of achievement" instead driven these series of final ratings ? There is a decided disconnect with this analysis: "In 2021 and 2022, 36 aid investments worth about one billion dollars were rated as unsatisfactory".

In life, everyone has different battle to fight for their betterment. The true warrior defeats and stands out on the battle as a legend. Brother you are legend of your own battle. Your future looks promising. All the best in your future endeavor. Last and not the least is your hard working and struggle parents, make them proud and happiest parents as they deserved that. Love you my Ekni brother.

https://devpolicy.org/pacific-islanders-in-australia-census-results-20230331/

Hi Stephen and team, very interesting to read this and thanks for sharing. Two thoughts: 1. The World Bank has Implementation Status Reports which have ratings allocated by managers while a project is still operational. These ratings are the equivalent of the DFAT "ongoing" ratings, no? It would be interesting to aggregate a random sample of ISR ratings and compare them against the ratings given in the respective ICR and see whether the WB also has an optimism bias. 2. In addition to the correlation between the abolition of ODE and the decline in ratings, I wonder whether there is also a correlation with initiatives that commenced shortly after the merger between AusAID and DFAT. If we assume that many Australian-funded initiatives are ~4 years, is it possible that initiatives commencing in 2014 or 2015, and getting rated from 2018 or 2019 onwards are also behind the decline in ratings? I'd love to see some analysis of that.

Just wondering: are the levels of effectiveness and efficiency, confidence in the systems to monitor and respond to these, and communication about same related to the integration of the development cooperation program within DFAT?

I think your explanation is on the money. Given our new govt’s announcement of establishing an Evaluator General I wonder if DFAT will eventually move to rebuild its Office of Development Effectiveness and the independent oversight committee.

There will always be other considerations and tradeoffs which will prevent a real, full-scale economic, social, and political union unless there are very real forces that compel the participants to put aside their own petty concerns and join together for the greater good. Even Europe was not united until it was decided beyond reasonable doubt that the next great war on the continent would be the last war it ever saw.