Comments

Really sensible response, thanks Terence!

Really interesting reflections on an often-overlooked part of the research process: returning the findings to those they came from. Thanks for your open and honest sharing about challenges, positives and principles of this process. It's something close to my heart as a researcher and helpful to learn from how others have approached this.

Hi Suzanne, Good to hear from you. Sorry we didn't get to chat in Wellington. With all aid projects some selection has to be made: which country to work in, which region to work in and so on. This is as true with projects that are evaluated using RCTs as any other method of evaluation. With RCTs, once that selection has been made, the actual allocation of people, or villages, or appropriate units, to the treatment is done randomly. What would be a more ethical approach? With respect to internal validity, with random allocation, so long as sample size is large enough, the group of people (or villages, or whatever) that receive the treatment will be effectively identical to the group that does not receive the treatment. (It is also possible to test for this, and control for this, at least on observed traits using regressions). That means it is very much possible to compare outcomes in this sense. With respect to external validity, it is not guaranteed that a treatment that worked in one location will work in others. This is a serious limitation. Although: (a) it does not invalidate utility for individual evaluations and (b) can be overcome *to some extent* with meta-analysis. There are many other problems with RCTs -- there is much that matters in both aid work and development that they can tell us nothing about, they don't answer how/why questions well -- and so on. But their strengths are real enough. They eliminate confounding factors, they reduce subjective assessment (albeit not entirely in all instances). As for positivism: it seems to me that the null-hypothesis significance testing, which forms an integral part of how people read the result of RCTs, isn't really positivism at all, rather it's Popperian or post-positivist. But as to whether this broader family of approaches can be applied to social science...sure it's applicability can be questioned, but so can that of post-structuralism, critical theory, interpretivism, and so on. Debates amongst philosophers will continue for a long time to come. I really enjoy listening to podcasts about this stuff in the weekends. For now though, at the very least I think it's fair to claim that broadly empirical approaches to social science produce enough useful knowledge to warrant their continued use. I'm very happy to concede that this should be done with humility about what we can and can't know. This should be the case with everything in aid work. Have a good week. Terence

Thanks Garth, I think you make a really important point with the following: "Like so many people the authors appear to have an inflated view of the scale of aid both in terms of donor and recipient income and energy. Given how little effort is put into aid it is amazing what has been achieved especially in areas that are well understood such as basic health services, water, sanitation, education and agriculture."

I take issue with randomised controls not having ethical issues - apart from who gets to select which village (just one example) has access to a particular programme that conditions are such that its not possible to compare outcomes, which is what RCTs aim to do. The notion that positivist methods can be applied to social research is questionable at best

Dear Michael: I enjoyed re-reading your article and attending your seminar at UPNG. Agreed that strengthening the judiciary may be a way to counter the excesses of the PNG Parliament. However one should also be wary of buying into the neoliberal agenda which includes " a relocation of power from political to economic processes, from the state to market and individuals, and finally from the legislature and executive authorities to the judiciary." (Thorsen and Lie 2010).

I agree with their emphasis on changing the rules of the international economic system but can't we do this at the same time as providing useful development assistance? Like so many people the authors appear to have an inflated view of the scale of aid both in terms of donor and recipient income and energy. Given how little effort is put into aid it is amazing what has been achieved especially in areas that are well understood such as basic health services, water, sanitation, education and agriculture. The progress to date shows that the main impediment to further progress in all of these areas is lack of funding rather than lack of knowledge or international economic arrangements.

Brilliant Michelle, many thanks for sharing this solution. I too have agonized about whether I have my Zambian mining mining impact research story right and how to share the findings to those involved who are not big readers. It also picks up on an aspect of a paper I wrote for the last Dev Bull, No. 79, Jan. 2018, about the problems of research. This inspires me to do some more work on this aspect when I finally finish the seemingly endless editing.

Another excellent short read. I wonder, though, whether it is possible even in the medium term to strengthen the judiciary and the Ombudsman Commission to a stage where they can begin to function openly, honestly and effectively. Numerous programs over many years have had that aim, but politicians have continued to erode institutional independence, objectivity and functionality, undermining competence, honesty and good intentions, and transparency and accountability.

Why does AU continue to ignore that a Vanuatu private sector-led initiative is already building a cable to SI without aid support? VU has awarded the cable supply contract to a USA supplier (not China). The enormous AU cable grant to SI and PNG is distorting the market.

Genius, Ashlee! And so insightful. I'm definitely saving this one, and sharing it widely. Thank you!

Once again, sadly, one of the main frontline institutions to protect the security of our blue continent is missing from diplomatic discourse. While there are easy aid millions for police, spies, politicians, bureaucrats, judges, business people and all manner of "civil society" organisations, journalists and news media must scrimp and save pennies to be the first and last recourse of the public. In short, there is a fatal lack of public policy space for the Fourth Estate. If Biketawa+ is to have any hope of success, along with myriad other declarations past, present and future, the main avenue for the ears and voice of the people must be paid full and proper attention. Journalists and news media have subsidised the rest of society for far too long, through long hours, low pay, high stress and poor profits. As climate change assumes ever more severe proportions, along with myriad other complexities, our islands cannot afford to ignore this sector any longer. https://www.forumsec.org/biketawa-declaration/