Paul Theroux once said that the short story is the most diabolically difficult literary form to master. I daresay he never tried writing a mid-term review of a justice or policing program.

In the early part of the last decade, I worked as a jobbing consultant and a few of these assignments involved conducting such reviews. Some of my oeuvre is available here, here, and here.

I felt reasonably well qualified to be involved in these efforts. I’d published academic articles and co-edited a book on the subject and worked for three years in a policing program in Timor-Leste.

And it was certainly a line of work that generated many unique experiences. I met astonishing people doing memorable things. I acquired many a good anecdote.

But no matter whether it was in Papua New Guinea, Timor-Leste, Marshall Islands or anywhere else, a number of factors contributed to making these assignments fiendishly challenging to write up.

The first was politics. No matter where we turned, we ran up stubbornly against it, expressed in all its various forms.

Some of this politics was personality-based, the time-honoured story of people from different walks of life just not being able to get along. I recall two co-chairs of a program dedicated to fostering partnership within the law and justice agencies in a Pacific nation unable to be in same room as each other.

Some resided within the ‘political economy’ of the country, between and within institutions and political groupings. Friendships, alliances, institutional restraints, and subterranean court politics — such who played guitar or tennis with friends in high places — seemed important too, as it is everywhere. I read Hilary Mantel’s Bring up the Bodies on one assignment that involved copious turf politics and remember thinking what I was experiencing was nothing new.

Other times, such politics was the reality of bilateral relationships between states. The diplomatic meta-narrative within which these programs take place is of #partnership and #reaffirmingcommitments; hard for the reviewer to square with partners that were, betimes, lukewarm or uninterested.

The second was complexity and how to render a messy situation in a tidy manner. To borrow the words of Harold Macmillan, the late British Prime Minister, it was ‘events, dear boy, events’ that determined the jagged and mercurial course of implementation more than the design document. Such ‘known unknowns’ include changes in governments and, with them, policies, as well as political leaders, heads of agencies and bureaucrats who change their minds or simply do not grasp the issue, setting plans often entirely off course. An extreme example, but once, in Vanuatu, the government in Port Vila wrote a bold new chapter in the story of the bilateral relationship by expelling all the Australian Federal Police (AFP) staff working on a police development program halfway through our review. (The AFP returned a year later.)

The writer faces the challenge of trying to write a story where much cannot be said and where even the most soft-gloved of observation and constructive criticism in the first draft can generate consternation, whether from the staff being reviewed, the institution they are working for, their partners or the funder.

And such sensitivity around what a report may say is completely rational. Trepidation within a donor or managing contractor about reputational damage to their brand is palpable. Success in promotion rounds and winning new contracts depended upon hawking achievements, not tarrying on missteps. No Ambassador or High Commissioner wants the contents of a page-turning review jolting the happy narrative on their social media feed. Contractual insecurity snaps at the heels of the consultants preparing the reports. I didn’t get paid for these assignments until the report was accepted, an imbursement arrangement which always concentrated the mind.

Removing all sources of consternation means that the end product always feels curtailed, akin to reading a novel with some chapters taken out. Is it surprising that, if and when eventually published, these documents garner such few readers? A few years ago, the World Bank was candid enough to disclose nearly half their reports did not have a single reader.

In 2015, after one assignment, the more bracing and surrealistic details of which might get included in a mystery novel sometime, I hopped off the treadmill.

I salute the hardy souls that have perfected this task for which one gets little praise, and often wondered whether there is something sui generis about law and justice evaluations that makes them tricky. Does the ambiguity of the topic make it difficult to assess? Do those evaluating, say, a market development facility run into similar challenges?

I find writing difficult and admire the talents of the many skilled authors working within the narrow confines of this genre. I take joy in trying to discern the intent behind their intentionally ambiguous word choices.

But I often read them and think ‘I’m sure I’ve read this story before’, and wonder how much learning these painstakingly put-together efforts generate. An admirably brisk evaluation of a New Zealand-funded Community Policing program in Bougainville conducted in 2019 resounded points made in a review I was involved in back in 2013. Which begs the question of just how governments that commission these reviews actually use these documents? Who reads the final draft? Is all the effort and angst of producing them actually worthwhile?

I’ve often wondered whether there is an argument for not publishing these documents or, at the very least, having some parts public-facing (and diluted as needs be) and some parts not. There are good reasons why tweets are public and diplomatic cables embargoed for 25 years and, in a similar vein, why politicians’ memoirs are more readable than their talking points.

On 1 October, I’ll be chairing an online session at the Law and Justice Development Community of Practice about the art of reviewing law and justice programs. All are welcome to join — register here.

We’ll be looking at three reviews undertaken in 2019: Indonesia, Solomon Islands (not uploaded yet; always intriguing when that happens) and Vanuatu. We are going to explore the headline findings of these three reviews and what they tell us about what works (and what does not) in law and justice assistance. We’ll ask how much these reviews are contributing to learning, program practice and the direction of law and justice assistance.

I’m looking forward to the conversation and learning from people with greater literary gifts than me. I’d love you to join us. Got a question that you’d like me to ask? Email me.