This blog is based on research jointly funded by the Development Policy Centre and the Research School of the Asia Pacific at the Australian National University, the Bobst Centre at Princeton University and Waikato University.
Aid donors and multilaterals make a point to operate in post-conflict and fragile states. The World Bank proudly states that fragile and conflict affected countries are priority areas, while AusAID itself directs 50% of its country specific aid to fragile states.
For all projects in post-conflict environments and fragile states, understanding the local conditions is vital for success. For many projects this also means gathering accurate data from civilians. However, people in these countries, particularly those who have recently survived conflict, may be unlikely to truthfully reveal their views across a range of topics such as equal rights, violence against women, freedom of speech and political violence due to fear of sanction by peers or authorities. Yet this information is important for donors when developing aid programs and for the international community when attention is drawn to post-war reconciliation issues.
One important consideration in post-conflict areas is whether people feel safe. Without safety, any benefits from development activities are surely muted. In 2009 the Sri Lankan government declared victory over a rebel group, the Tamil Tigers, after a 30 year war. Yet Sri Lanka has recently been in the news for all the wrong reasons; accusations of past human rights violations, claims of persecution by both Sinhalese and Tamil Sri Lankans and allegations of torture at the hands of the Sri Lankan authorities
However, notwithstanding a recent HRW report, most claims are anecdotal and/or based on a relatively small number of interviews. As such, in a recent Development Policy Centre discussion paper, John Gibson from Waikato University and I systematically tested whether people feel unsafe from death, torture or abduction in post-conflict areas of Sri Lanka. We hired an independent survey firm to randomly interview over 3,000 people in July 2012 across the key districts of Jaffna, Kilinochchi, Mullativu, Trincomalee and Vavuniya. These include the district that held the LTTE’s (the defeated rebel group) former de-facto capital and the district where over 200,000 people were trapped on 3.4 sq kilometres of land (about the size of New York’s Central Park) during the final stages of the civil war.
Understanding the sensitivity of any questions related to safety, we applied the ‘item-count technique’ (ICT) which asks questions indirectly, thus increasing the likelihood of honest answers. Recent applications of the item-count technique target biased voting reports, vote-buying and eliciting illegal migration rates. The ICT works by varying lists of questions. For example, in our study one random group of people are asked how many of the following statements are true:
- Religious worship is important to my life now;
- There are many opportunities for me to find work now;
- Children in my area are generally healthy now.
A separate random group of people are asked the same question with the additional statement:
“I do not feel safe from abduction, death and torture now.”
Given the randomisation process and large sample size, the difference between the average number of statements regarded as true in the first group compared to the second group estimates the percentage of people who do not currently feel safe from abduction, death and torture. Questions relating to safety are also asked directly of the first group.
The results are revealing. Using the ICT method, perceived lack of safety is the same for the majority Sinhalese and the minority Tamils: 29% of both groups are revealed to not feel safe now. Yet when asked directly, only 19% of Tamils and just 4% of Sinhalese respondents in these post-conflict areas reported currently not feeling safe. Qualitative analysis reveals the main reasons for fear include robberies (leading to death), alcohol-related and sexual violence against women (considered torture), elephant attacks, as well as some fear involving army activity and abductions by people in ‘white vans’. While fear in post-conflict environments is expected, unfortunately we do not have data from other countries for comparative purposes.
When we investigate whether people feel safe, direct questioning suggests 56% of people in post-conflict areas feel safe, while the ICT technique reveals 42% feel safe. Approximately 29% of people cannot commit to feeling either safe or unsafe.
Results suggest that data sourced from sensitive questions asked directly in post-conflict areas or fragile states may be inaccurate and lead to potentially counter-productive policy prescriptions. Given that development programs (and numerous academic papers!) use such data, it’s clearly important to get it right. Applying the item-count technique represents one possible option.
Dinuk Jayasuriya is a Research Fellow at the Development Policy Centre. The Sri Lankan government gave concurrence for the research but did not influence the design or outcomes of the study.
The devpolicy blog should have more articles like this. I think it is good to have a blog arguing about ‘big picture development’, but there are 100s of other blogs doing that. In contrast, there are very few which look into the METHODS of development. More posts on survey techniques, consultation methods, public communications strategies would be really useful contribution to the development blogosphere.
And easy way to start would be to create a new tag called ‘methods’.
Hi, interesting post, especially the Item count technique, I don’t know if this technique can be used for victimization surveys and domestic violence surveys, specially violence against the women, if that is the case, what would be the advantages of disadvantages of their use?
Thanks
Thanks Jorge. The benefit of the item-count technique is that it provides a degree of anonymity. In cases where survey anonymity is guaranteed (i.e. an anonymous internet survey with high response rates), then literature shows there is no significant difference in answers sourced from the item-count technique or direct questioning (see here). In instances where the survey is not anonymous (say direct questions related to sexual violence in the recent PNG HIES) then there is likely to be a difference between these answers and those sourced from the item-count technique approach.
The item-count technique can be used for your stated purpose however large sample sizes are required for statistical significance – which is a disadvantage. Kosuke Imai from Princeton University provides a nice (albeit at times technical) summary of the advantages and disadvantages of this approach.
Thank you very much.