An overview of Burrell, J. What constitutes good ICTD research. Information Technologies and International Development. Volume 5, Number 3, Fall 2009, 82–94
In her quest to delineate what constitutes good ICTD research, Burrell expressed that disagreements in this emerging field are bound to be rife. One of the factors that often play a role, she contends, is that of research quality in a interdisciplinary community. Economists, sociologists, computer scientists and media scholars are bound to disagree regarding what constitutes good, sound research practices. She cites an example where Waverman et al (2005) conduct an econometric analysis of the impact of mobile phones on the national GDP that “suggest that greater mobile phone penetration contributes to growth in GDP”. Although this findings relies on two widely accepted econometric models, the article does not explore the micro-economic forces at play. This kind of deduction from a causal relationship might be a bit “unsettling” for qualitative researchers, especially those who conduct observations. She cites one of her own studies as an example: a study of West African e-mail scams (Burrell, 2008) where the subjects she interviewed were not randomly sampled, nothing on “statistical significance”. Yet, within her study the significance in the analysis was not in identifying averaged “trends in scamming” – as she says “the who, where, how often, and how much questions” (83) but instead try to answer the “what, why and how” in order to more fuly understand the social context and ecology in which “scamming e-mails and activities are constructed from the little-known perspective of the scammers themselves” (83).
With such vast methodological differences, Burrell argues, it is important to remember that “these particular techniques represent differences in research question and approach” – “not research quality” (83)
In order to delineate the task that is being undertaken, Burrell identified 4 key features of ICTD research overall:
“1. Regardless of how we define the phrases “information and communication technology” and “development,” ICTD research broadly involves a consideration of human and societal relations with the technological world and specifically considers the potential for positive socioeconomic change through this engagement. This intersection of interests narrows down considerably what is relevant from the broader realm of theory in the social sciences, in development studies, policy, relevant professional fields, and in engineering.
2. Our work as researchers often entails considering and interacting directly with people in a socio-economic strata quite different from our own and in distant locales. For many researchers, being an outsider requires that we come to terms with the multi-faceted differences (political, economic, cultural, ecological) from previous experiences and what is formally taught in our fields of study.
3. ICTD research involves both studies of the interaction between people and technology as it exists or evolves, as well as active intervention work—introducing a new device, system, or policy to achieve some objective of development. This wide range of objectives demands certain sensitivities of method.
4. Due to its breadth as well as its range of research activity, ICTD is richly multidisciplinary, and therefore there are challenges of communication and a lack of a shared foundation of concepts and terms. Some of these characteristics of ICTD research are shared with a number of other disciplines. Yet, the combination points to a unique set of methodological concerns.” (83, 84)
Burrell furthermore explores the two concerns of research proposed by Gaskell and Bauer “that cross methodological boundaries” namely Confidence and Relevance.
Confidence includes the ways in which researchers demonstrate that a legitimate eresearch process lead to legitimate data and analysis and presented legitimate findings. In ICTD these concerns are broken down into accuracy, transparency and soundness of method, and empiricism.
Relevance looks at whether the research has broader significance within a field.In ICTD this is broken down into novelty, disciplinary relevance, and generalizability.
Whether the phenomenon under study is adequately describe or captured in its reporting with some degree of precision. “Perfect accuracy may not be achieveable, but greater accuract is always desired” (85) even if you are an interpretivist or an ontological realist, “generations of knowledge beyond the pure subjectivity of the researcher are desirable”
TRANSPARENCY AND SOUNDNESS OF METHOD
Explicit information such as where, what, who, the systems introduced, how these were constructed, what were the underlying processes and how data was collected, all on which the analysis rests, help to increase your own and others’ confidence in the research.
Extrapolations of findings, or systems, need to be “buttressed” by an empirical foundation of claims that are “well and clearly supported with good evidence”: these claims need to be “of the world” and not arguments that purely follow on theoretical or philosophical ideas.
Some aspect of the research must not have been done before. Yet something in the work should “extend the state of knowledge in the world” (87)
One needs to situate the new research within the ongoing dialogue produced by the relevant research community.
How does the research fit in with society in a broader sense? Accuracy cannot be compromised. If a certain finding is true for one population, it does not mean that it would be true for another. And even amongst the same population, the nodes in the relative network might slightly shift and have completely different results. One needs to outline the extent to which ones’ findings are generalizable and to what extent they are not.
REMAINING POINTS OF CONTENTION:
One of the main remaining points of contention is that of the philosophical foundations on which research is built. The differences between scientific realists and interpretivists.
Furthermore, researchers are essentially human and are there biased by their own socialization and belief systems: these can be viewed as a pollutant (or bias) or as a resource in itself. Researchers, through the act of difference, can verbalize aspects of others’ social practices which would otherwise seem normative or transparent. To be able to bridge this notion (whether subjectivity is seen as a bias or a resource) is through intense reflexivity.
Lastly, the difficult question of what is significant for the global development agenda? ICTD research is concerned with socio-economic development: better education, improvements in health, increasing freedoms. Yet recent studies have indicated that non-instrumental uses of technology – such as music, film, social networking, and other forms of entertainment – can drive interest and comfort with technology and on a sociological level affect the way the poor and marginalized envision and process a “better life”. Yet, entertainment is not on the research agenda. Thus we need to outline what constitutes something “akin to development according to the populations we study” (91).