Listening at scale: AI-assisted causal mapping for inclusive evaluation
Ouvindo em escala: Mapeamento causal assistido por IA para uma avaliação inclusiva
Gabriele Caldas Cabral, Steve Powell
Abstract
Evaluation practice often faces a trade-off between depth and metrics, frequently resulting in the marginalisation of local voices. Traditional methods of coding texts, such as interviews, risk filtering diverse narratives through pre-defined, donor-driven frameworks, perpetuating a form of epistemic violence that prioritises ‘data’ over ‘story’. We propose an alternative: AI-assisted causal mapping as a method for ‘listening at scale’ using a zero-codebook approach where Generative AI serves as a ‘low-level’ assistant to extract causal links from large volumes of unstructured text without an imposed codebook. A case study of Tree Aid in Burkina Faso demonstrates how this method reveals complex outcomes which might be missed by models that use a pre-determined codebook. We introduce ‘soft recoding’ as a technique to align local narratives with donors’ conceptual frameworks without erasing original data. We conclude by reflecting on the tensions between bottom-up storytelling and top-down accountability, arguing that while AI has its own biases, rigorous human oversight can use it to amplify rather than silence marginalised voices.
Keywords
Resumo
A prática de avaliação frequentemente enfrenta um trade-off entre profundidade e métricas, resultando muitas vezes na marginalização das vozes locais. Os métodos tradicionais de codificação de textos, como entrevistas, correm o risco de filtrar narrativas diversas por meio de estruturas pré-definidas e orientadas pelos doadores, perpetuando uma forma de violência epistêmica que prioriza os ‘dados’ em detrimento da ‘história’. Propomos uma alternativa: o mapeamento causal assistido por IA como um método para ‘ouvir em escala’ usando uma abordagem sem livro de códigos na qual a IA generativa serve como um assistente de ‘baixo nível’ para extrair ligações causais de grandes volumes de texto não estruturado sem um livro de códigos imposto. Um estudo de caso da Tree Aid no Burkina Faso demonstra como esse método revela resultados complexos que, muitas vezes, são perdidos por modelos que utilizam um livro de códigos pré-determinado. Introduzimos a ‘recodificação suave’ como uma técnica para alinhar as narrativas locais com os requisitos dos doadores, sem apagar os dados originais. Concluímos refletindo sobre as tensões entre a narrativa bottom-up e a responsabilidade top-down, argumentando que, embora a IA tenha seus próprios preconceitos, a supervisão humana rigorosa pode usá-la para amplificar, em vez de silenciar, as vozes marginalizadas.
Palavras-chave
Referências
Arrillaga, Elisha Smith, Grundhoefer, Seara, & Im, Christina. (2025). AI With Purpose: How foundations and nonprofits are thinking about and using Artificial Intelligence (44 p.). Cambridge: The Center for Effective Philanthropy. Retrieved in 2026, January 29, from https://cep.org/wp-content/uploads/2025/09/ CEP_AI_Layout_FINAL.pdf
Atari, Mohammad, Xue, Mona J., Park, Peter S., Blasi, Damián & Henrich, Joseph. (2023). Which humans? PsyArXiv. https://doi.org/10.31234/osf.io/5b26t
Axelrod, Robert. (1976). The analysis of cognitive maps. In Robert Axelrod (Ed.), Structure of decision: The cognitive maps of political elites. Princeton, NJ: Princeton University Press.
Better Evaluation. (2024, July 1). Causal mapping. Retrieved in 2026, January 29, from https://www. betterevaluation.org/methods-approaches/methods/causal-mapping
Causal Map. (2024). Retrieved in 2026, January 29, from https://causalmap.app/
Choudhary, Shivani, Chatterjee, Niladri, & Saha, Subir Kumar. (2022). Interpretation of black box NLP models: A survey. arXiv Preprint. https://doi.org/10.48550/arXiv.2203.17081
Copestake, James, Davies, Gabby, & Remnant, Fiona. (2019). Generating credible evidence of social impact using the Qualitative Impact Protocol (QuIP): The challenge of positionality in data coding and analysis. Myths, Methods, and Messiness: Insights for Qualitative Research Analysis, 17.
Eden, Colin. (1992). On the nature of cognitive maps. Journal of Management Studies, 29(3), 261-265. https://doi.org/10.1111/j.1467-6486.1992.tb00664.x
Escobar, Arturo. (2018). Designs for the pluriverse: Radical interdependence, autonomy, and the making of worlds. Durham: Duke University Press. https://doi.org/10.1215/9780822371816
Laukkanen, Mauri. (1994). Comparative cause mapping of organizational cognitions. Organization Science, 5(3), 322-343. https://doi.org/10.1287/orsc.5.3.322
Lynn, Jewlya, Stachowiak, Sarah, & Coffman, Julia. (2021). Lost causal: Debunking myths about causal analysis in philanthropy. The Foundation Review, 13(3). https://doi.org/10.9707/1944-5660.1576
Mignolo, Walter D. (2011). The darker side of western modernity: Global futures, decolonial options. Durham: Duke University Press. https://doi.org/10.1215/9780822394501
Powell, Steve, & Cabral Caldas, Gabriele. (2025). AI-assisted causal mapping: A validation study. International Journal of Social Research Methodology, 1-20. https://doi.org/10.1080/13645579.2025.2591157
Powell, Steve, Copestake, James, & Remnant, Fiona. (2023). Causal mapping for evaluators. Evaluation. London: Sage UK.
Ray, Partha Pratim. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, 121-154. https://doi.org/10.1016/j.iotcps.2023.04.003
Santos, Boaventura de Sousa. (2014). Epistemologies of the South: Justice against epistemicide. Boulder: Paradigm Publishers.
Stanford Center for Digital Health. (2025). Generative AI for health in low & middle income countries (52 p.). Stanford: Stanford University. Retrieved in 2026, January 29, from https://cdh.stanford.edu/research portfolio/generative-ai-health-low-middle-income-countries#:~:text=Generative%20AI%20(GenAI)%20 has%20the,implemented%20in%20low%2Dresource%20settings%3F
Tummons, Jonathan. (2014). Using software for qualitative data analysis: Research outside paradigmatic boundaries. In Martin Hand & Sam Hillyard (Eds.), Big Data? Qualitative approaches to digital research (Vol. 13). Leeds: Emerald Group Publishing Limited.
Tree Aid Project. (2024). Retrieved in 2026, January 29, from https://treeaid.org/
Submetido em:
29/01/2026
Aceito em:
26/08/2026
Facebook
LinkedIn
Mendeley
Reddit
Email