American Evaluation Association Conference 2023

A growing suite of methodological approaches make causal analysis possible in complex, dynamic contexts where the more widely known causal approaches (experimental and quasi-experimental designs) are not appropriate. This full day training session will prepare participants to better understand when and how to integrate a deep causal understanding into an evaluation design. It will clarify how thinking about causality predictively (e.g. in upfront theories of change) is different from analyzing and understanding how, where, and why change is happening.

This session will focus on the power of participatory narrative methods, and aims to bring to life how they can deepen exploration of causal pathways with those most directly engaged in creating change. Often evaluators using participatory methods do not pay sufficient attention to causal analysis in the stories they encounter, undermining their potential to be more than anecdotes of individual experiences. The facilitators will set up this think tank session by briefly sharing their experiences using two participatory evaluation methods that include storytelling - change stories and outcome harvesting

This session is designed to bring to life the practice of participatory causal analysis, recognizing that making visible how and why systems are causing harm, and telling the stories of how and why change is happening when people take action, is more than just evaluation. It is a way to transform the system by combining the storytelling tools of evaluation, the power and insight of those most affected by the system, and the willingness of many different actors to seek to change the system. The presenters will bring to life the CLARISSA project, combining their own storytelling about the project and evaluation elements with videos of children and their advocates talking about how participatory action research is helping them tackle the worst forms of child labor. 

This session is the result of work by another critical initiative, the Inclusive Rigour Working Group, and has leadership from one of the Causal Pathways network leaders. The session is designed to bring to life the power of narrative methods for building ‘inclusive rigor’ in evaluation. Through illustrating how participatory narrative methods can be used within learning oriented evaluation designs, we will reflect on how stories can ground our understanding of what has been achieved, for whom and how, through the lived experience of system actors paying attention to the often excluded populations.

Outcome Harvesting is a method for telling evaluative stories about what has emerged from an intervention in a complex context. It is powerful in its ability to discover not only the predicted outcomes, but also what else is emerging and why. It is a storytelling tool that is often deployed in deeply participatory ways and can make visible many different voices and ways of understanding what is changing. At its best, it is not only about the outcomes: it is also about deep exploration of how and why those outcomes emerged - of the causal pathways leading up to them. This session will share three stories from very different types of initiatives in different contexts and geographies, each of which was implemented with careful attention to understanding the causal pathways of change.