aes18 International Evaluation Conference

Ikarini Mowson (Clear Horizon), Byron Pakula (Clear Horizon)

Development aid is transforming from direct service provision to influencing policies to promote systemic change and achieve development outcomes. More and more aid programs are seeking to become catalytic drivers through influencing policies. These influencing programs have some distinct elements that mean traditional approaches to design, monitoring, evaluation and reporting are not as relevant. 

Drawing on experience in facilitating and developing monitoring and evaluation frameworks for policy influence programs, this paper presents some practical lessons that can be applied by designers, managers and evaluators. 

First, we must understand the five main characteristics of policy influence programs including complexity; unpredictable links between cause and effect; the scope and scale may move away; policy goals and commitments may change; and outcomes / impacts may be delayed. Second, there needs to be clear definition of policy changes expected in the programs. Policy changes may be defined in a very broad manner that allow the program to capture policy decisions and processes, including implementation. Policy changes could also be defined to capture every step in the policy cycle. Third, use people-centred approaches to theory of change including stakeholder analysis, in order to step out causal pathways and make sure intermediate outcomes are clearly articulated. Fourth, monitoring systems can be strengthened by using light approaches such as influence log to sufficiently capture the intricate details that are often not known if they will be the triggers of change. Fifth, apply multiple evaluation methods to measure influence, particularly methods to assess the contribution of an intervention to policy change rather than outputs or outcomes. Evaluating contribution is more realistic, cost-effective and practical than seeking to establish attribution or using experimental approach. Some outcome harvesting tools such as outcome mapping, episode studies or significant instances of policy and systems improvement (SIPSI) could be used in the evaluation.