Summary of Findings from Outcome Harvesting Mid-Term Evaluation of NIMD Country Programmes in Colombia, Mali, Mozambique and Tunisia

As part of the Mid-Term Review of the Strategic Partnership (SP) and Dialogue for Stability (DfS) programmes conducted in 2018, NIMD contracted consultants Goele Scheers and Ricardo Wilson-Grau to evaluate the mid-term progress of four NIMD country programmes (Tunisia, Colombia, Mali and Mozambique) using the Outcome Harvesting methodology. The main purpose of this pilot approach for NIMD was to assess:

1. To what extent do the outcomes — intended and unintended, positive and negative — achieved by the Mali and Mozambique SP programmes and the Tunisia and Colombia DfS programmes in 2016-2018 represent patterns of progress towards their respective SP and DfS programme objectives?
2. How well do the system-actor-culture outcomes match country-level ToCs?
3. How suitable is the Outcome Harvesting methodology for monitoring and evaluating NIMD’s outcomes?

The evaluators found first that the four NIMD partners have achieved outcomes in 2016-2018 that represent progress towards their respective programme objectives. All NIMD programmes influenced both expected and unexpected changes in political actors. They contributed to changes in the official rules and procedures governing politics, bolstered the legitimacy and capacity of political actors and influenced the behaviour of political elites in ways that strengthened political democracy.

Secondly, the evaluators concluded that the four NIMD programmes did achieve outcomes in line with their specific country-level theories of change (i.e. results areas/intermediate outcomes). NIMD’s staff and partners consider that overall, they made “great” contributions to “highly significant” outcomes. In Colombia and Mozambique NIMD had notable incidence on influencing change in the official rules and procedures governing politics, i.e. NIMD’s definition of “systemic” change. In Mali, NIMD’s incidence on the number of outcomes representing change in the legitimacy and capacity of political actors and on the behaviour of elites was all similar, at least in terms of the number of outcomes. And in Tunisia, the proportion of outcomes representing systemic change was considerably less.

Finally, on the suitability of the Outcome Harvesting methodology for NIMD, the evaluators advised that the possibility of substantiating NIMD’s outcomes can problematic due to the both political sensitivities of the work NIMD and its partners do, and the trust that is so vital to their success. The mere act of consulting with independent third parties about NIMD’s success in influencing other societal actors to change their behaviour carries serious risks of misunderstanding and distrust.

But NIMD programme managers and staff involved in piloting Outcome Harvesting in Colombia, Mali, Mozambique and Tunisia, with the limitations suggested above for substantiation, consider it a potentially useful tool for NIMD. After piloting the method in the four countries, the evaluators note a consensus amongst NIMD’s staff and partners that they consider the method more effective than NIMD’s current monitoring and evaluation tools for identifying their results and learning from what works. In addition, they appreciate the facilitative and coaching approach that Outcome Harvesting brings and the fact that they ‘own’ their outcomes. This is important, as their participation in the process is a crucial factor in its success.

The full Executive Summary of the evaluation can be downloaded below. If you would like to view the full Mid-Term Evaluation, please contact our Planning, Monitoring and Evaluation (PM&E) team at [email protected].