Rural women in Manica, Mozambique expressing themselves through dramaI recently attended a seminar organised by SAMEA (the South African Monitoring and Evaluation Association) on the strengths and weaknesses of conducting Impact Evaluations (IE) considering that there is a tendency to conduct IE towards the end of a programme or project, or even not at all! The guest speaker at the seminar was Dr. Howard White, an internationally reknowned evaluation guru who has led the World Bank’s IE programme and trained close to 2000 DfID staff on M&E, and most recently was instrumental in establishing the International Initiative for Impact Evaluation (3ie). Phew, quite an impressive resume!!
In any event the seminar was extremely informative in that it raised several key issues that either support or contradict the MfI approach. I hope that this little report- back and some of the questions that I’ve raised from those issues that I found particularly useful, stimulates some debate amongst us MfI converts!
Dr. White gave a substantial input into the origins of IE, namely the need for the big and powerful international aid agencies (USAID for the USA and DfiD for the UK) to justify their billions of dollars worth of investments in terms of the difference this has made on global development. There’s such a proliferation of information and knowledge generated from IE’s since the 1990’s, yet very little (in my opinion) has changed with regards to global poverty. Is this a question of how much, or how often, or even when we analyse poverty alleviation goals through IE’s, or is it a question of the poor quality of the analysis of poverty used to inform policy change??  Well into the 21st cenury very same challenge remains intact- how do we really know that we are making a difference!
The speaker suggests that the logframe (although limited to logical and linear reasoning in the causal chain), it is without a doubt the most feasible mechansim by which to measure and show the direct links between inputs and outcomes. Granted that its main critique is that it is highly prositivist in orientation, the stark reality of our time is that policy making itself is undeniably highly prositivist in approach. The logframe helps us to understand how it is that X leads to Y in the most cost- effective and efficient way, which the non- positivist approach doesn’t seem to do. As someone steeped in the non- positivist approach to conducting IEs, I found this argument very difficult to digest, and yet at the same time I found myself wondering whether despite all our rhetoric on the dangers of positivism and the wonders of participatory M&E, we are still accustomed to working this way- it is after all quick-and-dirty! So let’s be honest, do we really engage with time- consuming and expensive participatory M&E that has at its core the empowerment of the poor and marginalised??
There were other interesting arguments put forward, but in the meantime I found the issues raised in this blog the most stimulating. It would be great to hear other practitioners’ responses to whether as development practitioners we are genuinely shifting the goal posts of poverty alleviation through M&E, and IE in particular.