SMIP Team in EA


The first engagement of ASSP/ASDP-L with Strengthening Management for Impact (SMIP) started with a training workshop on ‘Managing for Impact: from theory to practice’ held on November, 2007. This was followed by designing a participatory and learning-oriented monitoring and evaluation (M&E) system for ASSP/ASDP-L. Designing of the M&E system followed a participatory approach giving full consideration to stakeholder information needs and participatory data collection tools. The participatory data gathering methods were pre-tested and on-job training of the methods was given to staff of ASSP/ASDP-L and key stakeholders at the district and Shehia level.

Mr. Zaki K. Juma , Programme Coordinator of ASSP/ASDP-L pointed out the following changes in the Programme since the engagement of the programme with SMIP:

  • Greater engagement with stakeholders and beneficiaries: – has been strengthened and increased. Now ASSP/ASDP-L has good collaboration and contact with stakeholders and beneficiaries.
  • Participation: – after the training on MfI and development of M&E data collection tools; the participation of stakeholders in the Programme activities has increased. The stakeholders are now challenging ASSP/ASDP-L with questions to change/improve some of the data collection tools so that they fit the environment in which they are used. Beneficiaries, now, give feed-back on the tools to Agricultural Services Facilitation Team (ASFT).
  • Working as a team: – the ASFT works in teams, the interaction amongst the ASFT team has increased.
  • Linking M&E with management process: – this has increased. Previously ASSP/ASDP-L has been doing M&E for the sake of accountability, but now we use M&E for management process, for open, constructive and critical reflection of the programme management. •Internalizing M&E tools to beneficiaries: – beneficiaries feel that the M&E tools are part and parcel of the Programme activities and their engagement with the programme.
  • Carrying out Programme activities efficiently guided by the annual work plan and budget: – previously the annual work plan and budget was seen only at the end of the year – to find out that one/two activities were not completed and there are cases where left-out activities were done on ad-hoc basis. But now we refer to AWPB continuously as frequently as possible to ensure that we follow it.
  • Linking operations and activities with the programmes goal and objectives: – improvement in this area is under way. Link of activities with goals and objectives is given emphasis in the programme now.

Post by; Mr. Zaki K. Juma. , ASSP/ASDP-L Programme Coordinator April 3, 2009 Zanzibar, Tanzania

The SMIP team is currently in the process of supporting the Zanzibar ASSP & ASDP-L projects to roll out their participatory and learning oriented M&E Plan. One of the first activities was to try and build the capacity of and motivate District Facilitators (extension officers, subject matter specialists etc) to facilitate the Farmer Field School (FFS) monitoring sessions.We did this through training – consisting of 1 1/2 days in a “class-room setting” and 4 1/2 days in the field working with the farmers. (more…)

  We were on Day 5 of discussing and designing a participatory M&E system…with lots of positive feedback on this “new way of doing things”. Many individuals had come up to us, or said in plenary how excited they were to be involved in designing a system where the information would help the farmer, the researcher, the extension officer in making their own decisions. 

In the managing for impact approach; we recognize that it’s important to identify up-front what incentives need to be put in place to motivate people to get actively involved in M&E. 

So – we asked individuals to put on cards what incentives they thought would be important for them. Once they’d done so – we turned over blank flip charts with titles of different types of incentives; and asked everyone to walk around and place their cards where they thought they best fitted:

  • Financial
  • Material (new equipment etc)
  • Clear link to decision making
  • Recognition (e.g. being invited to give a presentation, or a pat on the back..)
  • Clear roles & responsibilities for M&E
  • Opportunities for training/skill-building
  • Others
I suppose you will not be too surprised to hear that the flip charts titled “financial” and “material” incentives were filled to the brim! 
Together, we reflected on this and the discussion gradually led us to development in general and the way we view it….
Now, it’s all very well for us to stand there looking down on people asking for more per diems/DSA…or a top-up on their salaries or any other form of “posho”! 
But…when I think about it a little – it makes a lot of sense. We’ve all read the endless criticism about development – and how it’s not making much of a difference. So – perhaps…over time – it’s gradually arrived at a point where the benefits of development aid projects are limited simply to “posho”.. make a quick buck while you can before the project ends! And honestly – when you think about the farmers listening to programme staff who’ve just driven up in their snazy 4-wheel drives or the “experts” flown in from abroad staying in posh(o) hotels….. you can see the message being given off right? Development project = posho! 

So, until we find ways to restore faith and belief in aid projects – that …with a little hard work & integrity…they can make a positive difference to people’s lives ..they can put a dent in this endless cycle of “poverty”…

Until we can find ways to restore faith & belief in M&E systems …that the information collected will really be used to help people make better decisions about their own lives – and influence the decisions of others…. I’m afraid “posho” will continue to be the stronger motivator!! 

Since my post on “Logic beyond the Logical framework“, we’ve now almost finished consolidating the work of the 4 different groups and putting it altogether into one MandE matrix.

The first step was bringing together the theories of change as viewed by the different stakeholder groups into one Programme theory. This is what it looked like (different colors represent changes related to different stakeholder groups and the arrows illustrate linkages/relationships between changes – both within stakeholder groups and between them).

ASSP ToC

From this – the Prog. was then also able to revise their logframe by clustering the outputs, outcomes (changes in behavior) and overall impacts from the diagram – which feed into the first column of the logframe. The items in red are internal/external factors that may influence the success of this theory. These feed into the 4th column of the logframe (risks/assumptions). However, they are also used to identify ways in which to strengthen the logic and reduce the probability of the risks (the green boxes are recommendations for changes to the theory). **Remember – this wasn’t a planning workshop ..if it was – we would have started from the top (impacts) and worked our way down. Instead, we were simply trying to ensure that everyone had a clear understanding of the strategy already designed and, if necessary, identify areas in which it needed to be strengthened. 

As I mentioned before – we also used this to develop the M&E matrix. Each step in the theory of change needs to be monitored, as does the relationships between them as well as the internal/external factors.  So – when thinking about information required for performance questions such as “To what extent were the intended impacts contributed to, why, why not?” – the programme stakeholders referred to the theory of change they mapped out (e.g. changes in % of income originating from sales of agricultural products in households participating directly in the Prog.). This part of the M&E matrix looks something like this: 

Information needsThe next step is to think through & plan for actual data collection. At impact level, it was felt that the data/information should be collected through both participatory impact assessments & externally conducted impact assessments. Here’s an example of the part of the matrix that relates to this: 

 

 

Data collection methods

 

Last, but far from least,  will be information use – defining and planning for the forums & other mechanisms that will be put in place in order to enable critical reflection on and the use of the data/information collected for decision making. 

Project inspection teams while conducting their review missions, almost always, ask whether projects have a monitoring and evaluation system. On the hand, I feel, they fail to give  equal attention to what happens to the information collected and stored through the monitoring and evaluation system .  

This was the background which motivated us to develop and incorporate a critical reflection mechanism as part of a participatory M&E plan. We thought that having a deliberately prepared and implemented critical reflection events will insure that the collected information is used for learning and decision making.

This reflection relates to a process we followed to understand the existing information sharing avenues the different stakeholders of a public agricultural development project have and come up with suggestion to use these as departure point  in  creating a mechanism of critical reflection for the project.   

 

The process followed as follows;

Phase 1. Group work

  1. Workshop participants were asked to form groups with their stakeholder colleagues and list down regular meetings/events organized by their stakeholder group and the frequency of these meetings/ events within the year on A4 card  (they indicated  frequency by producing copies of the meeting name  as number of meetings held  e. g . twelve copies for monthly,  four copies for quarterly, two for biannual, one for annually meetings)  etc..
  2. They were  also  asked to write issues discussed and reports made  in these events  on the same card (different color cards were given to the different stakeholder groups)
  3.  Facilitators prepared a timeline with months of the year on brown paper and spread on the floor.
  4. Then  participants were asked to put their card as  it fits to the timeline  
   

Phase 2. Tour of the market place

   

  1. Participants were asked to go around the displayed timeline and review the meetings being organized by other stakeholder groups including issues discussed in these meetings. While doing his they were also told to put small card with a similar color which identifies their stakeholder group on meetings that they attend   that are organized by other stakeholder groups.  Furthermore, they were asked to put similar sized cards but NEW written on them to show the meeting they want to attend as new participant.

 

Phase 3 Group work

  1. After   undergoing though this  process, participants  were asked to reflect on the exercise and discuss around the following  issue ;
    • Their general feeling about the exercise.
    • Their reaction to the new stakeholders who showed interest to attend their meeting
    • New issues they want to be discussed in the meetings organized by  others but they also want  to attend.

 

  Phase 3 participants

  1. Participants were asked to present results of their group discussion
  2. At last the facilitator made a wrap-up

 

My reflection of the process and the outcome

  1. The process made participants to learn about meetings or regular events organized by the different stakeholders which had a working relationship with them.  
  2. They also learned how they work in isolation as seen from the number of meetings organized by most stakeholders without  including other stakeholders
  3. I found the exercise to be useful in helping participants to understand  the  system best and suggest possible ways in creating a mechanism of critical reflection for projects involving different stakeholders.
  4. However, due to lack of time,  the process didn’t go further to come up with suggestion on how the whole critical reflection events should be coordinated in participatory way. This may be the area that should be explored further. 

 

When “managing for impact” – we try and avoid jumping straight into identifying indicators and, instead, first be clear about the questions we would ask about progress & performance. Once the questions are clear – we then think about what information we would need to answer these questions. 

However, the concept of performance questions can be a difficult one – especially for those that have had very little/no experience with M&E. So for the Zanzibar Programme (ASSP & ASDP-L), we used a set of generic questions and simply asked them to review these (adding/amending as they saw fit) – and focus on identifying the information required to answer these questions. We also tried to break down the questions so they were as simple as possible. For example;

  • To what extent have the expected changes in impact occurred?
  • To what extent were the changes in impact a result of the Programme’s interventions? 
  • To what extent were these changes the right changes to try and influence in order to meet the needs of the community & other stakeholders? 
This appeared to work and the Participants identified some extremely good information requirements to answer these questions. Needless to say, the information requirements they identified reflected a much deeper insight into the context than any M&E expert from elsewhere could ever do! 
If the M&E system is successfully implemented – the ASSP/ASDP-L stakeholders will definitely be having some very rich discussions during their critical reflection forums!

The ASSP/ASDP-L workshop in Zanzibar was an M&E planning workshop & not a strategic planning workshop. However, we could hardly start talking about monitoring until everyone in the room (including ourselves) was clear about what the Programme was expected to do, achieve – and therefore what should be monitored.

While I have used the Theory of Change to start off M&E planning workshop before, there were a couple of things we did differently this time that I feel worked quite well & I’d like to share. 

Logframes, as many of us know, are often developed by some external “expert” and may not necessarily make a lot of sense to those actually immersed in implementing the project. Also – there are times when the neat & tidy “component clumping” of activities, outputs & outcomes is very far removed from the intertwined nature of a project’s reality.

In a bid not to re-live challenges we faced before – we decided to ask individuals from similar stakeholder groups to work together to map out the logic of the ASSP Programme from their perspective. In other words – we asked them to:

  • Identify the services/products (“huduma”) that they as a stakeholder group were going to receive from the project; and then
  • Map out the changes in behavior that were expected to occur as a result of these huduma; and
  • How the changes in behavior would eventually contribute to changes in peoples lives

The facilitators probed and challenged the participants as they carried out this exercise – reminding them that changes in one stakeholder group could affect the behavior of another (and vice versa).

Finally, the participants were asked to identify and include in their “maps” internal/external factors that could have an effect on their theories of change; as well as recommendations for changing this theory (in this case we focused only on negative effects because of time).
What came out of this exercise was a very “rich picture”!

Asking stakeholders to focus on the huduma they were going to receive;

  • Ensured that they knew exactly what services/products to expect from the Programme and the changes in behavior (“ma badiliko ya tabia“) expected as a consequence;
  • Provided an opportunity for them to question and recommend changes to this theory, which in turn;
  • Started to build a sense of ownership of the Programme; and
  • Reduced the scope of what they needed to think through, as well as allowed them to think through an area they knew most about – resulting in much more detail than you would normally get.

The exercise also;

  • Enabled them to see how the changes in their behavior would or could affect others (e.g. if farmers started to pay for, monitor & assess the service providers, they in turn would influence the quality of services being delivered by the service providers); and
  • Illustrated to them that no one stakeholder was responsible for or had all the power to ensure the success of the Programme – everybody was dependent on everyone else and everyone affected everyone else.

Of course, we as facilitators, will have to relate all this back to the Programme Logframe. But I feel it’s a very small price to pay in return for the wealth of information & the exchange of views & knowledge that came out of this process. Also – the more detailed the theory of change, the easier it was for us to identify the information needed to monitor the Programme.

So, I’ll definitely think about using this process again under similar circumstances!

Next Page »