Evaluation of the sustainability of the Metekel Zone CDTI projeGt, Ethiopia May/June, 2009

{World Health Organization African Programme for Onchocerciasis Control Evaluation of the Sustainability of the Metekel Zone CDTI ProjeGt, Ethiopia May/June, 2009 Dr. Johnson Ngorok, Team Leader Dr. Margaret K. Akogun Dr. Tekola Endeshaw Mr. Abraraw Tesfaye Mr. Birhanu Melak Mr. Tamrat Belete RECU LE I 0 Jijih ?iri3 t , : t APOC/DIR 1) )"' t' t t.. ITable of Gontents Page List of Abbreviations.......... ................3 Acknowledgements... .4 Executive Summary E 1.0 lntroduction 2.0 Methodology....... 2.1 Sampling.. 2.2 Protocol.. 2.3 Performance of the lnstrument. 2.4 Limitations 2.5 Team Composition................. ..7 ...........8 ...........8 o ............v . ..........10 '. ,., .,, ,..11 ,..,,,...,,12 v 3.0 Findings and Recommendations 3.1 Zonal Level.. 3.2 District Level.. 3.3 Front Line Health Facility Level... 3.4 Community Level.. 4. Conclusions......... 4.1 Grading the Overall Sustainability of the Project.. 5.0 FeedbacUsustainability plan development workshop. 6.0 Advocacy visits and debriefing. Appendices: Appendix 1: Evaluation Time Table Appendix 2: Programme for Feedback/Sustainability Planning Workshop Appendix 3: SWOT Analysis Appendix 4: List of Respondents Appendix 5: List of Documents Reviewed Appendix 6: Sustainability Plans. 14 .14 .23 31 38 44 44 49 .....51 t J 2 aList of Abbreviations APOC CDD CDTI EPI FLHF HSAM tEc MOH NGDO NOCP NOTF REMO SWOT TCR ToT WHO ZOTF ZHDH African Programme for Onchocerciasis Control Community Directed Distributor (of ivermectin) Community Directed Treatment with ivermectin Expanded Program on lmmunization Front Line Health Facility Health Education/Sensitisation/Advocacy/Mobilization lnformation, Education, Communication Ministry of Health N on-Governmental Development Organization National Onchocerciasis Control Program National Onchocerciasis Task Force Rapid Epidemiological Mapping of Onchocerciasis Strengths, Weaknesses, Opportunities, and Threats Therapeutic Coverage Rate Training of Trainers World Health Organization Zonal Onchocerciasis Task Force Zonal Health Desk Head a t J Acknowledgements The Evaluation Team for MetekelZone CDTI would like to thank APOC Management for the opportunity to be of service to the people of Ethiopia. The team appreciates the logistical support and administrative arrangements that enabled the evaluation be conducted without much difficulty. ln particular, the team would like to acknowledge the following persons and institutions for their help: . The Director and the staff of the African Programme for Onchocerciasis Control (APOC) in Ouagadougou. . WHO Country Office, Ethiopia, in particular Mr Tamrat Belete for in-country logistical arrangements. . The Carter Centre Country Representative for stepping in to take up the role of the MoH which was in the midst of a restructuring exercise. . The Metekel Zone officials (Zonal Administrator, Zonal Oncho Coordinator and the Head of the Health Desk) for the valuable support provided. . The District Health Teams of Pawi and Dangur districts for the arrangements and making available the documents required for review. . The representatives of Mandura and Guba districts for participating in the feedback and sustainability planning workshop. , The Heads of the six FLHF visited for participation and mobilising the communities for the evaluation. . The community leaders, community directed distributors and community members of the twelve communities visited. . The scout, Mr. Abraraw Tesfaye, for planning the evaluation. ' 4 fl EXECUTIVE SUMMARY Metekel zone, located in the north-western part of Ethiopia, is one of the three zones in Benshangul Gumuz People's Regional State. The zone is administratively divided into seven woredas (districts) and 125 kebeles (village/group of villages). Of the seven districts, four i.e. Dangur, Guba, Mandura and Pawi are oncho endemic. CDTI project was initiated in Metekel in 2004 and is being implemented in the four endemic districts. The evaluation was carried out at four levels; zonal, district, FLHF and community. Overall, the project performance was rated for each indicator per level as in the table below. Using the instruments as a "lens" for assessing the "Aspects of evaluation", the evaluation team found out the following tntegration; There was strong integration of CDTI activities into the overall health programmes. Oncho control activities were included in both the annual operational and the five year strategic plans; transport and materials were shared across different programmes and health staff carried out support supervision in an integrated manner. Resources; The integrated use of vehicles, equipment and other materials provide a good basis for sustainability of the programme. Although government direct funding commitment for oncho was not clearly spelt out, funds from government and other sources were made available for oncho control when activities for the different programmes were conducted in an integrated manner. The commitment of the NGDO partner to continue providing financial support reinforces the sustainability of the project 5 E,,c tr lll o. .g ! g. o!, ooJ ottr EDO .=o o> =; Ea" @tr o .N o o = C" .E= 'E6 F- EstU'E ED>o'5 E€ t o CL otr c, v.I ootr ociI o ED r! o oo oo(l o Zone 3.2 3.3 3.0 3.8 3.3 3.7 2.5 3.0 2.3 4.0 3.21 District 3.3 3.5 3.2 3.5 3.2 3.8 2.7 3.8 2.8 4.0 3.38 FLHF 2.8 3.0 2.8 3.5 3.2 3.7 2.3 3.5 2.2 4.0 3.1 Comm 3.3 2.7 3.3 3.5 2.3 3.7 2.0 3.0 2.97 Overall average 3.20 I a Efficiency; Efficiency of the use of resources was achieved by integration, conducting training sessions in cost free venues and limiting the duration of trainings. However, supervision, trainings and HSAM activities were not targeted; they were carried out routinely every year. These potentially waste limited resources. Simplicity: Simplicity was demonstrated in several ways, for example; delivery of mectizan from FMOH to the community level using existing structures and procedures, delivery of reports by CDDs to the FLHF and onwards as part of routine activities and without using extra resources; trainings being carried out in premises owned either by the government or the community. Attitude of staff: The health staff and the CDD expressed great interest and willingness to continue carrying out CDTI activities after APOC project phases out. The health staff considered CDTI as part of their routine duties. For the CDDs, this was despite the fact that they were not receiving financial or material support from their communities. Community Ownership: Although the involvement of CDDs and community supervisors was commendable and promoted sustainability, the community leadership and the rest of the community members did not show active involvement and ownership of the programme. Effectiveness: Full geographical coverage and therapeutic coverage rate of over 65% were achieved during the past three years. Therefore on the basis of the following: Aspects; five of the aspects were helping while two were partially helping the sustainability of the project. Critical elements, Allthe five critical elements for sustainability were present. Quantitative; The average score is far greater than 2.5 the evaluation team concluded that Metekel Zone CDTI project, in spite of some imperfections, meets the description of being "FULLY SUSTAINABLE" t t 6 tI 1.0 INTRODUCTION Project location Utete1et zone, located in the north-western part of Ethiopia, is one of the three zones in Benshangul Gumuz People's Regional State. The zone is administratively divided into seven woredas (districts) and 125 kebeles (village/group of villages) and has a total surface area of 259,932 km land mass. The climate of the zone is typically tropical with two distinct seasons i.e. the rainy and cool season from June to October and the hot and dry season from November to May with temperatures ranging from 25 to 42 Degrees Celsius. The topography of land consists of low land (82%), semi highland (10%) and highland (10%). The economic mainstay of the inhabitants is agriculture involving both crop farming and the herding of animals. Gilgel-Beles is the zonal capital and the seat of the zonal administration. Onchocerciasis Control in Metekel Before the initiation of the CDTI project, oncho treatment in Metekel was clinic based. REMO survey conducted prior to initiating the CDTI project revealed that onchocerciasis was endemic in four woredas of Benshangul Gumuz People's Regional State; Dangur, Guba, Mandura and Pawi. Apart from Pawi district which was found to be oncho hyperendemic, all the other three districts were oncho mesoendemic. CDTI project was consequently initiated in Metekel by APOC in 2004 in the four districts. Of the four, three of them (Dangur, Guba and Mandura) are located in Metekel zone while Pawi is a special woreda which also has the status of a zone. The four districts have a total of 145 villages The health infrastructure in Metekel CDTI project area consists of 1 zonal hospital, 4 health centres, 19 health stations (18 government and one NGO) and 11 government health posts. The Carter Centre is the NGDO partner 7 2.0 METHODOLOGY 2.', Sampling Sampling was carried out by the scout using primary and secondary criteria in line with APOC Guidelines. Primary criteria considered the geographical and therapeutic coverage rates of the study areas. ln this project, the onchocerciasis-endemic districts, the FLHFs as well as the communities had all achieved a geographical coverage of 1OO%. Therefore, sampling considered therapeutic coverage rates and the level of onchocerciasis endemicity which varied across districts, FLHFs and communities. The secondary criteria considered accessibility and the convenience of working in the potential study areas. Consequently, two districts i.e. Pawi and Dangur were sampled for the study. Pawi was hyper-endemic while Dangur was meso-endemic. To complete the study samples, three health posts and two communities under each of the health post were sampled. Sampling of Districts (Woredas) The four districts in the zone i.e. Dangur, Pawi, Guba and Mandura were considered for sampling. Two of them i.e. Guba and Mandura were excluded on the basis of the secondary criteria. Guba district is located 150 kms away from the zonal capital and 188 kms from the headquarters of the evaluation team which made it rather inaccessible considering the time available for the evaluation. On the other hand, inter-group clashes had been reported in Mandura district which potentially posed security risks. Consquently, the remaining two districts i.e. Dangur and Pawi were sampled. Sampling of Front Line Health Facilities (FLHFs) The FLHFs were sampled using the primary and secondary criteria. lnstead of the recommended two FLHF per district, three FLHF were selected for each of the two districts in order to make up for the required six districts. After excluding some FLHF on the basis of accessibility, three FLHFs were randomly sampled for each of the two districts as shown in the table below. t 8 Selected District Therapeutic Coveraqe Rate Dangur 69% Pawi 74% District Selected FLHF Therapeutic Coverage Rate Dangur Manbuk Health Centre 70% Gublak Health Post 77.7o/o Buri Health Post 51% Pawi F Health Centre 75% Ketena 2, Village 131 Healt[Pqs'L- 78.3% Ketena 1, Village 7 Health Post 73% Sampling of communities Lastly, two communities from the catchment areas of each of the six FLHFs were randomly selected after excluding the remotely located and inaccessible ones. A total of twelve communities were therefore sampled as detailed in the table below. 2.2 Protocol Research Question: How sustainable is the Metekel CDTI Project? Design: Cross-sectional, descriptive. Population: The Metekel CDTI project, its NGDO partner (Global 2000/The Carter Center), the staff involved in onchocerciasis control at woreda and FLHF levels, the project communities, with their leaders and CDDs. lnstrument: * A record sheet, structured as a series of indicators of sustainability. The indicators were grouped into nine/ten categories/ groups. These groups represent critical areas of functioning of the program. * The instrument assesses sustainability at four levels of operation. t 9 FLHF Community Therapeutic coverage rate Manbuk Health Centre Ketena 1 65o/o Ketena 3 79% Gublak Health Post Gublak Village 79% Kola Villaqe 72.3% Burj Health Post BuriVillaqe 46.1o/o Asama Wuha Village 35% Felege-selam Health Centre Felege-selam Town 76.3% Medin Village 57% Ketena 2, Village 131 Health Post Village 131 84% Village 29 74.2% Ketena 1, Village 7 Health Post Villaqe 6 76/% Village 7 71% i I I * The instrument guides the researcher to collect relevant information about each indicator from a variety of relevant sources. Sources of information: * Documentary evidence and observations. * Verbal reports from persons interviewed. Analysis: * Data from all sources was aggregated, according to level and indicator. * A qualitative summary of the situation regarding each indicator at each level was made. This was aggregated and summarized for each category of indicator for each level. * Based on the information collected, each indicator was graded on a scale of 04 in terms of its contribution to sustainability. * The average 'sustainability score' for each group of indicators was calculated, for each level. * Finally an overall assessment of sustainability was made, by considering the 7 aspects and 5 critical areas of sustainability. Recommendations: * These were strictly based on the findings of each program evaluated. 2.3 Performance of the instrument Generally, the four questionnaires are well-designed and appropriate instruments for evaluating the sustainability of CDTI programme at the different levels. However, the team noted the following possible areas of improvement: o There are repetitions of information required for the indicators within the same instrument; for instance, integration in planning, support activities as well as monitoring and supervision. Although this is necessary, it was sometimes irritating to respondents when they were asked the same question several times. lnterviewers should preferably avoid repeating questions if information for this had been collected earlier. o The guidelines for conducting the feedback and sustainability planning workshop are not clear and comprehensive. The template needs to be updated; for instance, there are no columns for the sources of funding. I , l0 I2.4 Limitations During the course of the evaluation exercise, the evaluation team encountered the following limitations: . The main medium of exchange in Ethiopia is Amharic. The external team members had problems in communicating directly with respondents whose abilities to speak in English was limited. This constrained and prolonged the process of data collection. On the other hand, most of the documents were also written in Amharic which made the verification process slow. . The erratic electricity supply slowed down data collation and entry. On some days, the team changed location and once had to travel to another town located one and half hours away. Time spent travelling was lost and the team had to adjust its working schedule to take advantage of the availability of electricity. . Bad roads and distances between communities meant longer working hours and more days for data collection. The stress and fatigue, as a result, sometimes impacted on the input and output of team members. . The coincidence of the evaluation exercise with the bazaarllrade fair in Pawi district, a social occasion which comes up once every two years, limited the number of community members and district officials the team was able to meet as most of them went for the trade fair. . The visit of a renowned political leader to Dangur on the day the evaluation team was visiting the district made it impossible for the advocacy visit with the Dangur District Administrator to take place. ln addition, some key district officials were unavoidably absent from their offices. o Due to the need for security and other briefings in Addis-Ababa prior to departure for the projects evaluation sites, the team arrived to the project area two days later than planned. The programme and appointments had to consequently be rescheduled. ll 2.5 Team composition Dr Johnson Ngorok S ightsavers I nternational Nairobi, Kenya E-mail : jngorok@sig htsavers. org Tel: +254722 567897 Dr Margaret K. Akogun Department of Psychiatry Jos University Teaching Hospital Jos, Nigeria Email: akoqunmk@vahoo. com Phone: +234 8039695047 Dr Tekola Endeshaw The Carter Centre Addis-Ababa, Ethiopia Email: tekol ca Tel: +251-1-91 1172856 Abraraw Tesfaye Organization for Social Science Research in Eastern and Southern Africa (OSSREA) Addiis-Ababa, Ethiopia E-mail : abrarawt@yahoo. com Tel: +251-91 1315196 Berhanu Melak The Carter Centre, Bahir-Dar Office, Ethiopia E-mail : Bmelak05@yahoo.com Tel : 058-220-55 1 49(office) ; 09- 1 8-77-0764(m obile) I Mr Tamrat Belete t2 tAfrican Programme for Onchocerciasis Control World Health Organization Addis-Ababa, Ethiopia E-mail: tamiratb@et. afro.who. i nt Tel: +251-91 1731248 13 3.0 FINDINGS AND RECOMMENDATIONS 3.1 Findings at the Zonal Level Metekel Project: Sustainability at Zonal Level 4.5 4 ^ 3.5s fJ ! 'fl z.s! tr2(!b 1.s 0.5 0 .*o'"" .C *"-':""..'" .'-:*.""t $$""* "/ .""-- lndicators g ru Planning (3.2: Highly) There was a Five Year Strategic Plan, 2006-2010, and yearly Operational Plans. Both plans contained CDTI activities under malaria and other vector borne disease control programme. ln addition, there was a detailed CDTI Activity PIan (in Gantt chart) which included all the key CDTI activities. The yearly plans did not vary from year to year and were not targeted to specific needs of each year. There was a Zonal Oncho Task Force which comprised the Head of Zonal Health Office, the Head of Zonal Drug Store, The Carter Center, the Zonal Oncho Control Coordinator and representatives of the zonal administration. The task force did not have structured and regular meetings but the team was able to see the minutes of al least one of the meetings that had been held. All partners were clear about their roles; both APOC and t l4 EA The Carter Center provided financial and technical support while the government was responsible for the overall implementation of the programme. Although there had been discussion on sustainability during annual planning meetings, no post APOC sustainability plan had been developed. However, the members of the zonal health team were aware that APOC was reducing its funding and that government was required to take over CDTI activities at the end of the fifth year of the project. lntegration (3.7: Highly) Oncho was planned and managed under the malaria and other vector borne diseases control programme. There was strong integration of CDTI activities with other health programmes. The zonal oncho coordinator combined a number of CDTI activities during supervisory visits to the districts. During their supervisory visits, the health staff of the various programmes carried check lists, including the CDTI check list and supervised several health programmes. The annual health plans integrated CDTI activities. However, there were no trip reports showing the actual integration of activities during supervisory visits. Recommendation : Planning lmplementation 1. The annualCDT| plan should target specific needs and problems. 2. Zonal oncho task force ought to hold at least two meetings, before and after drug distribution. Prtority: High lndicators of success:i) Annual CDTI plans are based on identified needs and problems. iD Semi-annualZOTF meeting minutes. Responsible person'. ZOTF members Time frame: For No. (1) May-June Annually For No. (2) December and May Annually l5 I Recommendation : lnteg ration lmplementation Trip reports on integrated supervisory visits should be written and filed. Priority: Medium lndicators of success: Written supervisory trip reports showing integrated activities. Responsible person: ZOC, members of zonal health team other Time frame: One week following completion of supervisory visit. Leadership (3.3: Highly) There is a focal person for oncho control programme who is also responsible for coordinating the activities of malaria and other vector borne diseases control programme. lt was evident that the Zonal Oncho Coordinator and the Head of the Health Desk were committed to CDTI and had taken full ownership of the programme. The Zonal Oncho Task Force, which also included other zonal sector leaders and is chaired by the Zonal Administrator, ensured that the zonal leadership was involved in overseeing CDTI activities. The members of the zonal task force interviewed were fully aware of the successes achieved and problems encountered during CDTI implementation and discussed these in detail with the evaluation team. Responsibilities for CDTI had been properly delegated to the district level, an observation that was further confirmed by district health team members. I Recommendation : Leadership lmplementation Strengthen the involvement and commitment of the zonal leadership to continue supporting CDTI Programme Priority: Medium lndicators of success: Minutes of ZOTF show the involvement of zonal leaders. Budget allocation for CDTI Responsible personi ZOC, members of zonal health team other Time frame: Dec and May Annually l6 Monitoring and Supervision (3.0: Highly) Supervision was integrated and the staff at the zonal level supervised those at the district level although they too occasionally supervised the FLHFs and the communities, especially so in cases where low coverage rates had been experienced. The distrid health team members were empowered to carry out CDTI activities, even though there was only one health staff assigned for oncho control and other vector borne diseases at the zonal level. This was mainly a result of TOTs conducted in collaboration with the NGDO partner and the supportive supervisory activities carried out by the zonal health team. Records including treatment summary sheets, inventory of equipment and financial records were filed. However, the records were sometimes hand written which compromised quality. The annual technical report, which was prepared by the NGDO partner, was not available at the zonal level. The technical report is of good quality. Superuision was integrated with other health programmes and was conducted on a quarterly basis. CDTI check list was used but supervision, which was intensified during drug distribution, was not targeted to specific needs. There were no reports of supervisory visits and it was therefore rather difficult to assess problem solving during supervisory visits and follow up action on recommendations from earlier visits. Efficiency in the use of resources for supervision was achieved mainly through integration. Recommendation: Supervision Monitoring and lmplementation 1. Supervision should target specific needs and problems identified mainly at the district level. 2. Zonal health team's supervision to the FLHF and community levels should take place only for "spot checks". 3. Project records should be typed and flled properly 4. Reports on supervisory visits, recommendations and follow up action should be written and filed. Priority: High lndicators of success:i) Reports on supervisory visits show evidence of targeting needs and problems ii) Good quality of project records. Responslble person: Zonal Health Team members Time frame: i) One week after each supervisory visit ii) Ongoing t7 Mectizan (3.8: Highly) Mectizan requirement was determined at zonal level based on updated census data from the previous year. A pharmacist from the zonal health desk collected mectizan@ from the FMoH using resources availed by the government and stored in the zonal drug store before collection by the district teams. Although there was sufficient supply of mectizan@ in the zone, there were incidents of temporary shortages in some districts which arose due to misallocation of the drug between districts. This was however managed promptly. The communities visited had decided on the timing for drug distribution to be from January to March based on the migratory pattern of some members of the communities, the farming season and the timing of major festivals such as Easter. However, the arrival and distribution of the drug took place rather late and extended into May and June in some communities. This was attributed to unavailability of the drug at FMOH stores at the required time. Overall, the procurement and distribution of mectizan is controlled within government drugs and supplies management system. Recommendation : Mectizan lmplementation 1. Due attention should be given to the allocation and delivery of mectizan to the districts based on updated and reliable census data. 2. The delay in the delivery of mectizan ought to be brought to the attention of FMOH. Priority: High lndicators of success: i) No reports of drug shortages in the districts. ii) Drug distribution taking place at the time decided by the communities. Responsible person: ZOC, FMOH, NGDO, APOC, Time frame: For No. (1) December Annually For No. (2) January to March Annually l8 Training and HSAM (3.3: Highly) The zonal oncho coordinator was trained as a trainer at the national level. ln turn, the zonal oncho coordinator, with the support of The Carter Center, conducted training for district health teams annually. The evaluation team observed that the district health team members were sufficiently empowered. The zonal health team was not involved in training staff at the levels below the district. However, training needs assessment to identify knowledge and skill gaps was not carried out and therefore training was not targeted. The trainings combined previously trained and newly recruited staff members without an initial basic training for the latter. ln order to contain costs, the zonal coordinator traveled to districts for the training which took place within the premises of the districts headquarters and lasted for not more than one day. The members of the zonal oncho task force were sensitized on the programme but this was carried out routinely without analyzing knowledge gaps. The zonal leaders were aware of CDTI programme and were supportive to the health team. Finance (2.3: Moderately) ln the five year strategic plan, CDTI activities were included and budgeted for. However, the one year operational plan did not specify the budget for oncho. lnstead, there was a budget allocated for malaria and other vector borne diseases which could be used for CDTI activities in line with the practice of integration. The zonal coordinator had estimates of funds pledged by partners for the current year: WHO-APOC, The Carter Center and the Government, but there was no projection of the estimated income for the following years. Recommendation: Training and HSAM lmplementation Target training and HSAM activities based on knowledge and skillgaps. Priority: High lndicators of success: Targeted training and HSAM carried out. Responsible person: Zonal Health Team, ZOTF Time frame'. As need arises. t9 Budgeting is carried out at the zonal cabinet level which includes the Head of the Health Desk. The budgetary contributions of the government, the NGDO and WHO-APOC were clearly spelt out for 2009 and the previous years. According to the Zonal coordinator, government funding had been increasing but we could not verify this assertion. Because of the integrated nature of budgeting, we could neither isolate government disbursement for oncho activities nor verify whether the disbursement for oncho was increasing or not. Project management was aware of budgetary deficit but this had not been quantified. The deficit was however managed using integrated funds. The Carter Centre confirmed, through telephone interview with the Country Representative, that they would continue supporting the programme for the next three years, but there was no written agreement to confirm this commitment. The control system for expenditure against budget was adequate; requisition was made against budgeted activities, approval was obtained and residual balances were calculated by finance department and reports availed to the zonal oncho coordinator. Transport and Other Materials (2.5: Moderately) There is one Toyota 4 WD pick up provided by APOC which, although quite old, was still functional. Additionally, there was a Toyota Land Cruizer for the Zonal Health Desk Recommendation : Finance lmplementation 1. Within integrated budgeting' funds committed for oncho should be clearly spelt out. 2. Expenditure analysis ought to show clearly the expenditure on oncho activities. 3. Budgetary deficit should be analysed and quantified; possible sources of bridging the gaps indicated. 4. A written commitment of support from The Carter Centre should be obtained. Priority: High lndicators of success: i) Oncho specific budget and expenditures shown. ii) Clearly indicated budget deficit with mechanism of bridging the gap. iii) A written agreement with NGDO partner. Responsible person: ZOTF, NGDO, WHO/APOC Time frame: By July Annually 20 Iprovided by government which is available for use by all health programmes including CDTI activities. The following equipment were also available: desk top computer, printer, photocopier, fax machine, LCD projector, television, generator, radio communication device and training/HSAM materials. Apart from the photocopier, all others were still functional though old and needed replacement. Maintenance of vehicles and equipment was integrated but there was no scheduled routine maintenance. Travel authorization was routinely obtained but the log books were not filled. Management was aware of the need for replacement of vehicles but this had not been planned. The head of the zonal health desk indicated that there is a possibili$ of replacing equipment but not vehicles; as there is integration, vehicles from other programmes can be used for CDTI activities. Human Resources (3.0: Highly) The zonal oncho coordinator was the only staff at this level who was actively involved in CDTI activities. The coordinator was knowledgeable and skilled in the key areas of CDTI programme but seemed to have a heavy work load. We noted the strong commitment of the coordinator which was also confirmed by the district health teams. Recommendation: materials Transport and lmplementation 1. APOC to replace the current vehicles and equipment. 2. lnclude replacement of equipment in the next strategic plan, 2011- 2015. 3. Ensure availability of reliable means of transport for oncho programme. Priority: Medium lndicators of success: i) Vehicles and equipment replaced. ii) The strategic plan includes future replacement of equipment. iii) Transport availability for oncho programme. Responsrble person'. WHO/APOC, FMOH ZOTF, Time frame'. APOC: December 2009. ZOTFIFMOH: June 2015 2t However, the coordinator would need capacity building in computer skills and data management. The coordinator had been in post for over five years and there had been no new health staff for the programme during this period. Coverage (4.0: Fully) The GCR was 100% in 2007 and 2008. ln 2006 the GCR was 96%; this was attributed to the intergroup conflicts which hindered treatment in some of the communities in one of the four oncho endemic districts (Mandura district). The TCRs achieved during the preceding three years were: 66.9% (2006), 71o/o (2007) and7O.5% (2008) giving an average TCR of 69.5% for the three years Recommendation : Finance lmplementation 1. Training and active involvement of other members of zonal team in CDTI programme. 2. Training the zonal oncho coordinator in computer skills and data management. Priority: Medium lndicators of success: i) Active involvement of other members of the zonal health team in CDTI. ii) lmproved computer and data management skills. Responsib le person: ZOTF, NGDO Timeframe: By December 2009 Recommendation : Finance lmplementation Maintain the TCR coverage and aim to achieve even a higher TCR. Priority: Medium lndicators of success: lncreased TCR. Responsib I e p erso n: ZOf F Timeframe: By May 2010 22 t 4.5 4 ^ 2trs "'"E3 C'ITt 2.5 =o2E"$,u 0.5 0 Sustainability at District Level lndicators .*o"" .u.."-" ".,. "".." """d .o"^.*'"--."' "/ :."'C ^-'"" ^-C .o.,"" 3.2 Findings at the District Level Planning (3.3)-Highly There was a five year strategic plan from 2006 up to 2010 for overall health programmes. Onchocerciasis control was included in the five year plan with a specffic line funded by APOC. There were yearly health plans which included malaria and other vector borne diseases. Onchocerciasis control programme was part of the yearly plan. There were also detailed annual activity plans on Gantt Charts posted on the wall and a comprehensive plan showing 15 CDTI activities. The plans were drawn by the District Health Team in consultation with the zonal onchocerciasis control programme coordinator. Although yearly plans considered the community migratory patterns (migration due to search for seasonal work in other areas and pastoral community), availability of mectizan@ at the required time remained a constraint even after the districts had informed the zonal health desk on the appropriate distribution time. 23 Integration (3.8)-H ighly There was an integrated annual operational health plan which included CDTI activities and other vector borne diseases such as Malaria and other health programmes like EPl. CDTI activities such as monitoring, supervision, HSAM were combined during trips. Reports also indicated that CDTI activities were combined with those of malaria control, EPI and health extension services. However, the evaluation team did not see any trip reports to confirm this. Leadership (3.5)-H ighly There were focal persons for CDTI activities in all district health offices who were also responsible for control of malaria and other vector borne diseases. All district health teams visited, did initiate key CDTI activities such as planning, monitoring, supervision and training in consultation with the zonal onchocerciasis coordinator. The district health offices remarked that they were well aware of the temporal support of APOC and they would find ways of continuing with the CDTI activities after APOC withdraws. Monitoring and Supervision (3.2)-Highly ln most cases, the district health teams routinely supervised the FLHF staff on onchocerciasis activities, while in some cases; supervision was reported to be targeted in certain situations. lt was particularly noted that communities that were near the district headquarters, the district health team worked alongside the FLHF staff to supervise drug distribution. This was seen as necessary for spot checking and problem solving. ln situations where the communities were remotely located and difficult to reach, the FLHF staff were mainly involved in spot checking. There was onsite management of problems when they arose. For example, shortage of mectizan@ in some communities was tackled by transferring from other communities Recom m endations : lnteg ration lmplementation 1. Documentation should be made of every activity carried out. 2. Trip reports should be documented Prioity: HIGH lndicators of Success: o Documentation of all reports Who to take action: Distict staff Deadline for completion : Continuous 24 which had excess mectizan@. Shortages were experienced due to migratory pattems. ln such cases, reports were made to the districts who further contacted the zonal level for additional supplies. This difference in the style of supervision can be explained by distances and accessibility of the communities. The district health teams, with the involvement of the FLHF statf, participated in solving community problems. This included discussion of problems with CDDs, household heads, community leaders and community supervisors. The district teams worked jointly with the FLHF staff in the implementation of CDTI activities at the community level. A system of publicly recognizing and rewarding the best performing health workers was reported at the district level. This applied generally for all health workers wtto successfully carried out CDTI activities. No evidence was found regarding follow up actions to implement recommendations of the previous monitoring visits. What was clear though was that problems were solved on the spot during supervisory visits and completed checklists were filed as a reference. The channels of reporting were found to be within government system. lt was reported that all health activities were sent on quarterly basis. However, CDTI reports were sent to the zonal health desk and onwards to the national level immediately after completion of Mectizan distribution. Recommendations: Monitoring and Supervision !mplementation 1. Supervisory activities should be targeted and based on identified needs 2. Feedbacks should be constantly given to the lower level Pioity: HIGH lndicators of Success: . Practice of targeted supervision o Evidence of written feedback Who to take action: District staff Deadline for completion : Conti n uous 25 Mectizan (3.5)-Highly Mectizan@ ordering, storage and delivery was controlled within the government system. The ordering system involved mectizan@ collection from the FMoH and delivery to the zonal health desk for onward distribution to the districts and FLHFs. ln general, the system appeared simple, effective and efficient. The mectizan@ order is based on previous years' census update and quantification parameter. Nevertheless, the evaluation team did not find any mectizan@ order form at the district level. ln most cases, allocated mectizan@ did not reach the districts drug store for onward delivery to the communities at the time requested. Most communities' preferred distribution to take place from January to March. However, mectizan@ distribution had been taking place between March to May. Problems of mectizan@ misallocation were reported. This resulted in shortages of mectizan@ in some districts during distributions. ln such cases, the zonal onchocerciasis coordinator was contacted who promptly delivered mectizan@ retrieved from other districts which had received excess. The problem of shortage was therefore internalto the zone. Training and HSAM (3.2)-Highly Trainings were carried out routinely for both new and previously trained personnel i.e. they were not targeted. The district health teams conducted training for FLHF staff in a central location within the district headquarters. FLHF staff in turn conducted training for CDDs and community supervisors in batches when they came to collect drugs. Recommendations: Mectizan Ordering and Supply lmplementation 1. Standard Mectizan order forms should be in place 2. Mectizan should arrive on time at the community level 3. Accurate allocation of Mectizan should be carried out Pioitv: HIGH lndicators of Success: . Order forms in place o Mectizan being distributed on time . Absence of shortages Who to take action: District staff Deadline for completion: During planning and distribution 26 aThe evaluation team was able to see lists of participants and letters of invitation for FLHF staff to report for training. The trainings were carried out with the assistance of the zonal onchocerciasis coordinator in collaboration with The Carter Centre. Resources for training were used efficiently; training lasted for one day and participants returned on the same day; training sites in the district headquarters were used to save on rental costs. However, the number of participants was not regulated which seemed inefflcient. HSAM activities were also reported to be carried out routinely for district leadership on annual basis. Political leadership was involved in social mobilization in community gatherings such as religious, schools and political rallies etc. ln Pawi district there were reports of very high turnover of political leaders which necessitated yearly social mobilization. HSAM activities were planned as evidenced by annual health plans including onchocerciasis activities but no schedule of HSAM activities made. HSAM was mainly carried out at the district level but extended to lower level in Glses where serious problems, such as lower coverage or refusals, were identified but this was done in collaboration with FLHF staff. There was evidence that people at different levels were cooperative. Community's demand for Mectizan@ was found to be high and there was increased covelage. District leaders were also involved in health education and social mobilization. We did see a letter from the district health office requesting the district administrator to wrtte to the Kebele (community/group of communities) administration to support CDTI activities. Recommendations: Training and HSAM lmplementation 1. Training and HSAM should be targeted and based on needs 2. Detailed HSAM schedule should be readily available Pioity:MEDIUM lndicators of Success; o Targeted evidence of training and HSAM activities Who to take action: Distict staff Deadline for completion: Duing planning and preparation phase 27 Finance (2.8)-Moderately There was a specific budget for CDTI activities funded by the three partners i.e. APOC, TCC and the Government (evidence was presented to the evaluation team) for the period 2006 - 2009; but this did not spell out the detailed specific activities. ln addition, in their annual health plan, there was a budget for malaria and other vector borne diseases control programme which was sometimes used for CDTI activities as well. lt was however difficult to separate out the specific budget for CDTI activities because of the integrated nature of planning and budgeting. Besides, the staff at this level did not seem to have a clear estimate of expected funds for the coming year and were not therefore actively using the available information for planning. The main cost containment strategy at this level was integration of various health programmes and conducti ng supervision. The government contribution in the CDTI specific budget showed an increase although there was a decline in 2008. lt was however difficult to assess whether government contribution for CDTI activities in the overall health budget was increasing because of the integrated nature of budgeted activities. ln some districts, disbursement information was obtained and did show that funds disbursed by government were increasing each year. However, in one of the districts, the evaluation team could not verify the amount of funds disbursed for CDTI activities and also whether or not the contributions were increasing yearly. The process of fund disbursement follows the following procedures: . lnitiation of payment request memos from any one of the health staff for authorization as per the action plan . Endorsement of the request by the Head of Health Office and flnally . Forwarding the request for fund disbursement to the Head of Finance Department in the pool system Evidence of such a request was seen. There was regular calculation of residual balances. District Finance Office prepared reports on year to date expenditure against budget and presented this to the health office. The evaluation team examined this repofi. o 28 Recommendations: Finance lmplementation 1. There should be clear estimate of expected income at the district level 2. Estimated income should be based on quantifled gaps 3. District management team should actively look for other sources of funding Piority: HIGH lndicators of Success: o Clearly estimated income in place r Funding gaps are quantified based on available sources . Funds from other sources are clearly identifled Who to take action: Distict staff Deadline for completion : Conti nuous Transport and Other Material (2.7)-Moderately All APOC donated motorbikes were not functioning. However, there were several other motorcycles and vehicles from other health programmes which were available to be used for CDTI activities. There were adequate training materials which were printed wlth support from several partners; FMoH, The Carter Center, WHO/APOC and Lions Club lnternational. Maintenance of vehicles was carried out with government funds. However, constraints arising due to inadequate government funds were sometimes experienced. Request of funds for vehicle maintenance is made by the health team through the finance office. Whenever there were serious transportation problems, staff members used public transport and sometimes walked on foot to carry out CDTI activities. It was reported that a system of authorization for use of vehicles was in place. However, the approval was sometimes not formal. ln the use of motorbikes, verbal approvals were practiced. Log books were not filled. The team observed that the vehicles were old and maintenance was becoming costly. Even though the management at the districts level was aware of the need to replace the motorbikes and other vehicles, there were no specific plans from the government. During advocacy visits, the district administrators pledged to include provision for replacement of the motorbikes in the next strategic plans after consulting with the regional administration. The immediate solution was however seen in the integrated use of available transport. 29 Recommendations: Transport and Other Materials !mplementation 1. There should be plans for replacement and repair of vehicles 2. Log books for movement of vehicles should be properly filled Pioitv: HIGH lndicators of Success: o Availability of written commitment for replacement of vehicles o Availability of maintenance schedule for transports . Properly filled vehicle movement log books Who to take action: Distict staff Deadline for completion : Continuous Human Resources (3.8)-High !y There is evidence of staff stability. Most of the staff in the district had been in post for periods ranging from 4 - 7 years and had undergone at least one form of in-service training on CDTI activities and other health programmes. The health staff interviewed at the districts expressed satisfaction with their work; they were highly motivated and expressed great commitment to continue with CDTI activities. ln the previous year, all health personnelwere awarded a certificate of appreciation and five of them were in addition received special awards. Rewards varied across districts and included; certificates, monetary rewards, public recognition etc. This was a big motivation for health staff. Salaries were reportedly paid regularly but per diems were paid when funds were available. Additionally, the alleviation of the suffering of the communities they serve was reported as a source of motivation. Coverage (a.0)-Fully Geographic coverage rates of 100% were achieved in each of the three preceding years (2006 - 2OO8) and therapeutic coverage rates achieved were all above the 65% minimum recommended by APOC. Recommendations: Coverage lmplementation 1. Staff should be encouraged to keep up with high coverage level Pioity:LOW lndicators of Success; High coverage wha to take action: Distict staff Deadline for completion : Continuous 30 !3.3 Findings at the FLHF Level Sustainability at FLHF 4.5 4 \t .9 o =ogl G o 3.5 2 3 .5 2 1.5 1 0.5 0 .*So*ndt,*o'*'o""*.d*""'€:"ood"do.o*o'*."Y./ n..".-'- ^."'" ^."" aT Planning (2.8)-Moderately There were no specific written CDTI plans in all the six FLHFS. However, there was a written annual plan for all health programmes in five of the six FLHFs visited and CDTI activities were "presumably" integrated in one of them. Note of mention is the fact that plan of CDTI activities were normally drawn at the district level and not at the FLHF but distributed to the FLHF to fit in their schedules. Recommendation : Planninq lmplementation FLHF statf should be trained and empower to make their own plan of activity Pioitv: Hish lndicators of success: CDTI plan in place at FLHF level Who to take action: District and FLHF staff Deadline for completion: At the beginning of each fiscal year. 3l lndicators lnteg ration (3.7)-Highly ln all the FLHFs visited, the staff combined several CDTI activities in a single trip. Similarly, CDTI activities were also combined with activities of other health programmes such as malaria, EPl, TB control and MCH. However, for communities which are far away, the health extension workers were reported to combine CDTI activities with 17 health packages as part of integrated programme activities in the communities. ln some instances, CDTI activities were carried out separately for communities close to health centres. There was no written evidence of integrated activities for verification. Recommendations: Integration lmplementation 1. FLHF staff should be encouraged to maintain integration of activities. 2. Written evidence of integration should be in place and FLHF statf have to be trained Pioity: HIGH lndicators of Success: o lntegrated activities conducted o Written evidence of integration in place Who to take action; Both District and FLHF statf Deadline for com pletion: Continuous Leadership (3.0)-High !y For the majority of the FLHFs visited, there was strong ownership and the health staff saw the CDTI programme as their own. However, in one health facility, there was uncertainty of the roles of FLHF staff and considered the CDTI activities are mainly the responsibility of the district health office and he saw the FLHF staff as playing a supportive role. The local leaders were involved in carrying out some form of social mobilization in conjunction with the FLHF statf in support of CDTI and other health programmes in the communities. Monitoring and Supervision (2.8)-Moderately The FLHF staff integrated several supervisory activities with other health programmes such as HIV/AIDS, malaria control, TB etc during community visits. This was especially so with the health extension workers whose routine work involved frequent interaction with the community members. However, there were no check lists for verification. Supervision was reported to be carried out routinely by the FLHF staff most of the time and intensively at the time of drug distribution in all communities because some of the I, 32 !CDDs and supervisors were not educated enough to perform and record adequately. Targeted supervision was reported only in a few cases when there were problems especially with remote communities and when there was scarcity of vehicles. Some of the problems identified were dealt with by the FLHF staff on the spot by providing information whenever possible. Examples of such problems included registers not being up to date, duplication of registration and refusal of treatment. There were no reports of other community problems which the FLHF staff passed on to the community leaders for them to deal with. lt appeared that the FLHF staff attempted to solve all the problems themselves. Problems and recommendation for further follow up were not documented and therefore verification was rather difficult by evaluation team Data from community registers and APOC reporting forms indicted the name of the household members, the amount of drugs family members received, therapeutic coverage, number of community supervisors and CDDs trained. Completed register books were sometimes submitted by the CDDs through the community supervisors after drug distribution. ln some cases, the CDDs submitted directly to FLHF staff. Community supervisors where sometimes reported to fill the summary forms and submitted them to the health facility staff in other instances the summary reports were filled by the FLHF staff who in turn further compiled for their catchments areas and submitted to the district office. The reporting is carried out within the health system in an integrated manner. The FLHF acknowledged the important services provided by the CDDs to their communities, but expressed disappointment over the fact that CDDs services were acknowledged only by encouragement by community members. Despite the situation the CDDs were willing to continue serving the communities. Recommendations: Supervision Monitoring and lmplementation 1. FLHF staff should be encouraged to use and document checklists of activities 2. Supervision should be targeted 3. FLHF should empower community leaders to solve community problems CDTI related activities 4. A system of recognition of CDDs should be in place Pioity: HIGH lndicators of Success: o Documentation of activities . Targeted supervision . Empowered community . CDDs recognized Who to take action; FLHF staff D eadli ne for com pletion; Conti nuous JJ Mectizan@ (3.5)-H ighly Reports show that FLHF staff were not used to estimating Mectizan@ requirement for their communities. Therefore there were no order forms seen at this level. Mectizan ordering was made by the district health team based on community updated census reports of the previous year. Reports also showed that in the past two years, Mectizan was not made available at the time requested by the communities i.e. between December to March (the time after harvest when most people are around). lnstead, the drug was received in March/April well into the farming season and when some communities migrated in search of work or pasture. This was true especially for communities settled in Pawi district. Temporal shortages of mectizan@ was reported in the six FLHF visited in the 2008 distribution due to misallocation of mectizan@ to districts. This problem was reported to the district health team and zonal level and additional drugs were promptly mobilized from other districts. The method of mectizan@ delivery was controlled within the Government system and appeared simple and effective. The system involved the FLHF staff collecting the drug from the district using any available means of transportation and delivering it to the CDDs directly or through the community supervisors or health extension workers. Training and HSAM (3.2)-Highly Training of CDDs and community supervisors takes place routinely every year before drug distribution and is not targeted to identified knowledge or skills gaps. Recom mendations : Mectizan lmplementation 1. FLHF staff should be involved in estimation of Mectizan supply 2. Timely supply of Mectizan as agreed by the community 3. Accurate allocation of Mectizan to communities based on communities requisition Pioitv: MEDIUM lndicators of Success; o FLHF staff involvement Mectizan supply estimation . Timely arrival of Mectizan to the community according to community request . No shortage at community level Who to take action; FLHF staff Deadline for complefion: During planning and distribution time 34 Efficiency in training was observed. Training sites were within the community, the training lasts less than half a day and did not have a monetary cost to the programme. The evaluation team observed that there were adequate training materials. HSAM activities were carried out routinely at the health facility during community meetings, in schools and during community visits by health extension workers. There was no indication that HSAM activities being carried were addressing information gaps or based on objective identification of needs among decision makers. Effectiveness of HSAM activities was evidenced by the increasing demand for mectizan@ and improvement in the coverage rate. No training records or checklist of activities were available for verification. Community leaders were not actively involved in HSAM activities Recommendations: Training and HSAM lmplementation 1. Training and HSAM activities should be targeted according to identified knowledge gaps 2. Training and HSAM activities should be recorded and properly documented 3. FLHF should encourage community leaders involvement in HSAM Pioitv:HIGH lndicators of Success; o Training and HSAM activities targeted and recorded . Community leaders involve in HSAM Who to take action; FLHF staff Deadli ne for com pletion: Conti nuous Finance (2.2)-Moderately There was a one year health budget for all health activities at the health centres to cover operational cost but none for the health posts visited; this was because there was no system of budget allocation at this level. However, the health centres budget did not specify CDTI activities. As there was no specific budget for CDTI activities in the health post, there was equally no disbursement of funds. The cost of carrying out CDTI activities was minimal and was absorbed in other programmes. Sometimes the use of personal funds was reported by the FLHF staff while general funding was being awaited. 35 CDTI activities were integrated and canied out as part of overall health programmes by the FLHF staff. For example, CDD training lasted for half a day and did not have any monetary cost; supervision was carried with no per diem payment. However, when performing other programme activities e.g. Malaria control, HIV/AIDS control and EPI in which CDTI activities were integrated, per diem was paid. Recommendations: Finance lmplementation '1. FLHF should prepare their own annual plans for specific CDTI activities and have clear estimates of CDTI activities. Piority: MEDIUM lndicators of Success; o Availability of budgeted activities Who to take action: FLHF staff Deadline for completion: Every year Transport and Other Material 2.3)-Moderately The motorbike provided by APOC for CDTI activities was deployed at the health centers but at the moment not functional. There was also one motorcar and four other motorbikes provided by government for other health programmes in the six health facilities. These were also used for CDTI activities. The transports available were partially maintained with the Government sources, but no plan of replacement was yet in place by the Government. Motorcycles were maintained within the district vehicle maintenance system. When vehicles broke down, the FLHF staff requested the district finance department to facilitate maintenance through the district health office which approves the request. Costs of maintenance were met from funds provided by government and other programmes. Training and HSAM materials were seen and reported to be enough. Any available means of transport, including walking on foot, were used as coping mechanisms whenever motorbikes were broken down. The vehicles in the health facilities were pooled, used in an integrated way between programmes and managed under the control of the Heads of the health facilities. ln most cases, trip authorizations were provided verbally and there were no log books at all. No maintenance schedules were in place at the FLFH but all repairs and maintenance costs were covered by Government funds. 36 tAccording to The Carter Centre, their current commitment to Onchocerciasis programme extends for another three years beyond which no plans are currently in place. During these three years, their financial input in likely to be in the range of US$10-12,000 annually. This willfinance only recurrent expenditure and not capital equipment. Recommendations : Transport and Other Materials lmplementation 1. Written trip authorization and filling of log book should be practiced 2. Regular system of vehicles maintenance schedule should be in place Pioitv:HIGH lndicators of Success; . Documented trip authorization and properly filled log books o Vehicle are in good condition Who to take action: Distict and FLHF staff D eadl i ne fo r co m p I etio n L Q e!!!!! o! s Human Resources (3. 5)-Highly The staff in most of the FLHF was stable except for a few cases. They were reported to have been working at the same health facility for more than two years. Where there were transfers, this was internal within the district and the transferred staff had been trained in CDTI activities. All health staff had attended in-service training for at least two to three times. ln some health posts, staff seemed to lack skill and self-confidence in carrying out CDTI activities. Coverage (a.0)-Fully Geographic coverage rates achieved was l}OYo for three years (2006 - 2008) and therapeutic coverage rates were all above the 65% APOC recommendation. Recommendations: Human Resources lmplementation 1. New staffs should be adequately trained Pioitv: MEDIUM lndicators of Success; Availability of skilled staff at all health facilities Who to take action: Distict staff Deadline for completion : Continuous Recommendations : Coverage lmplementation 1. Staff should be encouraged to keep up with high coverage level Pioity: MEDIUM lndicators of Success: High coverage Who to take action: FLHF staff and CDDs Deadline for completion : Conti nuous 37 3.4 Findings at the Community Leve! Sustainability at Community Level rl ; E,r g o E' b 4 3.5 3 2.5 2 1.5 1 0.5 o *""'""""- u..n'" o,C .d ..""" d- "-r "d lndicators 0 Planning and Management (3.3: Highly) CDDs decided on the visiting times for drug distribution (mornings and evenings), in consultation with other community members, based on the availability of the community members. The drug distribution often started at a central location but later on the CDDs carried out house to house distribution whenever the turn up to the central location was lower. This appeared to have increased the work load of CDDs although they did not complain. Besides drug distribution, CDDs also carried out the initial and subsequent annual census updates and health education activities. ln most cases, there were no specific problems reported except during the flrst two years of the programme when there were refusals as a result of severe side effects. The CDDs mainly worked with FLHF workers to solve this problem. There was little evidence of community leadership helping CDDS to solve problems. 38 Recommendation : Planning and Management lmplementation 1. Scale up the mobilization of the communities to gather at central locations for drug distribution. 2. CDDs need to work more with community leaders to solve problems Priortty: Medium lndicators of success:i) Reduced house to house visits made by CDDs. ii) Community leaders more involved in problem solving. Responsible person. FLHF staff, Community supervisors, Community leaders, CDDs Time frame: By December 2009 Leadership and Ownership (2.7: Moderately) The community leaders were not sufficiently involved in mobilizing community members for meetings and for drug distribution. As a result, the turn up for meetings and drug distribution was reportedly low in some communities. lt was also reported that community leaders, in conjunction with FLHF staff, were involved in the selection of CDDs in some communities. This was not in line with the CDTI approach and may have contributed to the relatively low community ownership. There was low therapeutic coverage in the early years of the programme but since then, community demand for the drug increased as its benefits became apparent resulting in improved coverage rates. ln a few cases, the CDDs went back and carried out hous+to house distribution when coverage rates were low. However, the involvement of community leaders was found to be inadequate in most of the communities visited. The community members did not provide incentives, either material or financial, to CDDs except for appreciation and encouragement. The CDDS did not demand for incentives and were happy to continue volunteering their services to their communities. Members of the community were not involved in self monitoring of CDTI activities though in some cases community leaders oversaw drug distribution. Community ownership of the programme was generally low. 39 Recommendation: Planning and Management lmplementation 1. Selection of CDDs should be carried out with the full involvement of community members. 2. There is need for sensitizing community leaders to take an active role in CDTI. 3. Strengthen community ownership of the programme Priority: Medium lndicators of success:i) New CDDs selected by community members. ii) Community leaders more involved in CDTI iii) Communities involved in self monitorinq activities and supporting CDDs Responsible person: FLHF staff, Community supervisors, Community leaders Time frame: December 2009 to June 2010. Community members interviewed could name the main benefits of taking mectizan@ and were sufficiently motivated to continue taking the drug for a long time. Monitoring (3.3: Highly) The activities of CDDs were supervised by the FLHF staff. ln most cases, CDDs directly submitted the registration books to the FLHF staff who compiled summary reports for the communities in their catchment areas. ln a few cases, the CDDs submitted the registration books to community supervisors to fill summary forms and submit to the FLHF staff depending on the level of literacy of the CDD. Regarding transport for the delivery of reports to the FLHF, the CDDs and community supervisors in most cases arranged their own means of transport. Recommendation: Planning and Manaqement lmplementation While selecting CDDs, literacy status should be considered. Priority: Medium lndicators of success: Literate CDDs selected. Responstble person. FLHF staff, Community supervisors, Community leaders Time frame: November to December annually. 40 Mectizan (3.5: Highly) There was adequate mectizan@ available for all eligible members of the communities even though delays were occasionally experienced. All communities obtained mectizan@ including those remotely located. ln the case of absentees and temporarily non-eligible persons, some mectizan was kept for them at the FLHF store for about one month. Most CDDs' records showed accurate census data which was good enough for calculation of the amount of mectizan@ needed. ln most cases, CDDs or community supervisors collected mectizan@ from the FLHF even for remote communities, but in a few cases the FLHF staff delivered the drug to the CDDs during their visits to the communities for other health programmes. Generally, CDDs and community supervisors walked to collect mectizan@ from the FLHF, and sometime did so for quite long distances i.e. more than 10 km, but were apparently happy to continue doing so. HSAM (2.3: Moderately) HSAM was carried out routinely before and during mass drug distribution but did not necessarily address the information gaps. Most of the time, HSAM was carried out by CDDs during house to house visits when community members were available in their homes. However, sometimes it was carried out by the FLHF staff. There was minimal involvement of community leaders in mobilizing and sensitizing community members. HSAM tended to provide information to the communities and encourage them to continue taking mectizan@. However, little attention was given to the contribution of resources to support the work of CDDs. Recommendation : Mectizan lmplementation The district or FLHF should make some arrangement for the delivery of mectizan to remote communities Priority: High lndicators of success: Mectizan is delivered to remote communities by appropriate means Responsible person'. District and FLHF statf, Community leaders Time frame: December to January Annually. 4t There was a fair level of community ownership of the programme as expressed by the CDDs and community supervisors' willingness to continue collecting and distributing Mectizan. However, this was not matched with tangible community support for the work of the CDDs. Finance (2.0: Moderately) Community members did not provide financial or material support to CDDs although they did provide moral support. Nevertheless CDDs expressed willingness to continue serving their communities as long as mectizan@ was available. Although there was no contribution of supplies (books, pens etc) by the communities, no gaps were experienced. Registration books were provided by the national programme while other stationery was not seen as an issue by the CDDs ln some communities, there was a misconception among the community members that CDDs were working for the government and were receiving some remuneration. This observation was surprising for a project in its fifth year and points to a gap in HSAM activities. lt may be attributed to the practice of paying volunteers by other programmes such as polio eradication campaign, EPI etc. Recommendation: HSAM lmplementation 1. Encourage community leaders to be actively involved in mobilization and sensitization activities. 2. HSAM should address the issue of contribution of resources by the communities to support CDDs Priority: Medium lndicators of success: i) Active involvement of community leaders in HSAM. ii) Resources contributed by the communities to support CDDs Responsible person: FLHF staff, Community supervisors, Community leaders Time frame: November to December annuallv. 42 Recommendation: HSAM lmplementation Sensitize the community members to correct the misconception that government is remunerating CDDs. Prtority: Medium lndicators of success: No community member believes that government remunerates CDD Responsrb le person: FLHF, community leaders Time frame: As soon as possible Human Resource (3.7: Highly) The CDD to population rations ranged between 1 CDD to 94 persons to 1 CDD to 280 persons. The households which the CDDs were expected to cover were within walking distances in many cases. ln a few cases, they travelled a distance that took up to 40 minutes. CDDs were knowledgeable and skilled in the key tasks they were expected to perform. There were a few cases of CDD drop-out; and when they dropped out, new ones were recruited and trained. All CDDs interviewed expressed willingness to continue with the distribution of mectizan@ as long as the drug was available. They were motivated by the benefits that the drug had brought to their communities. Coverage (3.0: Highly) The average therapeutic coverage rates for the twelve communities visited were; 64.3% for 2008, and 64.50/o for 2007.ln both years, the average TCR was below the cut-off point of 65%. This was attributed to: migratory nature of the population; long distance travelled by some CDDs and the timing of distribution which coincided with farming seasons and festivals; Recommendation : H uman Resources lmplementation Train more CDDs to increase the CDD to population ratio in line with the national target Priority: Medium lndicators of success: lncreased CDD to population ration Responsib le person: FLHF Time frame: December 2010 43 ln one of the communities where indigenous people live, CDDs were not competent enough to carry out CDTI activities in terms of disseminating information and mobilizing the community members. ln addition, census updates did not take account of people who had permanently migrated from their communities. 4.0 CONCLUSION: GRADING OF THE OVERALL SUSTAINABILITY OF THE PROJECT To reach a judgement on the overall sustainability of the Metekel Zone CDTI Project, the evaluation team considered both qualitative and quantitative methods. a) Qualitative method; the team considered the "Aspects" and the "critical elements" of sustainability as outlined in the Evaluation Guidelines. On "Aspects", the extent to which the findings were helping, partially helping, blocking or partially blocking the sustainability of the project were considered. Accordingly, the following conclusions were made on each "Aspect". Recommendation : Human Resources lmplementation HSAM sessions should address the constraints to the achievement of higher TCR observed in affected communities Priority: High lndicators of success: lncreased level of awareness and TCR Responsib I e p erso n: FLHF, community leaders Time frame June 2010 44 Aspect Judgement based on whether the findings wene helping, partiatty helping, blocking, partially blocking sustainability lntegration Helping Resources Helping Efficiency Partially helping Simplicity Helping Health staff acceptance (attitude of staff) Helping Community ownership Partially helping Effectiveness Helping lntegration : Helping Sustainability There is strong integration of CDTI activities into the overall health programmes. Oncho control activities were included in both the annual operational and the five year strategic plans; transport and materials were shared across different programmes and health staff carried out support supervision in an integrated manner. Resou rces: Helping Sustainability The integrated use of vehicles, equipment and other materials provide a good basis for sustainability of the programme. Although government direct funding commitment for oncho was not clearly spelt out, funds from government and other sources were made available for oncho control when activities from different programmes were conducted in an integrated manner. The commitment of the NGDO partner to continue providing financial support reinforces the sustainability of the project Efficiency: Partially Helping Efficiency of the use of resources was achieved by integration, conducting training sessions in cost free venues and limiting the duration of trainings. However, supervision, trainings and HSAM activities were not targeted; they were carried out routinely every year. These potentially waste limited resources. 45 Simplicity: Helping Simplicity was demonstrated in several ways such as; delivery of mectizan from FMOH to the community level using existing structures and procedures, delivery of reports by CDDs to the FLHF and onwards as part of routine activities and without using extra resources; trainings being carried in premises owned either by the government or the community. Attitude of staff: Helping The health staff and the CDD expressed great interest and willingness to continue carrying out CDTI activities after APOC project phases out. The health staff considered CDTI part of their routine duties. For the CDDs, this was despite the fact that they were not receiving financial or material support from the community. Community Ownership: Partially Helping Although the involvement of CDDs and community supervisors was commendable and promoted sustainability, the community leadership and the rest of the community members did not show active involvement and ownership of the programme. Effectiveness : Helping Full geographical coverage and therapeutic coverage rate of over 65% were achieved during the past three years. Regarding the"critical elements", the judgement was limited to "Yes" and "No" depending on whether these elements were sufficiently present to ensure sustainability. The evaluation team made the following judgement: Criticalelement Judgement Money Yes Transport Yes Superuision Yes Mectizan@ supply Yes Political commitment Yes 46 Money: Yes Committed funds were disbursed by partners, including government, and utilised properly even though funding constraints were experienced. Transport: Yes The Toyota 4WD pick up provided by APOC was functional but two of the motorcycles in the districts visited were non functional. However, the pool system of vehicles ensured that alternative means of transport was available for the programme' Supervision: Yes Supervision was regularly conducted although it was not targeted to address identifled problems. This was intensified during drug distribution to provide effective and prompt support to FLHF staff and CDDs. Mectizan@ supply: Yes The existing government system was effective in delivering mectizan to the FLHF. For most communities, the CDDs or community supervisors collected the drug from the FLHF. Only in a few cases did the FLHF staff deliver the drug to the communities during their visits for other health programme activities. For a few remote communities however, some difficulties were experienced in delivering mectizan. Political commitment: Yes The participation of the Zonal Administrator in the feedback and sustainability planning workshop was an evidence of good political support at the highest level in the zone. The enthusiasm and commitment he showed during the workshop was reassuring. This was also demonstrated by the District Administrator of Pawi during the advocacy visit. However, the political leaders at the community level were neither sufficiently mobilised nor adequately involved in CDTI programme. (b) Quantitative method: The team tabulated the scores of each of the indicators for the four levels i.e. zone, district, FLHF and community and further calculated the averages for each level and the overall average for the whole project as detailed in the table below. 47 E"tr 'E trg o- .9E o ott(E o odtr P.9 .= ut o> EEo= =th @ (E N (, o = ED .E= 'E6 FI Eeo.5 E.Etso:< E o CL otr(g tI ootr(E l! o EtGL o oo q, E'T(! L o Zone 3.2 3.3 3.0 3.8 3.3 3.7 2.5 3.0 2.3 4.0 3.21 District 3.3 3.5 3.2 3.5 3.2 3.8 2.7 3.8 2.8 4.0 3.38 FLHF 2.8 3.0 2.8 3.5 3.2 3.7 2.3 3.5 2.2 4.0 3.1 Gomm 3.3 2.7 3.3 3.5 2.3 3.7 2.0 3.0 2.97 Overall average 3.20 This is represented graphically below Overall Grading of Metekel CDT Project s o g fl, I 3.5 3.4 3.3 3.2 3.1 3 2.9 2.4 2.7 Zonal Dstrict FLHF Corrrrunity Overall Average Level Therefore on the basis of the following: Aspects; the team found five of the aspects to be helping while two were partially helping the sustainability of the project. Critical elements, the team found that all the five critical elements for sustainability were present. Quantitative; The average score is far greater than 2.5 expected for a third year project. On the basis of the aforementioned criteria, the evaluation team concluded that Metekel Zone CDTI project, in spite of some imperfections, meets the description of being .FULLY SUSTAINABLE" 48 5.0 FEEDBACK/SUSTAINABILITY PLAN DEVELOPMENT WORKSHOP A three day feedbacUsustainability planning workshop was held from the 27th to 29h May 2009 in Tana Beles hotel in Chagni town. Participants were drawn from the zonal and district level members of staff involved in the CDTI project. The proceedings of the workshop were as follows:- Day 1;27th May The workshop started with the registration of participants and distribution of workshop materials. A welcome address was made by the scout, Mr Tesfaye, who also requested the participants and the evaluation team members to introduce themselves. An opening remark by Sr. Yalmefikier Hika, the zonal onchocerciasis control coordinator, followed after which a presentation on the concept of sustainability, and the seven elements of sustainability was made by Dr Johnson Ngorok, the evaluation team leader. A briefing on the evaluation methodology and the method of scoring, considered as necessary for the participants to understand the evaluation findings, was presented by Dr. Margaret Akogun. The workshop proceedings then turned to its main agenda i.e. feedback presentation on the major findings of the evaluation; highlighting the strengths and weaknesses of CDTI implementation at each of the four levels i.e. zonal, district, FLHF and community. This was done in turn for each of the four levels by the evaluation team members. The presentations were followed by discussions which generated debate on the issues raised. The sessions on the presentations continued until lunch break. The afternoon session opened with a presentation on the SWOT analysis by the Team Leader following which participants were divided into three work groups. The zonal level participants formed one group while participants from the four districts were divided into two groups; one to work on district level issues and the other on issues for the FLHF and community levels. The groups carried out a SWOT analysis for each of the levels building on the feedback presentations made earlier. The first day was concluded with the presentations by the work groups which were followed by plenary discussions. The presentations on SWOT analysis are appended. The first day of the workshop ended at 5.00 pm. 49 Day 2;28th May The second day started with a review of Day 1 activities by Mr Melak. Participants then returned to their groups for the second group exercise i.e. to work on solutions to address weaknesses and possible ways to mitigate threats/challenges of the SWOT analysis. On completion of the exercise, the plenary was reconvened for the groups to make their presentations; this was followed by discussions. The third group work discussed resources needed to sustain project activities for the next three years following APOC's phase out i.e. financial resources, transport, equipment and material resources. This exercise was carried out in two groups; the zonal level and district level. The group work was also concluded with group presentations and plenary discussions. The fourth group work then turned to the second main agenda of the workshop; developing five year sustainability plans. This was seen as the most important part of the ToR of the evaluation team. The group work was flagged off with presentations on APOCs definition of sustainability and the sustainability planning template. The three key issues to consider i.e. cost containment, integration and mobilizing finance were also discussed. This time, the groups reassembled to their respective districts and zone for each to develop its own five year sustainability plan. The district groups were advised to take cognizance of the FLHF level in their planning. This activity continued for the rest of the day. Day 3; 30s May Participants continued working on their sustainability plans with the evaluation team members supervising and providing guidance. All the groups completed Year One i.e. 2010 Plan and plans for at least the following two years. Due to time constraint and this being a Friday, the group work could not be completed. lnstead, the evaluation team decided to discuss and polish up at least the first year of each of the five plans. The group work was therefore stopped and participants reassembled back to the plenary for presentations and discussion of the plans. Before the presentations of the draft sustainability plans started, a presentation on the "critical elements" of sustainability was made and participants were alerted that the evaluation team would be assessing their plans to ensure that all these elements were included. Presentations and plenary s0 discussions, which took a critical look at the different indicators of the planned work, thereafter proceeded but time could only allow for the discussion of the zonal and two of the district sustainability plans before turning to the official closure of the workshop Finally, in the closing ceremony, a debriefing of the major findings was presented by the evaluation team leader to His Excellency Mr. Gawi Jane, the Zonal Administrator, wtro participated in the third day of the workshop. The workshop was thereafter officially brought to a close by the Zonal Administrator who remarked on his awareness and interest in the CDTI project. He pledged for continued support for the program after APOC withdrawal. 6.0 ADVOCACY VISITS AND DEBRIEFING Advocacy visits and debriefings were conducted at zonal and district levels. Three advocacy sessions took place. The first with the Head of the Zonal Health Desk, the second with the District Administrator of Pawi and the third with the Zonal Administrator, Metekel Zone. Each of these advocacy sessions is discussed in detail below. a) Advocacy visit to Head of the Zonal Health Desk The evaluation team met with Mr Beyene Lire, the Acting Head of the Zonal Health Desk early during the evaluation exercise immediately after interviews with the Zonal Oncho Coordinator and the health team. During the interviews, three major issues that required advocacy with the policy makers had emerged: finance, vehicle and equipment as well as human resource. The team leader made a briefing on the background of CDTI programme and the objective of the evaluation mission. He emphasised that the CDTI project was designed to be implemented over a five year period during which time full sustainability of the project should have been achieved. APOC funding was to be reduced over the five years while government and other partners contributions were expected to increase. The Acting Head of the Zonal Health Desk was reassured that mectizan supply would continue and that government is expected to continue delivering the drug to the FLHFs for which vehicles are needed. Similarly, capacity for project implementation had been 51 developed over the years and that a minimum amount of finance was required to sustain the project. The team also presented the observation on staffing at the zonal level as only one health staff member was actively involved in the CDT project. The Acting Head of the Health Desk gave the following response:- o Regarding finance, he remarked that he was aware of CDTI activities and that they were being carried out properly. He also noted that sensitisation and mobilisation of the communities had been carried out and that the demand for the drug had increased. He reassured the team that oncho activities will continue being integrated into the health programme and that the programme will continue after the end of APOC support. He was confident that management will take care of funding requirement. o With regards to replacement of vehicles, he was less optimistic as these decisions are taken at a higher level. He however promised that he would fonrvard the request to the Federal Government through the Regional Health Bureau. The more practical measure he saw was integrated use of available vehicles for all programmes. . On staffing, the official mentioned that the existing government structure limited the number of staff at this level as part of the government strategy to reduce cost. He argued that although there was a shortage of staffing at the zonal level, there were many extension workers at the grassroots level. Finally, the Team Leader requested that in the next strategic plan, 2011-2015, budget allocation is made available for the replacement of vehicles and equipment. ii) Advocacy visit to the Pawi District Administrator The evaluation team met with Mr Aregan Alamirew, the Pawi District Administrator in the presence of the members of the district health team. The team leader gave a similar briefing as with the Head of the Zonal Health Desk and proceeded to present the two advocacy issues i.e. the need for continuing financial support for CDTI and vehicle and equipment replacement. ln his response, the District Administrator mentioned the following 52 aa The reassurance that mectizan supply will continue was welcome and a big relief. He assured the evaluation team that as long as the drugs were available, the district administration will allocate funding to carry out oncho treatment. On vehicle replacement, he confirmed that if replacement of vehicles and equipment is made this year as per APOC plan, the district will plan for the next replacement in four to five years time. Finally, he remarked that the demand for the drug was high and that the programme would continue being supported. iii) Debriefing session with the Zonal Administrator. The Zonal Administrator, Mr Gawe Jane, participated in the feedback and sustainabili$ planning workshop on the first and last days. This was impressive and a demonstration of political commitment to the programme at the highest level in the zone. At the end of the planning workshop, as already referred to, a debriefing session was conducted as part of the closing ceremony. The Team Leader presented the following debriefing: Background: APOC and the CDTI approach, the objective and indicators of sustainability evaluation. Key findings were summarized and presented. The strengths are inclusion of CDTI into the five year strategic plan for health programmes, integration of oncho treatment with other health programmes, committed leadership at all levels, knowledgeable and skilled staff with good attitudes and commitment, good supervision leading to empowerment of staff at lower level, simple and effective delivery of mectizan, good financial control, pool system for using the available means of transportation, commitment of NGDO partner (The Cader Centre), and good geographical coverage (100%) and therapeutic coverage (over 65%). The weaknesses include: targeting of activities to specific needs, lack of structured meetings for ZOTF, absence of reports on supervisory visits, delay in the delivery of mectizan to some communities, misallocation of the drug to the districts, problem of s3 budgeting and identification of gaps, old and malfunctioning vehicles, shortage of staff at the zonal level, insufficient involvement of community leaders reducing the level of community ownership. Finally, the team leader presented an appeal on behalf of the evaluation team; he emphasized three issues: allocation of some budget for CDTI programme, planning for replacement of vehicles and essential equipment, as well as active involvement of the political leadership in sensitization and mobilization. ln his response, the Zonal Administrator affirmed that the entire Zonal Task Force, the highest policy making body in the zone, was behind the CDTI programme and that he would ensure that in their deliberations and budget discussions, oncho programme would be considered. 54 Appendix 1: Evaluation Time Table DAY TEAM MEMBERS ACTIVITY Tuesday 12* - Friday 1sth May Scout a Preparatory activities for all aspects of the evaluation visit Saturday 16'May Johnson and Margaret . Anival in Addis . Check in at Atlas Hotel Sunday 17th May Johnson and Margaret a Team building in Addis Monday 18'May AllTeam . Welcome and briefing at The Carter Centre o Travelto Metekel Zone . Over niqht at Debre-Markos Town Tuesday 19* May AllTeam o Arrival in MetekelZone . Check in Tana Belles Hotel in Chaggni Town o Orientation on the Guidelines and lnstruments . Formation of sub-teams o Finalising the evaluation programme o Data collection at the zonal level Wednesday 20'n May AllTeam . Data collection at zonal level cont. . Advocacy visit to Head of Zonal Health Desk Thursday 21't May Sub-team A (Johnson, Tekola, Abraraw) . Data collection at zone . Data collection at Pawi district Sub-team B (Margaret, Berhanu, Tamirat o Data collection at zone . Data collection at Dangur district Friday 22no May Su-team A . Data collection at Pawi district cont. r Data collection at Felegesalam Health Centre Sub-team B . Data collection at Dangur District o Data collection at Manbuk Health Centre Saturday 23'o May Sub-team A Data collection at Felegesalam Town and Medin village Data collection at Ketena 2, Village 131 Health Post Data collection at village 131 and villaqe 29 a a a Sub-team B Data collection at Dangur district cont.... Data collection at village level a a Sunday 24th May Subteam A Data collection at Ketena 1, village 7 Health Post Data collection at villages 6 and 7 a a Sub-team B a Data collection at village level Monday 25tn May AllTeam Data analysis and completing master copy for each instrument 55 l Tuesday 26h May AllTeam a Planning and preparation for the feedback/sustai nability plan workshop. Wednesday 27th - Friday 29h May AllTeam a FeedbacUsustai nability Plan workshop Debriefing of Zonal Adm inistratora a Grading of the whole projegt Saturday 30' May - Monday 1't June AllTeam a Report writing Tuesday 2no June AllTeam a Travel back to Addis Ababa Wednesday 3'd June Johnson and Margaret a Johnson and Margaret return home. 56 ] U o o = ' o U -o = ! o o ll = ! o o -o E ' o U .o = ftr os o6 E ,tr oE oo = ! o o .o = o oo o =o E o o tr o bf E tlr E o o c o o P o lll 3 o o o E J E o v, a, E oo EE oG o .= E o = tr o v,cE o o fc oE oo = 3 o o -o = o6 -vo o tr o u,trs o oa UI tr o CL U E oG J! E c o (, ,1' o .E 6 E o oT'tr o ol t,c o tr o o .9 ol oe .Y o E o o) .Ec o CL o E o UfT' o .E ov, o o o E olc o o oJ o u,f o o E,)tr c o o E oE !c o c, o o .? o o o -g o 0 .ir L'6o; lr'EoE ot .CD UIo .st,tr E o otr oN U'otr otr o o U p a vt olct,C E u o IL E =o o - oc J c o IL 3 ut .> oc o o = v, c o c o o tr o vl o E f E E o o E o tr o 6 UIf(,2E b oc oa !c o tr o o c o0 o e J o! of o o vl o UI vl ocJ o o; o U'c o f6 !, Etr> .27 >yE+ op .J 03Scuti ;oiBo- iEo-tr Oof oo .i,j a o(v) ,id !oo ri,j a oo 6 ro 6 a roI or ro(\a 6 U' 6 rot c I roN di tr o0tr -tro o (lotr Ec o o =ot c o UIc o o tr o UI o G oo o I rot 'ii Io tr o uf E rlt oC' I o(q, o o(Y' I oI oo c{ I o g? o c.i I oo ci o(r) ci I oI(\a oI ro 3 o(Y, ci c, (a o ei euo Aec t--(n 6 C oo .9.oo '6 L ooo .C,oc o L o E o (L c o C) L o .C E .I! o tr .I IEg o -Jo= rOFFEl '=(, b,lq e5 cOolEO ER Otu!> E)Otrol 'i ett -lLt\C b'.1EEE9trtL o 5(, t!g E o oE o E E G E') o o. CL o oJ o E ci .x!,tr o o. CL ,: ") ,.j ',.::i 1i t -..: .) @n o o(L 6 c o C) .oo(U o oooEoc o o E o (L c oo L o -cF E a-! .U .= IE +r o - -o IFEo2otr L.= -I- 'E CDLEoc)s 9cDEoaO; c\t PS a-fco ERrFl !NCF rEE LJLLg IEI! o oIL E og 5eEOo-o ,Crb ,AE E og 5e:EO u-o ,crb ,iE E oU'c o -co ct ;o o .o j !c o =oE:XcfEis E og te:E O'u-o)EEb ,AE ltr o E, o aq E tr o alc o3 o o vl c ,t tr o e uE or tr oo '0 oG c o UItrE o o UI c oo u =LoG c o ultrE o o fc oE o aO E dc o(, c oE g €c E UI) UI c o v o 'ol o o c o J o 3 c o c o g f =o o u o e, N .v o 'of o o c o c o o tr o UI o A UIC .9 v,U' D u .2t, otr -9o !tr o v,tr o o C 0) U' o o .Y o =of o o o tr .9 E o E !,0 oc .Y o o 3 u o 'd EL o vttr o ! o0 tr o +FOE3t cLC 3E o7 (9 x o! of o o tr o tr o o tr o a o L tr J o 3 of o o tr o c o o c o v, o G roo I C' c o I roo II o(t, N I o - ros .ii I o(, dci lo ro dd a rot dd oo c,(9 o ood I ro ro .id ornl ^Bo ) 4 ; It :l ri '.-i .t ri. :a:4 ,hit 'c ,j$: .2 o\tr) O o c oo .oo o o oo oE oc o o E o g oo 'E o -c E a-! G .s IE *. o I) o ILC,b=otr L'= ,\ I_ r: rf)tEOOs 9cDEoaO;c\ Pg 'Eb !nrFl 6 crlEFG6t-J tr- o G!! o o IL E og 5E =oolt ,Erb ,AE E og 5e =6,U3fctbAE v, oo E o E E o o tr o o , o ut a tr U E oL tr oe '0 E o4 E o oE c o b p .E Et, o otr oN o E o E ! o oE tr o b .9 .E EE o o C oN o UI E,|tr Ec o .c o oc o .o oo 1A c .90 v,f u .2! !c o U'tr o o tr C, v,(, o. tr o CL =o o ._tr o tafU' dc o u 6c o 6 1'f(,IE utr o tr o os o U' D v, ! o E o o .EU' -9U u o v, c o E oo E o E oo UI of u,l4 o E )r E E! o(Y' t,j a o(v) ci oq, ci a o(9 o oo o a o(? o oo t a o(9 .Y) o c) $ I oC' s oorql ^Bo .L rt ,' .*-* ". i' + \ .il "j " i. Appendix 3: SWOT Analysis ZONAL LEVEL Strengths . lnvolvement of task-force in planning CDTI o lntegration of CDTI activities . lntegration of CDTI activities with the activities of other health programs . Active leadership at Zonal Health Desk o Effective system of Mectizan@ procurement and delivery Weaknesses o Delay in releasing budget . Lack of qualified professional in data collection and management o Health personnel's minimal computer skills . Failure to note and document feedbacks and subsequent actions taken o Poor documentation system . Failure to write reports on feedbacks to districts . Lack of regular or immediate maintenance for vehicles and office equipment Opportunities . Availability of budgets for other health programmes . Favourable policies and strategies on inter-sectoral collaboration . Donation of Mectizan@ . The arrivalof Mectizan@ during the dry season Challenges / Threats o Shortage of budget . Shortage of human power . Limited involvement of political leaders as they are preoccupied by other administrative issues o Shortage of vehicles DISTRICT LEVEL Strengths o Participatory planning involving FLHFs staff o Undertaking CDT activities integrated with other health programs o Availability of supportive leadership at woreda health office . lmproving therapeutic coverage . Timely delivery of Mectizan Weaknesses o Shortage of vehicles and essential equipment (computer, printer, photocopier) . Lack of regular maintenance schedule for vehicles and equipment o Failure to replace old vehicles . Failure to solicit other donors . Poor record keeping/filing practices mainly at FLHFs level o Delay in delivery of reports from FLHFs to district and upward . Lack of schedules for training and HSAM activities Opportunities: o Availably of resources from other government sectors (Education, Agriculture etc) . Favorable government policy regarding integration and inter-sector collaboration 60 . Availability of pack animals to transport drugs and other materials . lncreasing awareness and acceptance of the communities regarding ivermectin therapy o The increasing attention of the government to expand roads in the entire country Ghallenges / Threats . Governmental budget constraint o Delay in budget release from APOC . Untimely release of fund from TCC . Difficult topography and poor road network of the diskict . Migratory nature of the inhabitants o Scattered settlement of communities in some districts o Lack of qualified maintenance workers in the locality . lncreasing costs of spare-parts and maintenance service .. COMMUNITY LEVEL: Strengths: o Willingness of CDDs to render CDTI service o lnvolvement of community members in selection of CDDs Weaknesses: . Low CDD to population ratios in some communities . Minimum involvement of community leaders . Delay in submitting treatment reports from CDDs working in remote communities . Less involvement of community supervisors in carrying out Onchocerciasis treatment Opportunities: . Acceptance of the service by community members o Communities witnessing the benefits of Mectizan Challenges / Threats . Governmental budget constraint . Delay in budget release from APOC o Untimely release of fund from TCC . Difficult topography and poor road network of the diskict . Migratory nature of the inhabitants o Scattered settlement of communities in some districts . Lack of qualified maintenance workers in the locality . lncreasing costs of spare-parts and maintenance service Proposed Solutions to Weaknesses: . ZONAL LEVEL . Timely utilization of allocated budget and reporting o Facilitating training courses on computer skills and data management . Writing and documenting feedbacks after carrying out supervision . Lobbying for better budgetary allocations . Soliciting funds from various donors o Contacting the Regional Health Bureau and lobbying for a change in the zonal structure . Requesting the Regional Health Bureau to allocate budget for procuring vehicles and equipment 6t DISTRIGT LEVEL . Lobbying with higher officials regarding budget increase . Directing FLHFs staff to recruit and train more CDDs and community supervisors o Making arrangements for early HSAM and Mectizan distribution for migratory communities . Using pack animals for transporting Mectizan for remote and hard to reach communities o Note and fonrard feedbacks to FLHFs . Develop training schedule and execute accordingly . Scale up the knowledge and skills of FLHFs on report writing, documentation and forwarding . lncrease the knowledge and skills of district staff on report writing and documentation . Usang vehicles from other sectors COMMUNITY LEVEL . lncreasing the number of CDDs as per standard (1:50) . lnitiating more involvement of community leaders . Developing documentation and reporting system . Lobbying for voluntary resettlement program for the scattered communities . Promoting integration with other health programs o Provision of training for new health personnel at FLHF level 62 Appendix 4: List of Respondents ZONAL LEVEL (Gilgel Beles) Name 1. Sr. Yelernfikir Hika 2. Mr Beyene Lire 3. Ms Yeshiwork Addisu 4. Mr Simieneh Alemu 5. Mr Kef Yalew Muteta DISTRICT/WOREDA LEVEL 6. Mr Aschale Ayena 7. Mr Haimanot Gabeyehu 8.. Mr Andargachew Shibabaw 9. Mr Amsalu Bishaw 10. Mr Misganaw Tenaw MrWorku Teshale . Mr Arega Alamirew FLHF LEVEL 13. MrAshagre Gebeyehu 14. Mr Heyelom Berhanu 15. Mr Berhanu Yilma 16. MrTarekegn Lachamo 17. Ms Zemu Adem 18. MrAlemu Ejamo 19. Miss Hezeblanchi Mekunaw 20. Mr. Adamtachew Birhanu Responsibility Zonal Onchocerciasis Control Coordinator Acting Head of Zonal Health Desk Zonal Head of Drug Store Oncho. Focal Person for Zonal Finance Office Zonal Mother and Child Health Coordinator Head of Dangur district Health Officer Oncho. ControlCoord. for Dangur Health Office Head of Dangur Health Office Oncho. Control Coordinator Head of Pawi district health officer Health Service and programming Team leader District Administrator Head of Manbuk Health Centre Oncho. Focal person Manbuk Health centre Head of Felegeselum Health Centre Head of Ketena 1 village 7 Health post Health Ext worker, Ketena 1 Health post Health worker, Felegeselam Health centre Clin.Nurse and Head of Burj Health post Lab Technician - Gublak Health Post Health ext worker, Ketena 2, village CDD, Felegeselam town Com. Supervisor Felegeselam town Com member Felegeselam town Com member Ketena 2 village 131 CDD, Ketena 2 Village 131 CDD, Ketena 1 Village 6 CDD, Ketena 1 Village 6 CDD, Ketena 1 Village 6 Community member, Ketena 1 village 7 CDD, Ketena 1 Village 7 Community member Ketenal village 7 Communityleader/Health supervisor CDD Burjcommunity CDD Burjcommunity CDD Asama wiha community CDD Ketena 3 CDD Ketena 1 CDD Ketena 1 CDD Ketena 1 Community Leader Gublak Village CDD Gublak Village CDD Gublak Village 63 Location Metekel Metekel Metekel Metekel Metekel Dangur Dangur Dangur Pawi Pawi Pawi Pawi COMMUNITY LEVEL 21. Mrs Etenesh Molla 21. Mr Gashaw Yehunie 22. Mr Nigussie Gebremariam 23. Mr Esubatew Bitew 24. Mohammed Ahmed . Mrs YeshiGashru 26. Mr Tesfaye Abuye 27 . Mr Markos Arsicha 28. Mr Tesema Adda 29. Mr Shewelie Meskelu 30. Mrs Emamu Ayele 31. Mrs Kedija Ahmed 32. Mr Malade Azene 33. Umar Nuru 34. MrAbenai Malis 35. Mr Teshome Dilnasaw 36. Mr. Abere Tizazu 37. Mr. Getu Tinsae 38. Mr. Ali Mohammed 39. Mr. Nigatu Taye 40. Mr. Tariku Terefe 41. Mr. Yabes Lango 42. Mr. Zeleke Roro Dangur Dangur Pawi Pawi Pawi Pawi Dangur Dangur Pawi Pawi Pawi Pawi Pawi Pawi Pawi Pawi Pawi Pawi Pawi Pawi Dangur Dangur Dangur Dangur Dangur Dangur Dangur Dangur Dangur Dangur Dangur 43. Mr. Gedlu Tefera 44. Ms. Tobia Tefera 45. Mr. Sisay Sayih CDD Gublak Village CDD Gublak Village CDD Kola Village Dangur Dangur Dangur 64 Appendix 5: List of Documents Reviewed At zonal Ievel 1) CDTI Treatment Summary forms showing total population, targeted population, population at risk, total population treated, geographic and therapeutic coverage percentages for the fiscal years of 2006, 2OO7 & 2008 2) Five year integrated strategic plan of 2006-2010 and a one year plan of 2006, 2007 & 2008 at Metekel Zonal Health Desk for all health programmes including CDTI activities 3) lntemal offlce memo for expenditure autorisation, payment request forms, petty Cash and Cheque Payment Vouchers, cash transfer letters, debit and credit advices, generaljournals and ledger cards 4) Annual technical/performance reports 5) Mectizan requisition letters to FMoH, Mectizan distribution letters to districts, stock record cards, store requisition form, goods receiving voucher, store issue voucher 6) Training manuals, flip charts, posters 7) Travelauthorization forms and log books At district level 1) CDTI Treatment Summary forms from four districts showing districts' total population, targeted population, population at risk, total population treated, geographic and therapeutic coverage percentages for the fiscal years of2006, 2007 &2OOB 2) Five year integrated strategic plan of 2006-2010 and a one year plan of 2006,2OO7 & 2008 at the districts visited for all health programmes including CDTI activities 3) lntegrated activity schedule prepared for all health programmes 4) lnternal office memo for expenditure autorisation, payment request forms, petty Cash and Cheque Payment Vouchers, generaljournals and ledger cards 5) Annual technical/performance reports 6) Mectizan requisition letters to FMoH, Mectizan distribution letters to FLHF, stock record cards, store requisition form, goods receiving voucher, store issue voucher 7) Training manuals, flip charts, posters, timetables for health education At FLHF level 1) CDTI Treatment Summary forms showing total population, targeted population, population at risk, total population treated, geographic and therapeutic coverage percentages for the fiscal years of 2006, 2OO7 & 2008 2) Community supervisors CDTI summary form, CDDs treatment registers, annual technical/performance reports 65 3) Written verbal document for issuance of Mectizan to CDDs 4) Training manuals, flip charts, posters 66 q1fr oE H z t) o II a0 rir tF( o U)oIL oa U otr o a(J z ?n ?t? ?a(a o\ \o(a i c U 6t o t''( [a o\l,a ra o\ ra O ra o\ ra |,a ia inr-{ L e6)Oc) xc) I t*Ht*( F]tr Eeah o) a ia o\to o ra o\ ra iA o\ r,a r,o - ia ror-l L HO) ocJ HC) a-Eo s-E (\l - - N N GI o6l 6l N O GI t-l GI 6l N 6l q)I G Er U) -0)FItr= E8x {.. riLtroq) *EE E e.=' --HE F-\o o L u) s!L E: 6l r=e rd Fltr tr il 13at-a)o o UJ otD oo ult 2 J(L tsJ E z Fo) at, E, IIJ lll lr UJ F ad xoz IIJ TLo bo E fH t- o u) €) e)li oa Q o 0.,( O r- o z oo ca oEa O r-. o.l U) oQ $coO+\a)\o\oc.l a5 XC.l+# o \n O rr) la) r-$ e q)li 59-Xtr J o- .^ 8H9() oo -E>fuE j F 9LB .:4xJ €d-- L '=tE Fo ^qcxo-- uo'*d =as8 >,;oo *F 6g Etr e .. bo x ^ tr €[96=.EF'rg -.F';ots.'=tro x o ! box'-EEe58gilE E ^0)d bc EE F b# =.EE,gsEep ?ZHO)EE .E' rr) E!= E .2a -r3Eq- I.EE O E.Z o=Gt!i/L-g{ 6) o-(d .BgEE " tts=e )6 Ets-U o=(!u b9 a Eabo= :\ EE:€7)trO(s a th -tr OD I*LrA €) q,JFrtrE ?9|=6UL €-EFE.9ir hH=o (o qv UB-g€d8€ E()-r o€ - H 9!Fgtr e Eo7a ii (0,i >5 B.flE c9 v(l)Y,G Etr 9 9d€_^9ti EH=fEg;E alP .i !rHtr o\OON L< a. la) I -o C) frr O o.l I\o d U) 6) I Q) !c dAFc(!ti=tr 5ra3b- -I UEEESX 3ril-.-g- .=L, = >c\ o.) .oeSExtsF= O.(6q:E t-] o >.l<(! O -AG)t-6) -Evl/1P BEA.E .Ss t H!v 'a I e?trg;e8 !I L c) HOF -c o-rO.= oo ='tE'.8 H U .i9-9 = >! >E!rr!i(E€N€ -C)()()Pi cd-q (B V)tr,tr i+i 5EoE ^ od-?n a diJ.= .NN5 ts s'i ZG€ z a FTi P- g1 5.e >,.i. .{5LJ(B €\o t-(Ukll- oil F.( t h5UGI)lrn< V t.l rd ,rF-ri .9EN ea9EiZE4E :€!riEF<A > '.1ir frlt> =HEi z - Fra)a UD +i +r o ahq)(J lr oa U 0r a(, z r-z ah o Q O tr) ca lr) cl c..lrn o.l o\o cacl O rn oo N + r- c.) 0)|r q) lr 5 XE () E (/) o4io& >. 6)Etr() sbAor C))- v)o .L C)& (l)trtro) .YE sbAA C) lr(l) o. €g 0)= ?ats 3,1 EahEl-io X E }EE.X .E .E E FEE EEs E }88 EE.E E = $ E HE E ;= # sg35^rt.T5-88 aa o,G(! -E.iepE Ed EE:EEE BEF^3s B g*EE ;aEE [I Es aa ah at) xtr ah al qa -:,Atr.A c9 v Hl: b .9-cEFo BE ii(J '!E-8€e,.i ;UFaVLtri- di3sEE3EE &U f9E3€q,.i ,2(L,i (qIP (t)o- l.< I(,(d E.E og .= !rtrfi (n IO C)t! I >.Lrdo EN I TE ahq) (.) q) l-(!- € orR E.EE .j.d)v 9L!- :td!€4 E.E= -E 9:s I E. q ."'EEgEH( tv *9 tv - ch -() o(gFO o -!.^!H SgrH.-U!.:>-cji nr'E u o 'E.= 3 I E9 t-E Etr ()F - bo ^ Qn -o o .v 3- E l.F i, q./ cB A. a<>'a.EFTt C ()E cxd)o->UHg >B C, 9. -L(H !+i >, (,o'li 9 .:'xboE = tr; rUlrv{.rL .rFAd- .E=a F bcd (in tr o-L'IHAi' F.q\./ o ca Id dH.E o E oo)= >.9 tr't Y 'E.3'E H r,loP;--q t5'= f+ ii B*E8E o\\o F Q E]r- oilA FolF '-rlAOIi..rNl "-aH< EFts-r rd .91!r Ltaal x5l EEI lEl dsll-l EFTt]a - IrJtrrf zl-( tr0 Ja a0 ri Cr( o (t) q) q)fr oa U 0r (J o O. z o =a (.) U' o(.) (D LIoL 5 €)-frtr (l) oq d (.) (! a 6.) -oL o €o L o:. rr)6l eh cJ q) .r! . c)E":= e - v -t-l-{ V) rh .8", P EEE:Ef i€ Et E .=(s o.9lztr .= clt:€eF69Sts botr i, (a) ahxl- thor q:nitr,o o)l:()9!r-1l oElv !r !- -!9 .9 -'E ts -c!5F YX=(,A ol-l (3EU 8e.E EEOo)fri e olP ,; !tF+i I CUk- dU v,q) (.)(D E o a(t) .s .E(da '; Ol'5(€dF oQFO (! -6)ol IsZER5 e2E b =Liotr..=2oc) ^Eo)Fdij e) bobo F ...EUV .= c s/, x ^^oE E-E€HE = $= O *-L- c)ilxEf E BEEd€ 9E EFdtr E oa q< (.) oEgtr=o!PL<tro 6dO >E Or- t'rU frlh o& F{ FOFF(AO -6rYs!-i< EFF*.Ft .9 =LaEilF PE 1EJc! e", EFrtFrnf- L'FIal-l EEIEej zF{ E-ra)a t-l q) o0 cle r- t N I t-lo6t r-i(.) rih oil Fr Ha(l) a5 QN Jq) r-l tXE E>ri9EH20 ,N :do[=oz: |i (l)Fr> aq) IHiR El ri46)2A ?Frtra Pa 1 a0 E tr 0q) Itr a U o er \o oo n z cao\ lra !t$r- @ c.lra i ;.1 .Y ocaF C)ii o\o € 00 6l(a q) c)L $e&e a.8? SE B !0. A >rF ai u.9 _'tr E'o! r ==o)Ir. J O. aaa o oL Eqt) t) .9u Ea o)>- .-€6)^6-)LF=H - € EEgq;=FflEE 5 {E i E E.q E E t (gc *@ rEEa E L.:.- -id(!>oE.N = 19o E€ B : E E<.4 N-qE a. q) E t, oa U)q)/, o0L O Fr -? .EC) 5ET H{E €Eg€EE E G t<9>.(do c(dif<'o tr oH L K'I;\6dx5 3f, g q) artr E9r trk o\ c..l z Io ;1 O(0o = ciE; oe Io '= !iEg Pe oE lfEE*.E-€l 6 sE Egt I EEE FE E,E i aaa 4 eBH gE HE: fr E's E'x€ g; ."€E s 8-: EFA cd d i, d > a 0)o (.) -cF,! ! _ l_.1Rl ! r\bo6) Ov -^ .E<E n [iHE }tEiiJ e 5 )0) tr t?i'.=Fr--uPr (i P.,= cd n>\E L^ @ gE sfiy elYOtd -EHB5EFgEE>\./< 6J trtl. aZ c.l a I ,al tr- N q) u0da t Fi N Ic O6l r-iU tql- o& Fr I '- q) hEUNt3|-!o) E€ F-t a HEz.o ,N i^O lroz:4rL -ld H6)trl> -€)rLi:i o =Nl*'l -26 ?Fr3a Ja I, a b0 f* o 0o(JL a U o Fr o @$ € a z (.) cil c.) Oo o =a trt I-r o\ ol F- E;.: F c)l.) O co cl o\o ra) co 0)L FEil6 hto)ctHu cZ-E 5 a Ot EhhEBHo:E aa o d9! b.qE-oO{E aaa o aL Eokr?L)x rC) EA ,h9PBH-.'ts n ei-pH'E ET€9e H S;HE.E &t e s Es eE E.Ets E€ a aOa Ef,€=stEsH**r aa c) E o aD* L(h6)aailfr - H',! 5E$E EE o Y 6) -W ? rr S E.E A E"E k o d i<EFLAO H^ \J() o ortr EGI ti tr 6)(Hxoq: o0 eSg'E -.(g^-c< = = H'Eoo sR b-8= k C)! oc\OA6)Xax (r)() q) oE o 9c* 6)Eox "EHsrC.= h o) o) EH=r- tr-v l: a!riO .YUddFo{:tr a o oo q.t 'E EH.-v =oat;ioe!= e'EE€E +!T N E *s I a anq) I C) =P =,o>\ ' -51E H }ore :loH()b.A U E\JdtrN =E q=0uo,lO.E-q E:EEE -^--J'-*6)-L n f -0 E E aZ c.t $ a0 f= o (t) q,) Itr oa I o er lr}\o €\o c-\o(\ n z ala) a.t c.l Ot-- oo oo oEa rn r--@ I $ co(n c.t * ;.r F (Jii o\Orn(\ c-\o c.l q)L FEil6 o o C) a C) ! !o O. a AoX b.Eg5 8*o..6q.aoEo aaaa ot-S,dk 'ir e) EA g r: Et=tg*= EE# eE aoa gac) ! (U- ts *.e H e 3 E PFE'E56c-il :BEEx*Ai aa E f *€ *fi e*€ aa c) o 4r2 *L at,o)Cilg tr O >..hE:Er) tr () ij -=-c o9tsi'iar E 8 F8No{:N L o d |-!FLAO|io\-/ o ! (oL 8.9 0): OF Yi'; a\ o NO 5 q) ortr Fi fr O iot (E ii>{ +ro^ -v- $R2>\ e(d cia +rOO -Y :^gxBe ;\< - .2. (h q) (J q) >\ 'L li7\ iri oa5ES-EBE Hs gE s E i rii>8tr>>;io -v(Ji-voL<q.)F(dtrFdA.>q aa stsS o-r .=o EEE=eg aBO'F ."EbsEF(no.E(! a i!d c.rEg-€5 ". (uE5'ot :Z o'6,n CtE E=Er;Efc H H€ q I g H:: €BE.#E€-UE a oe o 'ra-a: e v- (l) =e.E3tHEoE'=.9Ja) r.E5: HH EL p.p.JE- trd a 7€ EE 8 L oa()E - h- - .E EH bE ri+ c _6trd,P67sE ;3o0) EesE<5cA (l)8= H aZ tr) \o c- ca r-. $ F{ N I N r-i Q rr]l- oil Fr lrl ,o)h5QN riiFl o) l-oITJ E "1 a rd9 -H>,o ,Ni^O fEoz:{kEi H9ILiR 26 ?F] t.ta ,)a t I \o0 Fr o oq)(, o CD U I O o E. o eIz !ra i E.! 0 (l) o 5(u & b € ,E- eEoJO.:() O--+eo E g gE E E ^ Et€EEE5E.g o 0 Eo c) !'? E cr) -!G,tr>(! tqe!occ6 ii >''= HEE.SE.A:i(* dFO'u o E 0 aooil oo q) A oH N E (g frOhEO.E9E=!rO(,s 8_3 o)otr .EEFfr tEo (€v o €) q) o ).- h0oF = tr .a o d'E .^ EEr iI €EESE a 0 Q) I (Ho otr E! o- o:iE'-kbts6 td()(n iDOL*d> u z a oo I t $r- $ =o6l= lN v0) xg ^€)ti5 !.2 ;Fra3s&opr= EHhi:r\ Glvo)Fl> frlvroEla ltO tiN r-'! r= 6) dqio- tEr z Fl Fr Fl-l FlF{tr zl-'( Fa)a .t a t ra) r-- +,O o .o-t l- o. Fo O oC oN oJ o+, o I J a o -o Ef z Cf o E o,c Ecf o a og f o U) o o TL (o(o r o) (o(o o) (o(o o) f- o(, oz lrDf-$sN lof-s$N lr,l-\tsN o o @ rool-r @tooN @ 1r)oN E o o) o)(o o ro o) o)(oo 1r) o) o)(o olo @ @ @ N ILI LL .9 E9oo 6 o ttr oN o) o)(o o ro o) o)(o olf) o) o)(o o ro @ @ @ l'* o oN oC\t NroN o oN r oN N oN o oN oN N oN o otr Gtrir o oo sdtrof,o r. x. flE6qo.hAa| Ee 3Foo I I