Evaluation of sustainability of East Wollega CDTI project, Ethiopia (Third Year)

World Health Organization African Programme for Onchocerciasis Control ! t y Evaluation of Sustainability of East Wollega CDTI Project, Ethiopia (Third Year) May, 2009 FRANCISCA T!LAHUN AYATEW I I OLAIVIIJU ABATE FIREW APOC EVALUATION EAST WOLLEGA CDTI/Third Year 1Acknowledgements The Evaluation team is grateful to the following persons and organisations for their cooperation, contributions and assistance towards the successful execution of this assignment a The Director, Dr. Mrs. Uche Amazigo,and staff at APOC Headquarters in Ouagadougou for making available the necessary financial and logistic requirements for the success of this assignment. The WHO, Carter Centre and Light for the World offices in Ethiopia for providing support, which contributed to the smooth functioning of the Evaluation Team. The National Onchocerciasis control programme staff for all the prel iminary a rrangements and facilitation. Mr Ensermu Jeldu,the Zonal Onchocerciasis focal person for East Wollega CDTI project and his team, for all their the logistic support. The Zonal Health Desk Head and his deputy for their kind support especially at ensuring that all the expected participants were around for the three days sustainability planning workshop. We equally thank Zonal Health Desk Head of the West Shoa zone for his participation. A very special thanks to the Acting Zonal Administrator of the East Wollega Zone, and all the other Woreda Administrators for your time and commitment ,We appreciate the fact that you were present all through the four days planning meeting. The Woreda Health desk Head and Onchocerciasis focal persons from the 10 CDTI Woredas of East Wollega and West Shoa. We thank you for your commitment. The female Health Extension Health workers manning all the Health post visited, for their hard work and willingness to share their CDTI data with the Evaluators. The CDDs, Kebele Administrators, community leaders and community members that provided very useful information that made this Evaluation possible. The Evaluation team finally appreciates all the drivers and local guides that assisted through this exercise. a a a a a a a a a D , I 2rl Abbreviations/Acronyms APOC African Programme for Onchocerciasis Control BPR Business Process Re-EngineeringCDC Communicable Disease ControlCDD Community Directed DistributorCDTI Community Directed Treatment with lvermectin EPOC Ethiopia Programme for Onchocerciasis Control FLHF First Line Health Facility HQ Headquarters HC Health Centre HE Health Education HFs Health Facility staffs HEW Health Extension Worker HP Health Post HSAM HealthEducation/Sensitization/AdvocacylMobilization IEC lnformation, Education, Communication LFTW Light for the World MoH Ministry of Health NGO Non-Governmental Organization NID National lmmunization Day NOTF National Onchocerciasis Task Force PHC Primary Health Care REM Rapid Epidemiological Mapping of OnchocerciasisRHB Regional Health Bureau TCR Therapeutic Coverage RateWHO World Health OrganizationWOFP Woreda Onchocerciasis Focal personWoH Woreda Health Office ZHD Zonal Health Desk ZHDH Zonal Heath Desk HeadZOFP Zonal Onchocerciasis Focal Person t I 3EXECUTIVE SUMMARY The East Wollega CDTI project was launched in November 2004 but actual reporting on distribution started in Year 2005.The 2007 updated census shows that the Zone has a total population of 1,175,853. The districts of the zone are sub divided into 367 kebeles, of which 339 are rural and 28 are urban. The kebeles are further divided into Gare (village).Each village consists of 30-50 households. Villages are responsible for all developmental activities including health. East Wollega CDTI projects has 10 districts (Woredas). They are Jima Ario, Leka Dhulecha, Diga, Guto Gida, Wayu Tuka, Sasiga, Sibu Sire, Bako Tibe,Boneya Boshe and Wama Hagalo. Recently the Bako Tibe Woreda was moved from East Wollega Zone to become part of West Shewa Zone.This therefore makes East Wollega Zone to have 9 CDTI Woredas while the West Shewa zone has 1 to give a total of 10 CDTI Woredas. ln the 10 CDTI Woredas there are 4299 villages and 8478 Community Directed Distributors (CDDs).The Project has 419 trained Health workers involved in CDTI. APOC has been supporting the project since Year 2005, According to the East Wollega Zonal Onchocerciasis focal persons the project is its fifth year technically but financially they are in their third year. Considering the age of the project and the need to conduct midterm evaluation for CDTI projects by their third year, APOC decided to conduct an evaluation with the following objectives. The general objective of the evaluation exercise was to determine the sustainability potentials of the East Wollega CDTI project by its third year of operation and assist in developing a plan for sustaining the project post-APOC. The specific objectives are: To assess the performance of the different groups of indicators of sustainability of CDTI projects in the East Wollega CDTI project To identify the factors that may block or help the sustainability of the project Discuss the outcomes of the evaluation exercise with the relevant stakeholders in the East Wollega CDTI project Develop plans for sustaining the East Wollega CDTI project post APOC ln order to carry out the above objectives, a team of three Evaluators was sent to the project. The Evaluation took place from the 18th - 23'd of May 2009,while the Feedback meeting and the development of sustainability plans took place from the 25th -30th of May 2009. ln carrying out the Evaluation, two Woredas, 4 Health Facilities and 12 villages were randomly selected from all the 10 Woredas .One additional Woreda was equally randomly selected as a reserve one. a 4The evaluation was conducted using four types of instruments developed by APOC. lnstruments were used at the 4 levels to access the projects performance regarding the routine activities and processes, there were 6 group of indicators to access this namely; Planning, lntegration, Supervision and Monitoring, Mectizan supply, Training and HSAM. Resources provided for the activities were accessed using 3 groups of indicators namely; Financing / Funding, Transport and other material resources and Human Resources. The result achieved were measured using Coverage indicators, both Geographic and therapeutic. The performance of the above indicators as well as the various aspects of sustainability (lntegration, Resources, Efficiency, cost-effectively, Simplicity, Health staff acceptance (Attitude of the health staff) and Effectiveness were used in grading the performance. ln addition documentary evidence from CDTI data, reports and plans, inspection of capital Equipments, Verbal reports from persons interviewed, Community Meetings and additional insight during feedback meetings provided additional insight to the evaluation team. The five critical elements of sustainability (money, Transport ,supervision ,Mectizan supply and political commitment),the seven aspects of sustainability in the project were qualitatively discussed and results agreed to by the team. The project was graded using these aspects and elements following the Evaluation guidelines. Qualitative descriptions of problems were deliberated upon and recommendations made. The sustainability judgment about the sustainability potentials of the project was therefore based on the quantitative assessment of the average sustainability scores of the groups of indicators as well as the qualitative assessment of the critical elements and aspects of sustainability of the project. The following are summaries of the findings; Community Level The community level was judged to have high sustainability potential based on the insights below; The CDDs plan and manage CDTI activities in close collaboration with the Health Facility staffs, kebele leaders and village chiefs . The main activities enumerated by CDDs as their key CDTI activities were; Conducting census updating, Requesting for Mectizan based on census population, lnforming communities the specific date of Mectizan distribution and reporting back on treatment to the HFs. The village registers were seen at the Health facility and confirmed that the above activities have taken place. Kebele leaders as well as village chiefs have impressive knowledge of CDTI activities. They know the impact of the disease, eligible population, as well as its benefits. Leadership and ownership of the programme are shown in various ways like selection of CDDs, initiating annual distribution activities, agreement on time and mode of distribution; they also participate in census updating, social mobilization and monitoring Mectizan distribution. , 5Community members have positive perceptions towards CDTI. They have given high value to Mectizan and wish they will be allowed to take Mectizan for unlimited number of years. All community members elaborated that Mectizan has multiple effects on their health. Besides Onchocerciasis, Mectizan kills lice, bugs, relief from persistent itching and expelling intestinal worms. It seems also that the communities decided not to provide any kind of support to CDDs because of the believe that it is a great honor to be appointed to serve their community. Community incentive to CDDs is also not common in communities visited even the CDDs do not expect it. CDDs are willing to continue to distribute Mectizan because they are happy to protect their communities from Onchocerciasis and other diseases Current CDD: Community member ratio is 1:80 CDDs are reporting their activities to health facility staffs, Some CDDs are literate enough to summaiize their activities and report it using village summary reporting formal, others submit the completed village registers and summary is done by the front line Health facility staff. The weighted average of therapeutic coverage for visited communities is 73% from year 2006 to 2009. Some CDDs did not update their community census accurately. Only one out of many CDDDs met talked about incentive. Front Line Health FacilitY Level At the FLHF level sustainability potential was found to be high with a score of 3.5 using the g group of indicators. The CDTI programme is functioning efficiently in some of the indicai'ors iike finance and transport but the system used was found to be innovative and unique for this project and Evaluators had to discuss further at the indicators and characteristics of the indicators in awarding scores to those indicators. Written plans for Year 2009 CDTI activities were seen pasted on the wall of most of the Health Facilities visited .The activity plans were written in the local language (Oromifa).No written integrated health plan was however seen at this level. HFs said they worked in close collaboration with the Kebele administrators in carrying out their duties including CDTI activities, however minutes of such meetings were not seen. The Health Facility staffs are in charge of all the health programmes in their kebele. Each HFs is manned by two female and activities are carried out in an integrated manner. The Evaluators believed them because the CDDs and communities commended their hard work but written evidence on integrated activities was not available. The health facility staffs takes full responsibility of CDTI activities in their Kebeles ,ln all the Health post visited, Health staff explains that their annual CDTI activities starts with a meeting with their Kebele administrator. Activities usually discussed and agreed upon include census updating, mobilization and Health Education, date of distribution, training date for CDDs and any other issues of concern that the Health staff might have. The Community structure is very much aware of CDTI and lends support to the Health Facility staff. However no written report of the planning meeting was seen at all the health post visited. The supervisory data is being transmitted entirely within the a 6government system. At all the Heath Facilities visited CDTI data treatment summary for three years were seen pasted on the wall. The summary provided information at a glance on CDTI performance in the communities under each Health Facility. The management structure in operation at the Health Facility makes routine supervision of CDTI activities easy and efficient. Each health Facility in the zone is manned by two female Health Facility Staffs The Health Facility staffs lives in the community where the facility is located .They are appointed for training by their community through the Kebele administrators but are being paid by the government . Supervision of CDTI activities is routinely carried out in an integrated manner alongside other programmes like Environmental sanitation, Malaria, Family Health home visit etc. FLHFs requests for Mectizan from the Woreda with a request letter written and stamped by the health facility staff. Copies of these were seen in all the Health Facilities visited. The CDDs receive their Mectizan allocation from the FLHFs based on the request calculated using updated census. CDDs usually go to the HFs either at home or in the office to collect more Mectizan or return the excess when they complete distribution. A record of Mectizan collected by each CDD was seen but details like Mectizan Batch number, Lot number, and expiring dates were not written in the record. Training is carried out routinely. FLHFs train CDDs annually in order to refresh their memory on CDTI. All the Health Facility Staffs visited, lacked training and need identification skill. No refreshment nor transport is provided for the CDDs when they come for training at the Health Post. The Health Facility Staff believe is not necessary since CDDs have accepted this work as a voluntary service to their community they eat at home before coming and their homes are not far from the Health post. The Kebeles support the training by ensuring that every community send their CDDs for the training .One striking observation regarding CDD's training in the Zone is the inclusion of construction of measuring sticks in the training topics. This was evident in the communities as CDDs showed evaluators measuring sticks of different designs made with local materials but accurately calibrated. HSAM activities are usually done routinely. Community awareness of the benefits of Mectizan is high and this has led to high treatment compliance. However new issues like amending distribution time to accommodate migrant farmers, or sustained compliance when signs of Onchocerciasis disappears are not addressed .HFs lack skill to identify what the objective of any round of HSAM should be. The health post don't manage budget. The FLHF workers do not get additional field allowance from Government they only depend on their salary which according to them is reasonable. This comment by one of the HFs summarizes this observation; "ln managing our Health Post, most of the materials we use are provided by the Woreda, we only use it as instructed by them to serve our people, because is our community that appointed us. Little things like pen or pencil for our activities we can buy from our salary - Alganesh Degago, Head of Loko Health post -Guto Gida, Woreda. Evaluators were also informed that when the Woreda calls them for additional meetings outside their routine activities, their transport cost is usually reimbursed. This GovernmenUCommunity system used in managing the Health Facilities helps the HFs function effectively. 7There is also no transport provision at this level since all health staffs lives in the community where their facility is located. They go on foot in carrying out their activities since activities are integrated and communities they serve are quite close. When they go to the Woreda for other health activities they equally transmit CDTI data. Training/HSAM materials are available. They are provided by the Woreda. Training/HSAM was sufficient and already translated in the local language. Front Line Health Facility staffs managing the Health Facilities are quite stable as they were appointed by their communities. They are efficient and hard working. Evaluators were informed that one of the criteria for their appointment is that they remain and continue to work for the community even after marriage. ln all the FLHFs visited, geographic coverage for the last three years is 100% and the therapeutic coverage is greater than 65%. Woreda Level At the Woreda level sustainability potential was found to be high with a score of 3.3 using the 9 groups of indicators At the Woreda a CDTI plan which is part of the overall integ-rated annual health plan exists. CDTI activities are integrated more closely with Communicable disease like Malaria, TB, Polio and other infectious diseases at the two Woredas visited. The Onchocerciasis focal persons are in charge of all communicable disease at the Woredas visited. According to head of the health office, planning is participatory involving all key stakeholders and approved by the Woreda council, ilo*er"r Evaluators did not find evidence to confirm partners' participation. Managers at this level said their staff combines two or more tasks on a single trip and also combines activities with other health programs. But no written evidence was seen to confirm how CDTI is implemented in an integrated manner with other health activities The Woredas initiates key CDTI activities evidenced by the presence of a detailed timetable stating the time that each CDTI activity will take place. There is a focal person in charge of CD1t in the Woredas visited. These focal persons are also responsible for other health activities of their respective Woredas. The CDTI activities are supervised and reported through the government reporting system. CDTI supervision checklist was seen but is not integrated with other health activities and there is no written supervision report. Mectizan orders are based on needs and requests from FLHFs and it is also based on the census report of CDDs. Normally, the Woreda Health Office collect Mectizan from Zonal Health Desk. Mectizan is available on time and adequate. No shortages were reported in the 2009 distribution, But in 2008 there was little shortage of Mectizan in both Woredas and was resolved by getting additional tablets from the zone. Mectizan collection, storage and delivery to lower level is within government system and it is simple, effective, uncomplicated and integrated. Transportation cost is covered from the government as Mectizan is transported in integration with other drugs. 8Training at Woreda is conducted routinely and not targeted at specific needs of the staff. Training topics does not seem integrated with other health activities. Training is given to both new and old staff every year. Political leaders are well oriented about the CDTI Program and they claim to be supporting the programme using the pool system. There was however no documentary evidence to show Governments funding for specific CDTI activities from the Health desk pool fund. lncome statement and ledger showing the amount of money transferred for the program from APOC and Light For the World as well as expenditures were seen at Zonal Finance unit. Bank Statements were also seen as well as retirement record of disbursed fund. The management is aware that APOC funding will stop after five years. But there is no evidence that the management have initiated any sort of plan to mobilize funds from dependable sources. They are confident that since CDTI is one of the health activities, the program will continue without interruption using government. There are about 8 motorcycles available for integrated health programmes at the two Woredas visited, two of which were provided by APOC five years ago. The APOC motorcycles are however getting old and not functioning effectively due to difficult train and rough roads Government regulations permit that the government budget be used for the maintenance of donated capital equipments, accordingly, the motorcycles seem to be maintained on a regular basis. The Woreda Health Offices approves movements of the motorcycles. All available transportation resources are put together in a pool to be used for all health related program. Trips are properly authorized by the Woreda Health Offices. The control mechanism is strong. According to the Woreda Health desk head, the Woreda is unable to replace the motorcycles because government budget allocated for the health activities is very minimal. Trained staff available for CDTI activities at this level are committed but inadequate and do not have enough skills on need assessment identification, targeted HSAM, data management /Reporting. The two Wereda has achieved and maintained 100% geographic coverages over the past three years. Average therapeutic coverage over the past three years was 81%. Zonal Level At the Woreda level sustainability potential was found to be moderate and had the least score of 2.8 using the 9 groups of indicators. The Zonal Health Unit has a written detailed integrated health plan with a section on Onchocerciasis. ln addition there is a separate plan for Onchocerciasis Control activities. Which was said to be is reviewed annually with partners. Minutes of planning meetings were however not seen. The pool system introduced by the Government in year 2004 is used in planning and implementation of all health programmes. The pool system is a policy for integrated use of money, vehicle and equipment .This year the Business Process Re-Engineering 9(BPR) which is integration of manpower for effective and efficient implementation of health activities was added to give an integrated package for Health. All Partners roles are clear and well defined at this level. The key partners identified were APOC, the NGDO (Light for the world), NOTF and the Zonal Health Unit. Sustainability plan have not been developed for post APOC period. The Head of the Health desk at the zone believes that since CDTI is an activity implemented in an integrated manner in a government system, there is no doubt about its sustainability. Theie remains a concern that with pool funding method of budgeting and fund disbursement and without a detailed Post APOC, sustainability plan, CDTI may receive very little support compared to what it will require to implement its very important activities. lntegration is said to be taking place at this level in the implementation of CDTI activities as well as other health programmes, however no written reports providing what was achieved in an integrated manner for all the programmes involved was seen. Leadership at the Zonal level are aware of the CDTI process .They equally have information on the progress and success achieved, but new emerging issues that is identified with effectlve supervision is unknown by the Zonal leadership. Delegation of duties is difficult because they do not have enough staff' CDTI activities are supervised by the Zonal focal person. Monitoring and supervision activities are carried out using supervisory checklist. Filled out supervisory checklist were seen, but there was no supervisory report. Filed copies of APOC technical report for the past two years were equally seen. The zone lost soft copies of their summary report for the first 2 years of programme implementation due to computer virus. Detailed inventory of Mectizan as well as fund management records were not with the Onchocerciasis focal person because Drug and fund management for the whole zone is handled centrally by another unit. Mectizan is managed by the Pharmacy department and Funds by the finance department. Mectizan supply is controlled within government system. Since the inception of the CDTI programme, the Zonal staff facilitates training of the Woreda Staff as well as orientation of a new staff posted to the health unit that might be involved in CDTI activities implementation. Training is carried out routinely at the Zonal level and for the Woredas. lt is not based on need assessment and not targeted. Staff at this level and the level below believes that the purpose of training is to refresh their memory. The Zonal level managers believe they can't do much about advocacy as their Health package budget is fixed and for any amendment to be done, it will have to involve advocicy at the higher level. At the lower level they belleve they can advice the Woredas administrators to give Onchocerciasis a priority attention. Other HSAM activities are carried out routinely, the reason for this is because Health managers do not have sufficient skill to identify new Health Education focus. Secondly they do not have enough manpower to carry out spot check supervision and identify these needs. lt therefore makes it difficult to know the new focus for HSAM. 10 Amount budgeted and released by APOC and Light for the world for Onchocerciasis activities were seen with the Zonal Accountant. Government Budget details for the pool financing of Health activities including CDTI were not seen at the Zone. APOC and Light for the World letters of agreement with the Zone were equally unavailable .They said they are yet to receive fund from Light for the World and APOC for year 2009. There was no evidence to show that Zonal managers are aware of the total fund that will is available to him this year and what they hoped to get next year. Therefore cost reduction /containment strategy is difficult. The zone has one project vehicle donated by APOC. lt is about 5 years old and still functional but according to the Zonal Focal person for Onchocerciasis is lnadequate for the 9 Woredas they have to supervise. The health unit has 3 additional vehicles which can be borrowed for CDTI activities. They also have a one desk top computer, one printer and one Xerox photocopier. All of these are about 5 years old and functional. The only challenge they have is with the photocopier, which they complained that its ink is very expensive to replace. There is no written replacement plan for transport and other materials resources. There is equally no written commitment from any partner to replace the vehicles or other material resources. The reason for this is Poor programme management skills and the believe that Government has a very lean budget and unable to replace the vehicles or motorcycles. Due to the BPR system adopted by the Government, Staffs in the health unit are now in pool and are involved in all the health issues in the unit. However there are two staff dealing more with CDTI, the Focal person and his assistant. They are committed and quite stable as they have been on the job for the past three years They are overloaded with different activities and therefore could not perform efficiently the need assessment and data management activities needed for CDTI activities. The zone has had good geographic coverages over the past three years. ln Year 2008, Geographic coverage was 99.7o/o.This was due to shortage of drugs in two Woredas. Year 2006 and 2007 geographic coverage was 100%. Average zonal therapeutic coverages was 74.5 for the past three years.73.4% in Year 2008, 74.60/0 in 2007 and 75.4o/o in 2006. Conclusion Based on the seven aspects of sustainability and in line with the guideline for grading the whole project using the five critical elements of sustainability ,the Evaluation Team concludes that the East Wollega CDTI project is MAKING SAITSFACTORY PROGRES S TOWARDS SUSTA N ABI LITY. The quantitative score of 3.3 for the pooled groups of indicators at the four levels supports the above qualitative score. 11 Way forward: As the East Wollega CDTI project is almost completing its fifth year technically there is the need to improve on the project's Efficiency and availability of Resources. 1. EFFICIENCY documentation. The project needs to be more efficient in Training, HSAM and A) Training: There is the need to ensure that training activities at all levels are carried out in an efficient manner. Training should be targeted. Based on need assessment identified using competency /skill monitoring strategies like pre/post test ,supervision, evaluation of reports etc. The training should therefore address that need and report of such training should be documented. Training on CDTI for all the Health Staff in the pool should also be considered in view of the BPR scheme. B)Health Education, Sensitisation,Advocacy and Mobilisation(HSAM) The Zone need to intensify advocacy at all levels in order to mobilize enough resources to sustain this CDTI project. The sustainability plan which has been prepared is an available tool that can be utilized. Health Education should be targeted to meet information and knowledge gaps. E.g. High number of refusals and Absentees in an area is a possible indication of Mectizan benefit information gap. Other gaps can be identified through effective supervision and spot checks. C) Documentation Documentation at the Zone is poor. lt is very important therefore that the Zone update its technical and financial records, Efforts should be made to ensure that this updating is completed before December 2009.Copies of the unavailable records should be requesied for from the relevant partners. Documentation will enhance the Zones efficiency especially in monitoring their progress and planning. 2. Resources: Human and Financial resources available at the Zone is inadequate A) Human Resources: The available trained human resource at the Zone and Woreda is insufficient for efficient management of CDTI activities. There is also capacity gap in the area of reporting and need assessment among the available ones. The Zone should therefore increase available human resource to improve their capacity. B) Financial Resources: The Zonal Government should fund CDTI activities by increasing its budgetary allocations and ensuring timely releases for its implementation. Evidence of Government actual release from the Health budget for CDTI activities should be clearly stated and documented. 12 TABLE OF CONTENTS Acknowledgements- Abbrevi ati ons/Acro nyms-------- Executive summary- ----1 Table of Contents------------ 1.0 INTRODUCTION---- -----------------14 2.0 METHODOLOGY---.- 2.1 Objective of the Evaluation 2.2 Sampli 15 2.3 lnstrument used for Evaluation 15 2.4 Sourcesof lnformation--------- 16 2.5 Analysis-- 2.6 Constraints------------ 16 3.0 EVALUATTON F|ND|NGS------- -------18 3.1 Sustainability at project (zonal level)------ -------18 3.1.1 Recommendation for the Zonal Level------ 23 3.2 Sustainability at the Woreda Level---- -- -26 3.2.1 Recommendations at the Woreda Level------ 3.3 Sustainability at the Front line Health Facility---- ------------32 3.3.1 Recommendation at the Front line Health Facility Level------ ----------35 3.4 Sustainability at the Community Level------------- -------------37 3.4.1 Recommendation for the Community Level---- ----------------40 4.0. Comparative Analysis of the Sustainability of the Four Levels----- 1 4.1. A quantitative judgement, based on the grades given to individual indicators--41 4.2. Qualitative judgment of project---- 4.3. Judgment using the five key aspects of the project -----44 13 4.4 Conclusion 5.0 FeedbacUPlanning Meetings Annexes -----------------46 49 , 14 1.0. INTRODUCTION East Wollega is one of the 17 zones of Oromia Regional State. lt is located in the western part of the country. The capital town of the zone is Nekemte, which is 331 kilometers from Addis Ababa (capital city of Ethiopia) .The catchment areas of East Wollega zone extends from Gibe river to Didesa river having with varied topography. East Wollega zone is bounded by Amhara regional state and Horo Gududru Zone in the North, West Shoa in the East, Jima zone in the South East, lllubabor in the South West, West Wollega zone and Benshangul Gumuz regional State in the West. The altitude ranges from 500 to 2600 meters above sea level. Agro ecologically, it is divided into lowland, middle land and highland areas. Annual rainfall varies from 1200-2500mm. The temperature reaches a daily maximum of 28 degree centigrade. There are four big rivers in the zone , namely Gibe which is between West Shoa and East Wollega zones, Didesa, Anger and Uke. The zone has many other small rivers and streams that drain into the basin.The climate is characterized by distinct rainy and dry seasons. The rainy season is from June to September. Harvesting is from November to January. The best time for distribution is in the months of February to March. There is good telecommunication access and fairly good road to the districts and their capitals. Some villages can be reached on motorcycle while others are only accessible on foot or on the back of animals. According to2007 census, the total population of the zone is 1,175,853. Among this, a total of 782,236 persons are at risk of Onchocerciasis in the CDTI area. Likewise, 11% of the populations are urban and 89% are rural residents. The districts of the zone arc sub divided into 367 kebeles, of which 339 are rural and 28 are urban. The kebeles are further divided into Gare (village).Each village consists of 30-50 households. Villages are responsible for all developmental activities including health.East Wollega CDTI projects has 10 districts (Woredas). They are Jima Ario, Leka Dhulecha, Diga, Guto Gida, Wayu, Tuka, Sasiga, Sibu Sire, Bako Tibe,Boneya Boshe and Wama Hagalo. ln line with this, the CDTI project areas consist of 4299 villages and 8478 Community Directed Distributors (CDDs). The East Wollega CDTI project launched around November 2004 but actual reporting on distribution started in Year 2005.The 2007 updated census shows that the Zone has a total population of 1,175,853. The districts of the zone are sub divided into 367 kebeles, of which 339 are rural and 28 are urban. The kebeles are further divided into Gare (village).Each village consists of 30-50 households. Villages are responsible for all developmental activities including health East Wollega CDTI project has 10 districts (Woredas). They are Jima Ario, Leka Dhulecha, Diga, Guto Gida, Wayu, Tuka, Sasiga, Sibu Sire, Bako Tibe,Boneya Boshe and Wama Hagalo. Recently the Bako Tibe Woreda was moved from East Wollega Zone to become part of West Shoa Zone. 15 2.0. METHODOLOGY 2.1. OBJECTIVE OF THE EVALUATION 2.1.1. The general objective for the evaluation exercise was to determine the sustainability potentials of the East Wollega CDTI project by its third year of operation and assist in developing a plan for sustaining the project post-APOC. 2.1.2 .The specific objectives are. To assess the performance of the different groups of indicators of sustainability of CDTI projects in the East Wollega CDTI project To identify the factors that may block or help the sustainability of the project. Discuss the outcomes of the evaluation exercise with the relevant stakeholders in the East Wollega CDTI Project Develop plins for sustaining the East Wollega CDTI project post APOC 2.2. SAMPLING The East Wollega Zone has a total of 10 CDTI Woredas including Bako Tibe which was recenly becamL part of West Shoa Zone. Thus, two Woredas, 4 Health Facilities and 12 villages were randomly selected from all the 10 Woredas .One additional Woreda *", "qirlly randomly selected as a reserve one. The Woredas, Health Facilities and Villages selected are as shown below: Table 1: Summary of selected Woredas, FLHFs and Villages of East Wollega CDTI P 2.3. INSTRUMENTS USED FOR THE EVALUATION The evaluation was conducted using four types of instruments developed by APOC lnstrument 1 : For National and Project Level lnstrument 2'. For Woreda/ District Level lnstrument 3 : For Front Line Health Facility Level lnstrument 4'. For Community Level VillageFLHFWoreda Wanebo Legetsebela Tumee Bedo Health Post Mayibasi Kambi Lafto Kewissa Health PostLeka Dulecha Haro Jeni Daleti Kitiesa Health Post Burka Woligelti Derartu Bua Oda Loko Health Post Guto Gida 16 lnstruments were used at the 4 levels to access the projects performance regarding the routine activities and processes. The following were the lndicators used; Planning, lntegration, Supervision and Monitoring, Mectizan supply, Training and HSAM lndicators used to check for resources provided for the above activities were financing / Funding, Transport and other material resources as well as Human Resources. Result achieved were measured using the Coverages, both Geographic and therapeutic. The performance of the indicators as well as the various aspects of sustainability (lntegration, Resources, Efficiency, cost-effectively, Simplicity,Health staff acceptance (Attitude of the health staff and Effectiveness were used in grading the performance 2.4. Sources of information Documentary evidence from CDTI data, reports and plans lnspection of capital Equipments Verbal reports from persons interviewed. Community Meetings Additional insight during feedback meetings 2.5. ANALYSIS At the end of field activities, the information collected on each indicator was graded on a scale of 0-4 (worst to best), in terms of its potential contribution to sustainability. The average'sustainability score' for each group of indicators was calculated, for each level, and a graph was plotted. Summary statistics for the scores were calculated for each level, and for each group of indicators, tables and graphics of these results were presented at feedback workshop. The quality of the overall project was also assessed using the different aspects and critical elements of sustainability present in the project. The five critical elements and the seven aspects of sustainability in the project were qualitatively discussed and results agreed to by the team. The project was graded using these aspects and elements following the Evaluation guidelines. Qualitative descriptions of problems were deliberated upon and recommendations were made. The sustainability judgment about the sustainability potentials of the project was therefore based on the quantitative assessment of the average sustainability scores of the groups of indicators as well as the qualitative assessment of the critical elements and aspects of sustainability of the project. 2.6. CONSTRAINTS The following were some of the major challenges faced by the team: lnsufficient Number of Evaluators: The East Wollega team had three evaluators instead of the expected six. This made it difficult to have a sub team and increased the workload of the available team members. Translation issues: The team members from Addis could speak Armaric which is an official language in Ethiopia but in the Zone the official language was Oromifa which they could not understand and couldn't speak. This made the team involve more local guides during data collection. 17 Power Failure: For some days during the Evaluation there was no power for data entry even in the hotel. This slowed down data entry. 2.7. TEAM COMPOSITION S/No Name Designation Address 1 Mrs Olamiju Francisca Executive Director, M ITOSATH/N igeria-Team Leader MITOSATH 605, Hospital place, opposite Green valley Suites, GRA,Jos Plateau State, Nigeria. m itosath@hotmail. com. +234-803331 8085 2 Abate Tilahun Programme Manager, Carter Center, Addis Ababa Ethiopla-Team member +251-911462483 abate_tilah u n@yahoo. com 3 Firew Ayalew Data Manager Carter Center, Addis Ababa Ethiopia-Team member +251-911462483 Firewa@yahoo.com 18 3.0 EVALUATION FINDINGS 3.1 Sustainability at Project (Zonal) Level Figure 1. Average Performance of Each group of indicator in the entire Project (Zonallevel) 4 4 4 ?( .9 o =oE'(E o a 2.s 2 1.5 1 0.5 0 .**e."-"C-"'t.."" $d oC "."t. ."s"".**""^o"d Group of indicators Planning (moderately 2.31 fhe Zonal health unit has a written detailed integrated Health plan with a section on Onchocerciasis. ln addition, there is a separate plan for Onchocerciasis Control activities. The Health plan is reviewed annually. Monthly and quarterly reports were not seen. The head of the Health desk said planning is done annually by the Health Committee .Minutes of planning meetings were however not seen. The pool system introduced by the Government in year 2004 is used in planning and implementation of all health programmes. The pool system is a policy for integrated use of money, vehicle and equipment .This year the Business Process Re-Re-Engineering (BPR) which is integration of manpower for effective and efficient implementation of health activities was added to give an integrated package for Health. All Partners roles are clear and well defined at this level. The key partners identified were APOC, the NGDO (Light for the world), NOTF and the Zonal Health unit. According to the Zonal Onchocerciasis focal person he said; "We ensure that all our partners are involved in the annual planning of CDTI activities because it is a partnership" However minutes of the planning meetings were not seen to confirm this claim. Light for the world employed a focal person for Onchocerciasis, who is based in Nekemt.His main responsibility is to provide technical assistance to the Onchocerciasis focal persons for the East and west Wollega projects. 19 Sustainability plan have not been developed for post APOC period. The Head of the Health desk at the zone believes that since CDTI is an activity implemented in an integrated manner in a government system ,there are no doubt about its sustainability. There remains a concern that with pool funding method of budgeting and fund disbursement and without a detailed Post APOC, sustainability plan, CDTI may receive very little support compared to what it will require to implement its very important activities. lntegration (HighlY 3.5) lntegration is tat<ing place at this level in the implementation of CDTI activities. According to the Onchocerciasis Focal person for the zone he said field outing like training is equally used for activities like HSAM. ln addition integration is taking place in the implementation of other health programmes in the zone. The opportunity of being in the field for Onchocerciasis activities for instance is equally used to implement other health programme like lvlalaria, Family Health or Environmental sanitation activities. The Head of the Health desk said; "We most times release funds for monitoring and supervision of communicable diseases. During this activity supervision is done by responsible officers in the unit for more than 3 or 4 diseases. lt is the pool system joined with the new government policy called BPR.' No written detailed activity work plan and reports were seen to show the effectiveness of the pool system. Leadership (Highly 3.0) Leadership ai tfrL ional level is effective and aware of the CDTI process .They equally have information on the progress and success achieved. The leadership is aware of some of the problems but new emerging issues that is identified with effective supervision is unknown by the Zonal leadership. Delegation of duties to other staff at the Zonal level is difficult because they do not have enough staff but active delegation takes place at the lower level. Monitoring and Supervision (Highly 3.1) The Zonal focal person supervises activities at the Wereda level. Hard and soft copies of treatment data and activity reports from the different Wereda for the past 2 years were seen .Filed copies of APOC technical report for the past three years were equally seen. The zone lost soft copies of their summary report for the first 2 years of programme implementation due to computer virus. betiiteO inveniory of Mectizan as well as fund management records were not with the Onchocerciasis focal person because Drug and fund management for the whole zone is handled centrally by another unit. Mectizan is managed by the Pharmacy department and Funds by the finance department. Zonal staff supervises mainly the Wereda activities and have already empowered the lower staff to supervise the activities at their level and the levels below them. Supervisory visits are usually carried out during Mectizan distribution. lt is however difficult to estimate the number of visits because supervision of health programmes is carried out in an integrated manner using resources available to the Health unit. 20 Monitoring and supervision activities are carried out using supervisory checklist. Filled out supervisory checklist were seen, but there was no supervisory report. Problems identified using monitoring system are addressed as they arise by the Zonal managers, some are identified by them and others passed on to them by the lower level. Successes are noted and staffs are commended verbally. According to the Head of zonal Health desk, most of the staff identified to be doing well are usually appointed to represent the Zone at meetings or recommended for a course. Those not doing well are usually rebuked and encouraged to improve. Mectizano Procurement and Distribution (Highly 3.5) Mectizan supply is controlled within government system. The Zonal Onchocerciasis focal person collects the drugs from the National Onchocerciasis office of Federal Ministry of Health of Ethiopia and hands it over to the Zonal Pharmacist. The resources used in the collection of Mectizan are from Government because they combine Mectizan drug collection with the collection of other drugs allocated to their Zone. The Zonal pharmacist is in charge of the release of Mectizan to the different Woredas with approval from the Zonal Onchocerciasis focal person. A very efficient inventory system called stock order is used for Mectizan flow .Stock order module 19 and 22 exist at all levels including the Health Facility. At every level stock order module 19 is used to receive Mectizan and stock order module 22 is used to issue it out. The stock order however does not capture the batch and Lot numbers as well as expiring dates. When the Zonal Pharmacist was probed further his response was; "lt takes a lot of time to begin to enter those details but I usually check the expiration dates". Leftover Mectizan is usually returned by the CDDs through the Health Facility to the Woreda and then handed over to the Zonal office. The Pharmacist then receives it from the Zonal Onchocerciasis Coordinator and enters it in his record against the returning Woreda. There was shortage of Mectizan during Year 2007 distribution due census problems but was re solved by getting left over drugs from other Zones. Training & HSAM (Moderately 2.0) Since the inception of the CDTI programme, the Zonal staff facilitates training of the Woreda Staff as well as orientation of a new staff posted to the health unit that might be involved in CDTI activities implementation. Lower level personnel have already been empowered to conduct training at their levels and the levels below them. Training is carried out routinely at the Zonal level and for the Woredas. lt is not based on need assessment and not targeted. Staff at this level and the level below believes that the purpose of training is to refresh their memory. Training is fairly integrated as staff at this level claim to train on CDTI during other programme trainings. There was however no evidence to prove this. 21 At the Zonal level managers believe they can't do much about advocacy as their Health package budget is fixed and for any amendment to be done, it will have to involve advocicy at the higher level. At the lower level they believe they can advice the Woredas administrators to give Onchocerciasis a priority attention .Presently this is not yet actively implemented. btner HSAM activities are carried out routinely, the reason for this is because Health managers do not have sufficient skill to identify new Health Education focus. Secondly they do not have enough manpower to carry out spot check supervision and identify these needs. lt therefore makes it difficult to know the new focus for HSAM. Obvious Health Education issues like community poor understanding of key CDTI messages written in English and Armaric was identified. According to the Zonal focal person, "We have addressed the problem because our CDTI booklet and posters have been translated into Oromifa language with APOC funds and my people like it a lot, you will see for yourself when you go to the field" Evaluators confirmed this when communities were visited. Some community leaders even showed us their copy which they kept very well and use in sensitizing their communities. Fi nance (ModeratelY 2.01 ln the Zone, amount budgeted and released by APOC and Light for the world for Onchocerciasis activities were seen with the Zonal Accountant. Budget details and letters of agreement were unavailable at the Zone and was said to be with World Health Organization office in Addis Ababa. According to record available at the Oromia regional Helttn Bureau the sum of $30,754 was approved by APOC support for support of CDTI activities in East Wollega Zone for year 2009.out of this $21,528 was said to have been released to the field but as at Evaluation time, the project was yet to receive the fund. Similar delays were encountered regarding funds released by Light for the world through the Oromia region. The delay was said to be due to some bureaucratic procedures at the region which partners are aware of and making effort to address. Lvidence of funding for CDTI activities from Government was not seen. lt was said to be embedded in a pool called Health Budget. And according to the Zonal head it is difficult to state. His Comment; "sometimes the Health unit will release funding for Monitoring and supervision of Health activities (Malaria, TB, Onchocerciasis)how do i then tell you how much of that fund will be charged to Onchocerciasis? lt is difficult. lt is Government policy. We can't change it'. There was no evidence to show that Zonal managers are aware of the total fund that will is available to him this year and what he hopes to get next year. Therefore cost reduction /containment strategy is difficult. There is no evidence to show Government Contribution to CDTI activities. There is however claim that they are using the Health Budget in an integrated manner to cover CDTI expenses like fuelling and maintenance of vehicles. There is no written document to enable an assessment of budget deficit and how to address it. lt has been like that from the programme inception till date. 22 At the zonal finance department requisitions and approvals signed by the Head of the Health Zone were seen. Ledgers and retirements were seen as well. But it was difficult to know whether or not budget lines were followed because budget breakdown and letters of agreement were unavailable. Transport and other Material Resources (Moderately 2.3) The zone has one project vehicle donated by APOC. lt is about 5 years old and still functional but according to the Zonal Focal person for Onchocerciasis is lnadequate for the 9 Woredas they have to supervise. The health unit has 3 additional vehicles which can be borrowed for CDTI activities. This he said is only available when the programmes that owns them are not in the field. They also have a one desk top computer, one printer and one Xerox photocopier .All of these are about 5 years old and functional. The only challenge they have is with the photocopier .They complained that the photocopiers ink is very expensive to replace, so they have not been using it .According to the head of the Health desk; "We like the photocopier because it can print on wider papers but the ink is expensive, we will appreciate it if APOC can give us another that is more universal and cheaper to maintain and use, that will complement this one." The zone believes that Government budget is too small to buy capital equipments and therefore rely on donors. But they are committed to efficient use and good maintenance using government funds. The Head of Health desk, said the zone in view of sustainability, will consider suggesting to Government to buy some of these needed capital equipments as from next year and see if it will be successful. Capital equipments donated by APOC are well maintained using Government funds. When maintenance issues are identified, it is usually reported to the head of the health desk who then issues instruction for it to be repaired using zonal funds. No maintenance record was seen but all APOC donated capital equipments seen were still functional after 4 years. The four vehicles that the health desk has are all functional .The management of all the vehicles is the responsibility of the zonal head of the health desk. Vehicles are authorized for use with an authorization letter and an outlet paper. The outlet paper according to the Onchocerciasis focal person shows where the vehicle is going to, purpose, no of days, no of occupants and stipulated return date. All the vehicles are in a pool and assigning depends on the activity to be done and vehicle availability. lf there is a diversion in the authorized use of the vehicle appropriate disciplinary action is usually taken against the defaulting staff. Copies of trip authorization letters and log books were however not seen. There is no written replacement plan for transport and other material resources. There is equally no written commitment from any partner to replace the vehicles or other material resources. The reason for this is poor programme management skills and the believe that Government has a very lean budget and unable to replace the vehicles or motorcycles. 23 Human Resources (HighlY 3.0) Due to the BPR system adopted by the Government, Staffs in the health unit are now in pool and are involved in all the health issues in the unit. However there are two staff dealing more with CDTI, the Focal person and his assistant. They Jre overloaded in different activities and it is difficult for them to be very effective and efficient in CDTI. They are quite stable as they have been on the job for the past three years. They are "quitty committed. However, they could not perform properly the need assessment and data management activities. Coverage (Fully 4.0) The zone has a good geographic coverage over the past three years. ln Year 2008, Geographic coveiage was -99.7%.fhis was due to shortage of drugs in two Woredas (DigJ with geograpnic coverage of 99.3 and Wayu Tuka with geographic coverage of 98.8%).The ,eison for the shortage was incorrect census. According to the Onchocerciasis focal person, the Census has since been updated and enough Mectizan supplied to them during year 2009 distribution. Year 2006 and 2OO7 geographic coverage was 100%. Therapeutic coveragL has been good, above 65%. Average zonal therapeutic coverages was 74.5 for the past three years.73.4% in Year 2008, 74.60/o in 2007 and 753% in 2006. 3.1.1 : Recommendations for the Project (Zonal) level Table 2:Recommendations for the Project (Zonal) level Recommendation lmplementation Planning: Develop a short and long term CDTI sustainability plan with cost reduction and containment strategies. Put in place proper documentation of past and present CDTI information like Treatment reports, supervisory visits, letters of agreement, Government contribution record etc Prio HIGH lndicators of success: Existence of CDTI sustainability plan Existence of complete CDTI information for effective planning and monitoring. Who to take action ZOFP,LFTW,WHO Deadline for comPletion July 2010 lntegration of support activities: Put in place documentary evidence to show how effectively CDTI is implemented in integration with other health activities. P HIGH lndicators of success: Availability of detailed record showing effective CDTI implementation in an integrated manner Who to take action ZHDH and ZOFP 24 Deadline for completion April 2010 Leadership: Ensure that the zone provide effective leadership for the project by having enough trained personnel that can support the implementation CDTI activities. Priority: HIGH lndicators of success: Availability of enough staff for proper supervision and effective delegation of CDTI activities. Who to take action ZHDH and ZOFP Deadline for completion October 2009. Monitoring and Supervision : Ensure that reports of Supervisory activities are written and filed properly .Appropriate feedback should equally be given to those concerned. Priority: It/lEDlUtM lndicators of success: Availability of report for supervision conducted and feed back given. Who to take action ZHDH and ZOFP Deadline for completion May 2010 Mectizan Supply: Mectizan ordering and distribution should be made based on updated census to avoid shortage. Zonal mangers should communicate to the lower level that CDD census updating is part of distribution activities. Effort should be made to record Batch and Lot numbers of Mectizan received at Zonal level as well as the expiration dates. Priority: HIGH lndicators of success: a) Adequate supply of [/ectizan at all levels. b) Availability of accurate census data at all levels for Mectizan ordering and distribution. Who to take action: ZOFP and the Zonal Pharmacist Deadline for completion April 2010 Training & HSAM Conduct training on need assessment and focus future training plans on areas of need. ldentify communities that need sensitisation and sensitise them Priority: HlGll lndicators of success: Availability of targeted training report. Reports of communities sensitised Who to take action ZOFP Deadline for completion October 2009 Finance The Zonal Government should fund CDTI activities by increasing its budgetary allocations and ensure timely releases for its implementation. Evidence of Government actual release from the Health budget for CDTI activities Priority: HIGH lndicators of success: a) lncreased amount budgeted and released for CDTI activities at the Zone. b) Documentary evidence of Zonal Government support to CDTI available. 25 should be available Who to take action: Zonal Administartor,ZHD Head and ZOFP Deadline for completion October 2009 Human Resource: lncrease the number of trained personnel available to support CDTI implementation in the Zone. Zonal project staff should be trained on need assessment, data management and documentation. Priori HIGH lndicators of success: a) Availability of sufficient trained Zonal team for effective CDTI implementation. Who to take action: Head of Zonal Health Desk Deadline for completion January 201 0 Transport and other Material Resource: Project can request APOC to consider the replacement of their old Zonal project vehicle, Motorcycles, desk top computer and photocopier. The Zone should begin now to put plan in place a Government plan to replace in the future some of the capital equipments for CDTI implementation. Effort should be made to estimate other materials needed like IEC/MIS materials, and ensure that they are adequately budgeted and sourced for from dependable sources before the project get to 6 P HIGH lndicators of success: a)Availability of replaced vehicle and other capital equipments. b)Availability sufficient IEC/MlS materials and other materials needed for CDTI activity implementation Who to take action: ZOFP,APOC [Vlanagement, ZHDH and LFTW Deadline for completion October 2010 Coverage Sustain 10oo/o geographic and achieve 80% therapeutic coverage by ensuring that REfvlO map, soft and hard copies of list of endemic communities and their populations are available for coverage data verification Priori HIGH lndicators of success: REMO map for the zone List of endemic communities Verified treatment coverage data Who to take action: ZOFP,NOTF and LFTW Deadl ine for completion October 2009 26 3.2 Sustainability at the Woreda Level Figure 2. Average Performance of Each group of indicator at Woreda level 4 4 4 og, .g o o!t o ,g) o = 4 3.5 3 )R 1 2 .5 1 0.5 U .*"e -dr-)""irr"f..""d -'c. ^.e"o"oo'"".0{./ ""r-* Group of indicators Planning (3.0): There is a CDTI plan which is included in the overall integrated annual health plan at two Woredas visited. CDTI activities are integrated more closely with Communicable disease like Malaria, TB, Polio and other infectious diseases. The Onchocerciasis focal persons are in charge of all communicable disease at the Woredas visited. The plan of the Woredas makes provision for all key CDTI activities and community requirements. According to head of the health office, planning is participatory involving all key stakeholders and approved by the Woreda council, However Evaluators did not find evidence to confirm partners' participation. lntegration (3.0): At this level there is integrated annual health plan where CDTI is incorporated. Though there is no written evidence, the staff combines two or more tasks on a single trip and also combines activities with other health programs. But there is no written evidence to confirm how CDTI is implemented in an integrated manner with other health activities Leadership (4.0): ln the two Woredas visited the management team at this level initiates key CDTI activities evidenced by the presence of a detailed timetable stating the time that each CDTI activity will take place. There is a focal person in charge of CDTI in the Woredas visited. These focal persons are also responsible for other health activities of their respective Woredas. Supervision/Mon itoring (3. 3) The CDTI activity reports are communicated through the government reporting system. The Communities report to the FLHFs and FLHFs to the Woreda Health Office. The Woreda Health Office then reports to Zonal Health Desk. But there is delay in submitting 27 reports especially from the lower levels (CDDs).The reports being sent contain all necessary data concerning CDTI activities. Financial report of all health activities is available with Wereda finance office. At this level supervision by the Woreda staff is carried out regularly during Mectizan distribution time. Supervision is conducted using supervisory checklist. The woreda supervisory visits mainly target FLHFs; however, spot check visits are sometimes made to the kebeles/communities. CDTI supervision checklist was seen. it is not integrated with other health activities and there is no written supervision report. problems observed during CDTI supervisory visits or obtained from other sources are addressed promptly in collaboration with the appropriate person in charge at all level. lf the problem is not solved by the responsible person in charge of that level, it will be reported to the next level for solution. Written reports were unavailable to confirm how success and feedback were handled. Mectizan Procurement and Distribution (3.4): Mectizan orders are based on needs and requests from FLHFs and it is also based on the census report of CDDs. Normally, the Woreda Health Office collect Mectizan from Zonal Health Desk. tt/tectizan is available on time and adequate. No shortages were reported in the 20Og distribution, But in 2008 there was little shortage of Mectizan in both Woredas and was resolved by getting additional tablets from the zone. Mectizan collection, storage and delivery to lower level is within government system and it is effective, uncomplicated and integrated. Woredas collect their tvlectizan supply from zonal level. A Government inventory module called module 19 for receiving and module 22 for issuing out is used at this level for drug and equipment control including Mectizan. Trans.-portation cost is covered from the government as Mectizan is transported in integration with other drugs. Training/HSAM (3.5): Training for both Woreda Health staff and FLHFs workers is given by Zonal Health Desk. The reason the zonal level staff gave for this is for making the training strong' But in 2008 training for Woreda Health staff was given by the Zonal Level health staff and the FLHFs staff were trained by the woreda Health Staff. Trainings are conducted routinely and not targeted at specific needs of the staff. Training topics does not seem integrated with other health activities. Training is given to both anO otO staff every year. The trainings focus on signs and symptoms of Onchocerciasis, lts Epidemiology, and Treatment, role of CDDs and supervisors and Census updating. No training module was seen at any of the Woredas visited. Political leaders are well oriented about the CDTI Program and they are supporting the program and are involved in HSAM activities. HSAM activities seem targeted and effective though there is no written report. Financial resources (2.0): lncome statement and ledger showing the amount of money transferred for the program from APOC and Light For the World as well as expenditures were seen at Zonal 28 Finance Unit. Bank Statements were also seen as well as retirement record of disbursed fund. Documentary evidence of Government release of fund for CDTI activities were not seen but the responsible persons claim that since CDTI is included in the overall annual health activity plan it has mandate to utilize funds allocated for the health activities in an integrated manner with other health activities especially for fuelling and maintenance of motorcycle. The management is aware that APOC funding will stop after five years . But there is no evidence that the management have initiated any sort of plan to mobilize funds from dependable sources. The management is confident that since CDTI is one of the health activities, the program will continue without interruption using government However there is no written commitment or plan yet to support the above claim. For all CDTI related activities at Woreda Health Offices, expenditure approval is given by the head of Health office. There is a strong central control of finance by the Woreda Finance Office. All the support given from Zone to Wereda for CDTI activities is managed centrally by the Woreda finance office. Finance office clearly record allocation of fund. When ever there is income and expenditure the accountants calculate it immediately and record the balance amount. The finance office is transparent. Transport and Other Materials (3.0) There are about 8 motorcycles available for integrated health programmes at the two Woredas visited, two of which were provided by APOC five years ago. The APOC motorcycles are however getting old and not functioning effectively due to difficult train and rough roads. Government regulations permit that the government budget to be used for the maintenance of donated capital equipments, accordingly the motorcycles seem to be maintained on a regular basis. The Woreda Health Offices approves movements of the motorcycles. All available transportation resources are put together in a pool to be used for all health related program. Trips are properly authorized by the Woreda Health Offices. The control mechanism is strong. Though the management at this level are fully aware of the necessity of replacing the existing motorcycles there is no written realistic plan so far made to replace them. According to Head of the Heath Office, since motorcycles available for the Woredas are in a pool to serve all health activities CDTI will not be left out. The Woreda Health Office is unable to replace the motorcycles because government budget allocated for the health activities is very minimal. Human resources (3.0) Though there is no guaranty for staff to stay in their position for long, the Onchocerciasis focal person and his assistant has been in their position for the last two years. Staff members responsible for CDTI at this level are committed and willing to serve their people. And staff salary is paid regularly every month. They are satisfied by the commendation they get from the beneficiaries. But no one has been rewarded because of CDTI activities. 29 Trained staff available for CDTI activities at this level are inadequate and do not have enough skill on need assessment identification, targeted HSAM, data management /Reporting. Coverage (4.0) The two Wereda has achieved and maintained 10O% geographic coverages over the past three years. Average therapeutic coverage over the past three years was 81% Lecha Dulecha therapeutic coverage was 82 % in year 2008 while Guto Gida had therapeutic coverage of 73o/o.ln 2007,Leka Dulecha had 80% therapeutic coverage and Guto Gida had 81% and in 2006, Leka Dulecha and Guto Gida had 84% and 860/0 therapeutic coverages respectively. 3.2.1: Recommendation for the Woreda/district Level Table 3: Recommendation for the Woreda/district Level: Recommendation lmplementation Planning: The LGA plan should be reviewed along with relevant partners in order to address areas of weaknesses and the Evaluation recommendations. P H IGH lndicators of success: a)Availability of revised CDTI plan b) Availability of minutes of partners planning meeting Who to take action WOFP, WHDH Deadline for completion : October 2009 lntegration of support activities: Document how CDTI is implemented in integrated manner with other health activities as indicated in the lntegrated Annual Health Plan of the Woreda Health Office Priority: HIGH lndicators of success: Availability of reports health activities that implement CDTI. on WAS integrated used ti Who to take action WOFP, WHDH Deadline for completion April 2010 Leadership: Ensure that the Woreda Provide effective leadership for the project by putting in place enough trained personnel that can suPPort the implementation of CDTI activities. Prio HIGH lndicators of success: Availability of enough staff for proper supervision and effective delegation of CDTI activities. Who to take action WHDH ,ZOFP Deadline for completion October 2009. Monitoring and Supervision: Priority: MEDIUM 30 Monitoring and supervision reports should always be written using the information obtained from the checklist. These reports should be properly filed and used for follow up, feedback and performance review at the Woreda. lndicators of success: Availability of written monitoring and supervisory reports. Who to take action WHDH, ZOFP Deadline for completion April 2009. Mectizan Supply: Mectizan ordering and distribution should be made based on updated census to avoid shortage. Woreda Onchocerciasis focal person should communicate to the lower level thatCDD census updating is part of distribution activities. Effort should be made to record Batch and Lot numbers of Mectizan received at Zonal level as well as the expiration dates. Priority: HIGH lndicators of success: a) Adequate supply of Mectizan at the Woreda b) Availability of accurate census data at Health Facility and Community levels for Mectizan ordering and distribution. Who to take action: WOFP and the Woreda Pharmacist Deadline for completion April 2010 Training and HSAM: The skill of Woreda staff involved in CDTI implementation need to be built on how to carry out need assessment for training and HSAM. The FLHFs should be empowered to be able to address in an integrated manner training and HSAM needs at their level and the community.. Training and HSAM should not be carried out routinely. lt should be targeted at addressing needs. Priority: HIGH lndicators of success: a)Availability of skilled staff for training and HSAM at the Woreda level b)Targeted training and HSAM carried out at Woreda level and the lower levels. Who to take action WHDH ,ZOFP Deadline for completion October 2009 Finance Woreda Government should fund CDTI activities by increasing its budgetary allocations and ensure timely releases for CDTI implementation. Evidence of Woreda actual release from the Health budget for CDTI activities should be available. Priority: HIGH lndicators of success: a) lncreased amount budgeted and released for CDTI activities. b) Documentary evidence of Woreda support to CDTI available. Who to take action: Woreda Administrator, WHDH and WOFP 31 Deadline for completion October 2009 Transport and other material resources. The Woreda should liaise closely with the Zone to ensure they get adequate transport and other materials needed for CDTI implementation in view of sustainability. Priority: HIGH lndicators of success: a) Availability of sustainability plan addressing the transport and other material needs of the Woreda. b)Availability of capital equipments and other Materials for CDTI activities Who to take action WOFP, ZOFP ,APOC and LFTW Deadline for completion : a)October 2009.b)Dec 2010 Human Resource. lncrease the number of trained personnel available to support CDTI implementation at the Woreda. Woreda project staff should be trained on need assessment, data management and documentation. A reward system should be instituted by the Woreda Head Desk to reward well performing staffs Prio HIGH lndicators of success: Availability of sufficient trained Zonal team for effective CDTI implementation. Reward system in Place for hard working staff Who to take action: Head of Zonal Health Desk Deadline for completion: (a)January 2010 (b)December 2010 Priority. HIGH Coverage Sustain 100oh geographic and achieve 80% therapeutic coverage by ensuring that list of endemic communities and their populations are available for coverage data verification lndicators of success: a)Availability of list of all Onchocerciasis endemic communities at the Woreda. b)Availability of verified treatment covera data Who to take action: WOFP and ZOFP Deadline for completion October 2009 32 3.3 Sustainability at the Health Facility level Figure 3. Average Performance of Each group of indicator at FLHF level 4 4 4aJ400 ; 350 -sr 3 00 * ;:;sor150I rooi 050000 .*'"ed"$""":*"f:"-ff"/ .So Group of indicators Planning (2.5): Written plans for Year 2009 CDTI were seen pasted on the wall of 3 out of the 4 Health Post visited. All activity plans were written in local language (Oromifa).lt was only in Bedo Health post that written plans were not seen .The health facility in charge of Bedo Health Post said she had it in her head but forgot to write it on paper and paste. lntegration takes place by the same staff in charge of CDTI activities but there no written integrated health plan seen at this level. HFs works in close collaboration with Kebele administrators in carrying out their duties. lntegration (3.0): The health Facility staffs are in charge of all the health programmes in their kebele. There are two Health staffs per facility. CDTI activities are usually carried out in an integrated manner. The HFs equally combines CDTI activities with other health programmes in the area. Leadership (3.0): The health facility staffs take full responsibility of CDTI activities in the Kebeles under their supervision. ln each Health Faculty two female Health extension workers are available and both are responsible for CDTI activities and shares responsibility amongst themselves. ln all the Health post visited, Health staff explains that their annual CDTI activities starts with a meeting with their Kebele administrator. They meet to agree on how to mobilise the community through their community leaders. Activities usually discussed and agreed upon include census updating, mobilisation and Health Education, date of distribution, training date for CDDs and any other issues of concern that the Health staff might have. 33 The Community structure is very much aware of CDTI and lends support to the Health Facility staff. However no written report of the planning meeting was seen at all the health post visited. Supervision/Monitoring (3.7) CDTI supervisory data is being transmitted entirely within the government system. The health Facility summarizes the activities carried out in her kebele and then reports to the focal person at the Wereda who then fonruards it to the zone. At all the 4 Heath Facilities visited CDTI data were seen. The record on CDD trained record, Treatment data and Mectizan inventory' Treatment summary for three years were seen pasted on the wall of all the Health post visited. This summary provided information at a glance on total population, Target population, Number of communities target and that treated refusal and absentees etc. Witn tf,i. information, it is easy at a glance to know the performance of the Health facility. The management structure in operation at the Health Facility makes routine supervision of CDTI aclivities easy and efficient. Each Health Facility in the zone is manned by two female Health Facility staffs. The Health Facility staffs lives in the community where the facility is located .They are appointed for training for that position by their community through the Kebele Administrators but paid by the government . For eifective supervision the two health staffs are usually assigned to different areas in the Kebele for CDTI activities and other health programme like environmental sanitation, family planning home visits etc. Movement chart for each of the health staff was seen on the door of the Health post to tell visitors on their where about. The Health Staffs have agreed with the community on the day to be at the health centre or time to attend to antenatal issues, every other time they are in the community supervising and attending to Health needs. Supervisory checklist were used but no summary report of supervision activities was written. Health Facility staff tries to address identified problems on their own since they are paft of the community. Those that they are unable to handle they refer to the community leaders or the Kebele administrators. Success and Hard work are usually rewarded, sometimes by using them as guide during polio campaigns or any other activity that requires a committed community volunteer. According to the Health staff , community selecting a CDD gives the CDD a sense of being respected and recognized by the community. Mectizan Procurement and Distribution (4.0): The FLHF staff uses similar inventory stock order for drugs and equipment used by the higher level .They use Model 19 form for receiving Mectizan. FLHFs requests for Mectizan from the Woreda with a request letter written and stamped by the health Facility staff. Copies of these were seen in all the Health Facilities visited. At Bedo and Kewissa health posts they complained, there was excess supply of lr/ectizan during the 2009 distribution period. This they explained was as a result of teachers and farmers population movement out of their area. 34 The CDDs receive their Mectizan allocation from the FLHFs based on the request calculated using updated census. CDD usually go to the HFs either at home or in the office to collect more Mectizan or return the excess when they complete distribution. According to one Health Facility Staff; "Mectizan is very important to me and my people, because of that I usually carry it home for safety and bring it back when I come to work. This is a temporary arrangement till I have a security guard for the Health post--Mesekerm Wakjira-Health Facility Staff in charge of kitiesa Health Post . Guto Gida Woreda. The Health Facility receives their Mectizan supply from the Woreda Details like Mectizan Batch number, Lot number, and expiring dates were however not seen in their record. Training/HSAM (3.3): Training is carried out routinely. FLHFs train CDDs annually in order to refresh their memory on CDTI. All the Health Facility staff visited lacked training need identification skill. No refreshment or transport is provided for the CDDs when they come for training at the Health Post. According to one of the Health Facility Staff; CDDs have accepted this work as a voluntary service to their community, no refreshment nor is transport provided because their homes are not far and they must have eaten at home" The Kebeles support the training by ensuring that every community send their CDDs for the training . The training of CDDs according to the FLHFs takes one day and an average of Eight hours. One striking observation regarding CDD training in the Zone is the inclusion of construction of measuring sticks in the training topics. Evidence of this was seen by the Evaluators when communities were visited. CDDs showed measuring sticks of different designs made with local materials but accurately calibrated. HSAM activities are usually done routinely. Community awareness of the benefits of Mectizan is high and this has led to high treatment compliance. However new issues like amending distribution time to accommodate migrant farmers, or sustained compliance when signs of Onchocerciasis disappears are not addressed. This is because HSAM is carried out routinely. There is lack of skill to identify what the annual objective of any round of HSAM should be. Financial resources (3.0) : There is no budgetary allocation at this level. The Government/Community system used in managing the Health Facilities is functioning effectively. The Health Post don't manage budget. According to one of the Health Staff she said during the last distribution exercise, APOC provided supervisory allowance for 3 days for them but they continued to work until distribution exercise was completed because activities are implemented in an integrated manner. The FLHF workers do not get additional field allowance from Government only their salary. This is because they are working in their own communities. Below is the comment by one of the Health Staff ; 35 "ln managing our Health Post, most of the materials we use are provided by the Woreda, we only use it as instructed by them to serve our people because is our community that appointed us . Little things like pen or pencil for our activities we can buy from our salary - Alganesh Degago, Head of Loko Health Post -Guto Gida. Woreda. Transport and Other Materials (4.0) There is no transport provision at this level since all Health Staffs lives in the communities where their facility is located. They trek to do their work Activities are integrated, so that when they go to the Woreda for other health activities they equally transmit CDTI data. Training/HSAM materials are available and they are provided by the Woreda. Training/HSAM was sufficient and already translated in the local language. Human resources (4.0) Front Line Health Facility staff managing the Health Facilities are quite stable as they appointed by the community members and belongs to that specific community. Evaluators were informed that one of the criteria for the appointment is that they remain and continue to work for the community even after marriage. Coverage (4.0) ln all the four visited FLHFs the geographic coverage for the last three years is 100% and the therapeutic coverage is greater than 65%. 3.3.1: Recommendation for the FLHF Level: Table 4: Recommendation for the FLHF Level: Recommendation lmplementation: Planning: Effort should be intensified by the HFs at ensuring that a detailed copy of the integrated Health plan is available at the Health post. Prio HIGH Indicators of success: Availability of integrated health plan Who to take action WOFP,HFS. Deadline for completion August 2009 lntegration: Ensure that the documentation of integrated activities is available for effective supervision and monitoring by the higher levels. Priority: HIGH lndicators of success: Availability of report on integrated activities. Who to take action WOFP,HFS. Deadline for completion April 2010 Leadership There should be documentary evidence of the HFs meeting with the Kebele leaders and other key MEDIUM Availability of minutes of meetings of HFs with Kebele leaders and other partners. 36 stakeholders Who to take action WOFP, H FS April 2010 Monitoring and Supervision : Target supervision on problem areas should be given high priority by the Health Staff. Checklist for supervision should be summarised and supervisory visit report written. Record of feedback given based on supervisory visit should be made available at the Health Post. Priority: HIGH lndicators of success: Availability of written report and provision of supervision feed back. Who to take action: WOFP,HFS Deadline for completion January 2010 Mectizan Supply: FLHFs should emphasize to the CDDs that census updating is part of distribution activities and should be carried out at the same time. This will enable accurate calculation of Mectizan need of the community. ln order to monitor Mectizan use and tracking in the event of pilferage, Mectizan Batch and Lot number as well as , expiring dates should be recorded. Priority: HIGH lndicators of success: a) Availability of Community Mectizan request based on accurate census. b) Mectizan data available with batch and Lot numbers as well expiring dates. Who to take action HFs Deadline for completion March 2010 Training and HSAM: Training and HSAM should be targeted at areas with need in view of programme sustainability. Priority: HIGH lndicators of success: Availabilty of more efficient CDDs especially in identified areas of need like census updating and recording of treatment. b) lmproved therapeutic coverage due to targeted HSAM. Who to take action HFs and WOFP Deadline for completion October, 2010 Coverage Sustain 100% geographic and achieve 80% therapeutic coverage by ensuring that coverage data are verified Priority: HIGH lndicators of success: a)Availability of verified treatment coverage data at the Health Facility Who to take action HFs Deadline for completion October 2009 37 3.4 Sustainability at the Community Level Planning (highlY,3.7) The CDDs are enthusiastic about CDTI implementation. They work closely with Heqllh Facility staffs, kebele leaders and village chiefs .The CDDs plan and manage CDTI activitLs by; Conducting census updating, Requesting for Mectizan based on census population, lnforming 6ommunities the specific date of tilectizan distribution and reporting back on treatment to the HFs. The villlge registers were seen at the Health Facility and confirmed that the above activities have taken Place. Leadership (highlY, 3.9) Kebele leaders as well as village chiefs have impressive knowledge of CDTI activities. They know the impact of the disease, eligible population, as well its benefits. They also participate in census updating; social mobilization and monitoring Mectizan distributions burprisingly, most CDDs are appointed as village chiefs so that they are able to solve COit pro-nte, when it is encountered. lf there are problems like shortage of Mectizan and refusals the Community leaders inform the FLHF who lives in the community and ensure it is addressed. Community members interviewed reported that they were involved in CDD selection and change them when necessary. They are equally involved decision regarding time and place of distribution. Figure 4. Average Performance of Each group of indicator at community level Group of lndicators 44 3.9 ..o""" "/ a"*" 4 A 3 2.5 2 1.5 1 .5 0 o 0 .P o B o u,g o """ od^sqe\o' ^rQ "rwo" ^-.re\sti ( 3.9 38 Monitoring and Supervision (fully, 4.0) CDDs are reporting their activities to Health Facility staffs, Some CDDs are literate enough to summarize their activities and report it using village summary reporting format, others submit the completed village registers and summary is done by the front line Health Facility staff. ln most of villages visited, CDDs have already completed distribution and submitted their register to the Health post. When inquired from the Health Facility on the reason for keeping the Village Registers, One of them had this to say; "lt was the decision of the Kebele administrators taken after we lost some registers to communal clashes. Some of them also do not keep it well, they allow their children tamper with it.-Miss Alganha Dagando-Loko district of Guto Gida. Mectizan Supply (highly, 3.5) Village chiefs and CDDs reported that they received sufficient amount of Mectizan for 2009 treatment period. CDDs requested the drug based upon updated census population. CDDs revealed that unused drugs for absentees and refusals were returned to Health Facility levels. The community register on total population incorporates some invalid data such as migrated household members and some persons that are dead. This affects their therapeutic coverage and drug requirement. ln addition, in Kitiesa, Health Facilities of Guto Gida Woreda, there were variations in the amount of Mectizan balance when compared with Woreda report. Mectizan is collected by the CDD from nearby health post not far from their community. They trek down for collection because the Health post is quite close to them. Training and HSAM (Moderately, 2.8) Comprehensive information was collected from village chiefs and CDDs regarding Health Education. Community members reported that Health education was provided at community meeting and house to house visits before inception of Mectizan distribution. Besides this, community members confirm that CDDs informed them on mode of treatment ,benefits of the drugs and possible side effects. CDDs pledge to maintain health education activities until the disease is eliminated. For effective communication of the CDTI messages to the communities, the CDDs showed their manual and Posters that has translated into Oromifa language (local language) which they appreciated a lot. Community members have positive perceptions towards CDTI. They have given high value to Mectizan. They wish they will be allowed to take Mectizan for unlimited number of years. All community members elaborated that Mectizan has multiple effects on their health. Besides Onchocerciasis, Mectizan kills lice, bugs, relief from persistent itching and expelling intestinal worms. Accordingly, communities are demanding to have Mectizan distributions for every six months. ln line with this, communities noted that, ".we need Mectizan for longer period even for our future generation. lf the drug distribution is delayed or interrupted we will appeal through our kebele and will fight to get it up to Woreda" Community meeting in Haro village of Kitiesa Health post-Guto Gida Woreda 39 Training is not targeted. lt is routinely done without need assessment. CDDs as well as the heJtth extension workers reported that CDDs received routine refresher training every year. The training was conducted at the FLHF levels for one day. The facilitators of the training were the health extension workers' The good thing is that CDDs are knowledgeable on CDTI programmatic activities- know who ls eligible for Mectizan, able to express side effects of lt/ectizan and well acquainted with Onchocerciasis. It seems also that the communities decided not to provide any kind of support to CDDs because they believe that it is a great honor to be appointed to serve their community. Financing (highly, 3.0) Community'inceniive to CDDs is not common in communities visited even the CDDs do not expect it. lt is evidenced in this statement; "l am treating my community and my people why should I expect incentive from them. The honor of selecting me to serve them and the moral appreciation I get from them is enough." -tt4r. Taye-Tafese from Kambi Village of Kewissa Head post of Lecha Dulecha Wereda. "Even if my community gives me incentive, I will be uncomfortable to accept because from serving them as CbO they have given the honor of becoming their community leader, thatls enough for me and I appreciate them."-Mr Alemu Derge CDD of Wanebo community of Bedo Health Post of Lecha Dulecha. Some community members, invited CDDs to have coffee during distribution time' One CDD however mentioned that incentive is needed for the CDTI work. Human Resource (highlY, 3.9) Data showed that' Community members ratio to CDD is about 1:80 across all communities. This shows that one CDD treats about 10-16 households. CDDs are living close to the households that they serve and somehow related. Some are blood relation, others marriage relation and others close neighbors. Most villages are confined together so that CDDs do not travel long distance for distribution. CDDs are willing to continue to distribute Mectizan because they are happy to protect their communities from Onchocerciasis and other diseases like persistent itching,lizard skins and blindness. A few CDDs said if possible APOC should provide small amount of payments during distribution time, like payments received from immunization campaign and other out reach health service programs. Evaluators advised him to learn from other committed CDDs. There is no significant CDD attrition in the areas visited. Coverage (fully, 4.0) The partial 2009 distribution data indicated that the therapeutic coverage ranges from 69% to 75% The 2OO8 data shows that therapeutic coverage ranges from 68% to 80 % across all villages . ln 2007 and 2006 treatment data indicated that therapeutic coverage were between 57o/o to 84% and 72% to 80%, respectively. Therefore, the weighted average of therapeutic coverage for visited communities is 73% from year 2006 to 2009. 40 3.4.1: Recommendation for the community Level Table 5: Recommendation for the Village Level Recommendations lmplementation Training/HSAM Training and HSAM should be need based and targeted for cost effectiveness and efficiency lf communities are willingly to select more CDDs this should be encouraged and training conducted for them. The option of using old and experienced CDDs to train others should be explored but supervised by the HFs. HSAM should address the need for continued compliance even when signs of Onchocerciasis are no longer visible Priority: HIGH lndicators of Success: a)Availability of enough CDDs in the community. b)High community awareness on the need for long term compliance Who to take action: HFs, CDDs and WOFP Deadline for completion : March 2010 Mectizan Supply Mectizan should be requested based upon accurate census data. The community registration book should not contain invalid data like migrated households and died population. Census updating should form part of distribution activities. Priority: HIGH lndicators of Success: Accurate updated census communities. Calculation of coverages accurate census. of endemic based on Who to take action: CDDs, FLHF and Woreda Onchocerciasis focal person Deadline for completion February, 2010 Finance lf the kebeles are willing they should be allowed to provide refreshment and any training material they deem fif during CDD training Priority: MEDIUM Indicators of Success: Enhanced committed and programme ownership by Kebele administrators. Who to take action: FLHF levels and Woreda health staffs Deadline for completion February May,2010 41 4.0. Comparative Analysis of the Sustainability of the Four Levels 4.1. A quantitative judgement, based on the grades given to individual indicators The scores given for the various sustainability indicators during the Evaluation of East Wollega CDTI project is shown below. Table 6: Quantitative indicators at different levels of the project Levels Groups of lndicators Average Planning lntegratio nof support activities Leaders hip Monitoring& Supervisi on Mectizan supply Training& HSAM Finances Transport Human Resource s Zone z.J AE 3.0 3.1 3.5 2.0 2.0 2.3 3.0 4.0 Woreda 3.0 3.0 4.0 J,J 3.4 16 2.0 3.0 3.0 4.0 FLHF 25 3.0 J,U 3.7 4.0 n 0 4.0 4.0 Commu nity 3.7 0 3.9 4.O 3.5 2.8 3.0 0 3.9 4.0 36 J.J 3.4 Figure 5. Quantitative judgment scores at different level 3 Average 2 1 0 Zone Woreda FLHF Community Overall(Project level) Project level The average overall grading of the nine groups of indicators for the entire project is 3.3. The Community scored highest in terms of CDTI implementation with an average score of 3.6.followed by the Health Facility scoring 3.4. 42 This makes the community to have a "Full" sustainability potential using the 9 group of indicators. The Wereda and the zone scored 3.3 and 2.8 respectively and this performance groups them into the rating of having "High" sustainability potential. Figure 6: Average performance of the different indicators for the entire pro,iect a .3.s .l 2. !i I,5 :! o. !; o 3.(i3.5 f.5 ,1.2 2.<) 2--9 7..-7 2^l "*ooo-o*ofo*-*",1*-"lr**"^1".""S .-<o- ts\o- u.d* ^.*d ^q,'.g("<-o- From the above figure it is clear that finance, followed by transport then Training/HSAI\I and planning are the group of indicators that are weak in the East Wollega project. Coverage, Mectizan supply, Monitoring and supervision,leadership and Human resources were seen to be strong. FigureT: Average performance of the different indicators at different levels Fi00ro A ouantihtivo lndielors byproj.c't by proj6c'l l.vols 1 Plannirs lnE8ntbn brdeahip Monitomg &supenision [{@liaSupplv Tniing&HSA1\i Fiauce Groups oflndicators Tanspod Hum rcsourr ovcagc aZDtr I Wo@da oFLHF o Cmmoily aAl Fqect leEl (A€trge) 43 At all the levels coverage was very good as well as Mectizan supply. Planning was best at the community level. lntegration was not applicable at the community level, best at the zone with a score of 3.5.Leadership was best at the Woreda level with a score of 4.Monitoring and supervision of CDTI activities were best practiced at the community by the health extension workers. Training and HSAM was best at the Woreda level. Transport was not applicable at the FLHF and community in this project but was better at the Woreda than the zone. Finance was best at the community, non applicable at the FLHF and had the same score of 2.3 for both Woreda and the zone. Human Resource is best at the FLHF and the community. Table 7: The erformance of different as cts of sustainabi 4.2. Qualitative judgment of project ryu ffi B Wffi8 44 Grading the seven aspects of sustainability, Evaluators concluded that five of the aspects were completely helping sustainability. These are integration, Community ownership, Staff attitude, simplicity and Effectiveness. Resources partly help sustainability especially at the Health Facility and community levels whereas it blocks sustainability at both the Woreda and the Zone.Efficiency is partly helping sustainability because CDTI is implemented routinely in an integrated manner alongside other Health Programmes. Efficiency is also blocking sustainability because of the competency of most of the health workers involved in CDTI implementation who have skill gap in areas like documentation and targeted activities. lf a percentage performance is used ,the aspects scored about 85% in terms of its support to sustainability. 4.3. Judgment using the five key aspects of the project - 'critical elements' of sustainability Table 8: CRITICAL ELEMENTS OF SUSTAINABILTY Money: ls there sufficient money available to undertake strictly necessary tasks, which have been carefully thought through and planned? (Absolute minimum residual activities). Transport: Has provision been made for the replacement and repair of vehicles? ls there a reasonable assurance that vehicles will continue to be available for NO NO . ri. f:): W 45 minimum essential activities? (Note that 'vehicle' does not necessarily imply '4x4' or even 'car'). Superuision: Has provision been made for continued targeted supportive supervision? (The project will not be sustained without it). Mectizan@ supply: ls the supply system dependable? (The bottom line is that enough drugs must arrive in villages at the time selected by the villagers). Potitical commitment: Effectively demonstrated by awareness of the CDTI process among policy makers (resulting in tangible support); and a sense of community ownership of the programme. YES YES YES 4.4. Conclusion Based on the seven aspects of sustainability and in line with the guideline for grading the whole project using the five critical elements of sustainability ,the Evaluation Team conctudes that the East Wellega CDTI project is MAKING SAITSFACTORY ?ROaRESS IOWARDS SUSTATNABTL/IY. Transport and money are challenged at the moment and not dePendable. Supervision is good at all levels using the integrated Health system of the government.. Mectizan supply is carried out efficiently using Government system which is dependable and provides enough supply and available on time. There is good political commitment at all levels including community resulting in a high sense of owneiship however there is poor political awareness on the need for a written plans to continue io make CDTI programme function effectively in foreseeable future without external funding. The aspects blocking sustainability of the programme in the Zone is Resources. There is no written evidence that sufficient money will be available from Government to undertake strictly necessary tasks, which have been carefully thought through and planned to sustain CDTI programme in the Zone. ln addition no provision has been made for the replacement of vehicles and motorcycles, although at the moment, maintenance is effectively done using Government dependable resources but replacement is obvious in the foreseeable future and plans are not yet in place for replacement. The Evaluators also concluded that the quantitative score of 3.3 for the pooled groups of indicators at the four levels supports the above qualitative score. 46 5.0. Feedback/sustainability planning plan development workshops Zonal The East Wollega Zonal feedback meeting was held on the 27th of May 2009 at the Office of the Deputy Head of Health desk. Those in attendance were the deputy Zonal Administrator, The deputy head of the Health desk, the Zonal Onchocerciasis focal person and three additional staff from the Health desk pool. List of participants is attached Annex-1 The Evaluation objectives, Methodology and Findings of the East Wollega evaluation was presented to the Zonal team as well as the conclusion that East Wollega CDTI project was making satisfactory progress towards sustainability The Zonal team sought for clarification regarding the scores and the results presented. Appropriate explanations were given. At the end the Zonal team agreed with the findings as a true reflection of the present state of CDTI implementation in the zone. The Zonal Onchocerciasis focal person was happy about the impressive performance of the Health Facility level and the community adding that it shows that they are strengthening the lower level who should own the programme. He promised that the Zone and Woreda will ensure that there is improvement in the highlighted weak areas. The Zonal Administrator assured the Evaluators that the Zonal team will hold further discussion and agree on the way fonruard which will be in cooperated in their sustainability plan. He thanked the Evaluation team for their hard work and the feedback. After the feedback meeting, there was a briefing of the Zonal team on the development of sustainability plan. The Evaluators went through the sustainability plan guidelines with the Zonal team and the plan development commenced and continued during the remaining days of the planning meeting. Woreda The Woreda level feedback meeting commenced on the 28th of May 2009. A total of 36 participants were in attendance. Participants wer/e drawn from the 10 CDTI Woredas and two zonal offices. Participants were Woreda Onchocerciasis focal persons, Head of health desk of the 10 Woredas and their Woreda administrators. East Wollega deputy Zonal administrator, Deputy Head of health for East Wollega and Head of Health desk of West Shoa . East Wollega Onchocerciasis focal person and some other experts from the East Wollega Health desk unit (see lists of participants in Annex -2). The meeting started with an opening remark by Mr. Sitotaw Fito, the deputy Zonal Administrator of East Wollega. ln his remark, he reminded participants that this opportunity of being part of the feedback /planning meeting for the zone should be used as a training opportunity for them and ensure that it is done carefully so that accurate plans are available at the end of the workshop. 47 The Evaluation findings were presented by the Evaluators, the following were the questions asked by the participants; a) There are some Woredas that are at risk of Onchocerciasis but were not included during the launch of CDTI, what is fate of these Woredas? Can they request for Mectizan? b) Why was community score low regarding training &HSAM indicator? c) How can Woredas assign a focal person for Onchocerciasis in the era of BPR( Business process re-engineering ). BPR does not support the assignment of a single person for specific program . The relevant persons provided further clarifications to the above concerns. The participants were pleased with the findings and were very willing to be part of the group work for the SWOT analysis and the development of the sustainability plans. Group work on different SWOT analysis for the different levels commenced and was followed by presentations. Comments and suggestions were made on the presentations and the different groups took note of it for amendment. The SWOT analysis is attached as Annex 4 . The sustainability plan development continued on day three for the Zones and the different Woredas. A draft presentation on their group work so far was made before Tea break in order to identify areas of weaknesses in the plan development. lt was obvious that participants needed to know more about on the guideline for the development of a post APOC sustainabilitY Plan. Mr. Chukwu Okoronkwo was asked to make a presentation on Key issues to note when developing a post APOC sustainability plan, this was quite useful for the participants. Group work continued for the Zone and the 10 Woredas ,First to produce the three years plan and later the remaining 2 years plan. On the fourth day, group work continued till Tea break time, after which presentation of draft plans commenced. Comments and clarifications were given after each presentation and the different groups were requested to make the necessary amendments on their plans. As the different Woredas were presenting their budgets, the attention of the Woreda administrators were drawn to the amount needed for CDTI and they all agreed that is realistic and they can make it available. The sustainability planning meeting ended at about 4.30 pm. The deputy Zonal administrator of the East Wollega zone in his closing remark thanked all the participants and the Evaluators for their hard work and requested that since the Ethiopian financial 48 year usually ends in June, effort should be made by all concerned to make sure the budget proposed is in cooperated in the next financial year made is available on time for CDTI activities. Mrs. Olamiju thanked the Deputy Zonal Administrator, and the Woreda administrators for their commitment and participation throughout the 3-4 days workshop. She also thanked the Health team from the 2 zones and the Woredas and encouraged them to fine tune their plans as agreed and submit before they leave. It was agreed that the Evaluation report and their draft five years sustainability plan will come back to them shortly for final input and signing. Address ResponsibilityFull Name S. No Nekemte-Zone Deputy Zonal adminstartor1 Sitotaw Fufa Nekemte-Zone Deputy head Zonal health2 Bedasa Fite Zonal Oncho focal personNekemte-Zone3 Ensermu Jeldu Zonal CDCNekemte-Zone4 Getahun Zewudu Zonal PlanningNekemte-Zone5 Habtamu Siyum Nekemte-Zone Zonal CDC6 Basha Nemomisa EvaluatorAddis AbabaAbata Tilahun7 EvaluatorNigeriaI Francisca Olamiiu EvaluatorAddis AbabaI Frrew Ayalew 49 ANNEXES Annex 1-List of Partic at Zonal Feedback m Annex 2--List of Participants at Woreda Feedback meeting/sustainability plan development workshop S. No. Full Name Address Responsibilitv 1 Endalew Geleta Guto Gida Guide 2 Busho Kumbi Guto Gida Onchocerciasis Focal Person 3 Abrham Bekele Guto Gida Woreda Adminstrator 4 Tilahun Dessisa Guto Gida Head, woreda health 5 Sitotaw Fufa Nekemte-Zone Deputy Zonal adminstartol 6 Bedasa Fite Nekemte-Zone Deputy head Zonal health 7 Ensermu Jeldu Nekemte-Zone Zonal Oncho focal person 8 Getahun Zewudu Nekemte-Zone Zonal CDC 9 Habtamu Siyum Nekemte-Zone Zonal Plannrng '10 Basha Nemomisa Nekemte-Zone Zonal CDC 11 Abiyot Bekele Diqa Woreda Onchocerciasis focal person 12 SamuelTolla Diga Woreda health head 13 Yonas Terfasa Diga Woreda Adminstrator 50 Woreda Person Onchocerciasis Focal 14 n Tollesa a Sasiga Woreda health head15 Getachew Asfaw 16 Habtamu Tolla Sasiga Woreda Adminstrator 17 Lemesa Terefe Sirbu Sire Woreda health head 18 Chala Gemeda Sirbu Sire Woreda Adminstrator person Woreda Onchocerciasis focal 19 Brrhanu Meseret Sirbu Sire Woreda Adminstrator20 Shiferaw Kebede Jima Arjo Jima Ario Woreda oncho focal person21 Kefalew Adnew 22 Diriba Bekele Jima Ario Woreda health head 23 Kitesa Mossisaa Leka Dulecha Woreda health head 24 Amsalu Tesema Leka Dulecha Woreda Adminstrator 25 Korsa Eba Leka Dulecha Onchocerciasis focalWoreda person Woreda health head26 Ahmed Yimana Boneya Boshe Boneya Boshe Woreda Adminstrator27 Gobena gemechu 28 Fromsa Hinkosa Boneya Boshe Onchocerciasis focalWoreda person 29 Busha Tesfave Wama Aqelo Woreda health head 30 Midekisa Gemechu Wama Agelo Woreda Adminstrator 31 Fikire Desalegn Wama Agelo Onchocerciasis focalWoreda person 32 Tariku Denqiva Wayu Tuka Woreda health head 33 Oliira Berkissa Wayu Tuka Onchocerciasis focalWoreda person 34 Abera Fita West Shoa-Ambo Zonal CDC 35 Gudisa Deyas West Shoa-Ambo Zonal CDC -Onchocerciasis focal person 36 Sichala Kore Shoa-BakoWest Tibe Onchocerciasis focalWoreda person 37 Abata Tilahun Addis Ababa Evaluator 38 Niqeria EvaluatorFrancisca Olamiju 39 Firew Ayalew Addis Ababa Evaluator 40 Chukwu Okoronkwo Niqeria Evaluator 51 Annex 3-SUSTAINABILIW OF EAST WOLLEGA CDTI PROJECT FEDBACK/PLANNING MEETING AGENDA DAY ONE Item Activities Time Facilitator 1 Registration of participants 8:30-9:00 Ensermu 2 I ntroduction of participanlq 9:00-9.05 Ensermu 3 Welcome and opening remarks 9:05-9:1 5 Honorable guest 4 lntroduction to the Workshop; What are the objectives and what is sustainability Methodology for Evaluation 9:15- 9:'10:00 Olamiju Abate 5 Tea break 10.00- 10.30 6 Feedback on achievements, issues and lessons from the evaluation on sustainability of East Wollega CDTI Project Zone level B Woreda level HF level Community level 10:30- 11:15 Olamiju Abate Olamiju Firew 7 SWOT analysis 11:15- 11:30 Ensermu 8 Group work Discussions on Problems identified and the solutions to these problems using SWOT analysis in Groups Plann ing/integ ration/leadershi p/monitoring and supervision Mectizan/Finances/train ing& H SAM Tra nsporUh u ma n/coveragq_ 11:30- 12.15 Olamiju, Abate Firew and I Report from groups discussions 12:15- 13:00 Group representative 10 Lunch 13:00- 14:30 Ail 11 Report from groups discussions continues 14:30- 17:30 52 Day 2 Day three Item Activities Time Facilitator 1 Reg istration of participants 8:30-9:00 Ensermu 2 Roles of the different levels and partners 9:00-9:30 Ensermu 3 Steps in planning for the sustainability in this project and grouping 9:30- 10:30 Olamiju 4 Tea Break 10:30- 11:00 5 Group work on Development sustainability plans of 11:00- 13:30 Ail 6 Lunch 13:30- 14.30 7 Group work on Development sustainability plans continues of 14:30- 15:30 Ail 8 Presentation of Group work 15:30- 17:00 Group leaders I Presentation on Key issues to note when developing a post APOC sustainability plan 17:00- 17:30 Chukwu Okoronkwo Item Activities Time Facilitator 1 Registration of participants 8:30-9:00 Ensermu 2 Finalization of group work on Development of sustainability plans 9:00- 11:00 Ail 3 Tea Break 11-11:30 4 Group presentation of draft sustainability plans 11:30- 13:00 AII 5 Lunch 13:00- 14:30 6 Submission of draft plans by the different Zones and Woredas 14:30- 15:30 All 7 Administrative issues 15:30- 15:30 Scout 8 Closing Remarks 15:30- 15:45 Deputy Zonal Administrator I Vote of thanks 15:45- 16:00 Olamiju 10 Departure 16:00 =t. o e& *H ro c(o I^at *R E,q;*,E(E+trclJ- i .l ESE; I 5Qg* o.g s€;g zIFf oo a I C o O) .C L (U F G)E't o L o_ c o o) .Ec '6 L-#6 .E (o co9Et'tr O() o-t (tr a E o) J oc G) o o- E tu I lutrIF o -aa ,ic')o v(tr9 (5 E 3PE!'(/)=(trU) ad cIL.vL L o o I .Y!EJJO5o o (5_ G) bo) ocibELE-O .r (E t=f oE fFE 8>oEOz a e E'= i =.=(trc -oE=(s P5>> Eco do'== ", E)+ I O O=: o< o (/,-c o-ri= o H€E ,E E OE 5 Eh* 5*N 6f o o E-c)^ s$Es e 'E= I a E e€ 5 o5 Etr =be E E ao LuzY lrJ = ?bL g -9 Eo ol (oI 9" E- =6 Eb R= Eo ;i= o?odt.o> E P(U oEE E9C -Y=oo(E0JE o)c I(! a L oo o_ I F oz lIJt.F U' .a EE. EE3*O= cE= .o E S= Eo E "E E o- gg r =Ex r z:'E 0) *c E€S E E 9EqE5BEq; o c o (! .N .F = a c)LC)o= ^ oYao-o IL b- ME Ee(tr= aB *OcL .O or) .ebtEEf(5qa F o 2s o)c .E E(tr E- .9(! L o) o CC =aL oE(5 oJO U'Z N (f) o E E(E E') o o- tro o G E') -96 =oG UI o .9 o (E tr o =o < 9, U' z o =@+ xotrgCE rO ILEt-cb;6z$ctIo- 'o O tt-EIO , ONN o-!+h 6(J ';-iNd I-_ EoHFEI O7i.EN- NL 'lJ E(! - ?ftNO 'N !C(tr o;or C ailLLO (tr 5lr 'r ci=^+.-LVNll ON E.o-f-r^o !5uNEI -.(/)N .EoFCi(E r (EC E oo og cl<oo(I, o)ocE' 'F o) EP .!p Co(!!o: :.trVe +; b&ea€ -o'=sr od9.9Oz- P! > gg6t; q -.- ! EEa E o oz C o o) .c .C(U- o oE 'aa '<(,, av, +E 6 E 06 o_f 3oc o.9 orS o= .:.9'U)J L 'o L o Sorutc o 9L(!O6*Co fP =H_ b L) .>):c -o'o -Y gEL AY9H i E? o ?3 L- .ob rrt (D'Ei.l o)o .E(!f oE tooo 4 =rE EC.=ooo c .:=N lai E 9 E FA ,;€.e B-A o) €Y -EE a'= >. uJ XE oS=Eq)? (/) o-E EOc) o)-(JE 6(! ti)ii uE.cFrfiE ob, =s6: €6dcP tt.oE c>.=o> E(/,LU >^E- EH +bd5 cOEEo) 'tro3; r+C O1g -Y t-() oJ( +0E 9"q oz p (tr o; E -o.Y - >.(tr (/)?i.L -L (,)E I? Ed 6 C o E EE3 96o =PP= HSoo(/)Ez O) aa? c (tra cE'<o (1).=EE; .g o! .9(g (U LO8 3g +E E O Lq) o ) h -c(! .e6 I Lto (5 fc L G E -E PH;br,;E A =HEEEE xE, fitFEsSdE gb E o o- o_fa o E o 'E -o L .aE c >,(U oEtro ,! o)t= 5E <a ro E-AE-E coC G)+ .-L PEEI' ;;€ e * Eg He 3 < 9+ A_E o o_ -L6.o -PE(5 = o'N ^L O ==9-4) E EE2. .Nb69 '-= ) aE ? gfiF rF o (! a Eo =E Eoo I .g '6bE =_Oho-va-E oro rI b c G .N oo orE ,C<cU)(trI LFod ooc(o C iL o)c9oiFO =afE $ rO (o t- o, t ro I oIN (5 E oN EC(55 (tr L .u) -.coEI!N< L tL oN .aa(! EL(5 o_ I(trocroftNtt o- LL oN Ec C' Io -N L LLs REC E3Eg(! -E -oNN EC(U O o o_ !c(u I oIN I o- LL oN Ec(U Io -N I (! oL =m .C =(! oI -o(ET ENOE OE o_ lJ- oN 'o c(! -o -N t o e& * u-t5o- U) o)YL o ts (! C o EE(tr o o o o- E uJ @ J aE oo io o(! E I o_f o C o o .N ,F f C(u .N o o ob9 -(trf,o 'E >\(5.= (/,x 'oo 9"q P2 to iEoc-oO.+OO Eq) .Nc(tr o) L o -;A- o oltr t-f/) '=aisEa(tr itE -cO!O cEc$o L-o6og r-O C c) Ea U) c)aa(5 oooc E o o) .g .E(5 L oE '= oL IL I U)Ec = c o (6p fu = 0) E .F € o- =3o E LL o E c(tr -C o P =o -c co (,, .Ec FE *o ooEu aE o-> a C o E) ooE o C o .F (5 .NE(tr cr)L o I G)t o o o o- E c) EC o 1'!(! o o E trl I CCoo '6'o 't'= 9boo- ae -C*o C .!v =uX(sU€ EOCO(EF o-E = oE = =* o (r=: o-ioL>zu* b r OO z o F :fJ o U) a - C o(r) .Ec '(g L op o L(L rF o o)CA =olroha -cLLOR6 o-> L o =oo_c(! E o o o,(5E o ! U) L o _o E C c o (o J o- oo o _o .(! o Icl I J C)(5 -oE 0)o C o L B r+ o -YC)(trJ o(! E ulrg.= .:lXC,ro 9"6 9ci az G9E(5lco Ec(5 C c) Eaaq) aa(5 E ooE C o=1 -Oo(trar(tr(,) -o9 o.E6xZZ (! og o E E o (E oo 6 iF o C o o-13 o) '=> d-oiE: o =LL)#o .Q.O-E U)a LrJzY TIJ = L(5 o -cb5 o)< -2, 'E od o o-o>EE odE o)q io L o(Ji E rsA o c> 8.8 5 E o EfooE 1'I o olf, o -Y() (EJ oIN od oI o =(5 o)C r G a L o o o- o,L< '= Z, .=< ScnF- od q) oE(5 ._c LL E o o-oc o LF oc9(5= FO - lllf9 G) o)([ o oO c o (tr L o,o C .g ! ct) L o!(! o) -J c o9'a-L.- 'Ez lJo .= o- SE c(5 .N o o t oF O oz o) .E c(5 o- (o l- @ O) oN cf) t touJoaz oc oN tro o (U 6 =oG UJ o o oo .Y Go 3 E'o oo .cI o o o c, o E o. dtrr)tO t o ^LLYt- * t-tt5(L -o -N I -NE =(tr+f IoIN I EE(u Io_O tt- -oNN z o FfJ oa -E s .-L_c rE';oc i:o(gFo-c c o cc.LLo a.a sE =o) Os Es € a '=F:fc))LC =; E QO e g,rE,E;# -gE ots .C8bo2 H ETa z6 rrr: c EcJl o O)9 o-.'P6=>o ao- * _U ^;E\U, tlt o-o F uJ E. -F EFE-A!FL " cD(5'- -c. |t)o 8'E *E ) Oo olo" oc -c >'=ac -c:i-q->(E JJL^lE€r-g Egu E =6 .o9= EOr o JP -OE-C - (tr ;P!E o LOE>(!?6ELL =oFE(tr ro) E G,o bg? K-o roi#EsiE5== F zlFtr o(L o- o o .E gafF;E o-c =(o oc o C) Ca)d) o5-6 tr- ---uL d.= a H.C FS?EE" rF o .= -os(5 (!0) q= + u_tt- EE }E 9=oE gH EUHbE E. E88. s 8 AJ#v-)f(/) (!Noo u) a U) UJzY I.JJ = Pb P 9(EE b3, _EOo)o- L Eq =aO 6= c-9 atE of?odg<o5>3 Eo '(5 oEr c =o()-E -Yfoo(!0JE o,c i=g-(5 @ L oo(L I b9 E oEao C -YLo iEt o +ESs-* -F C)z tIJtF @ :6(!EE = 1Il -L' .E ac o tsaO ; vt:uA !2(Err- c, P 1--.a B-E ='H = g[ _s;€ e,cE E X ESc'E 8o - (!=O- (5 cIY.siq+ 8u lF o c o (! .N J aobeo-f,ooiE I L c 0) E =E E oo od aa ocaaLEsh -'aq.E rO o '=EB!ooH.N'= =Eo)LLa € EcJs=a6 -E <r,:-C E ; b -oE A 3.H N E; E' f 3=s8 E oF O -Zg o) .Ecc o a C .o(! L o)o c oEa(E.=35 ?a '=)qd L-odc. >€ o doaz N (f) $ GE' o o B o .cl tr ol<(! o o o N (U o U' oo =o o o @ ooc .Y Go =o o tr o =6 o E'tr G .9, o> Gc F o Eq, o(olo Ec(5 -o_O tt- -oNN io E(o Io_O tt- -oNN bE e=is € Eo .rEa ** E 9t Ec(6 -o_O t-t- -oNN -oIN Eftp -o*NNt -5 r66I 'E_o ALdvoi: *fr od 61 FEa:E cOC o tror, ; rE e fr oo sE (! 5 >'El:o= o E.o'tr- = -J o cgO o'Eo-(!o N-t= 0)tr=r * JF o) .c .C(! LF * E o o orP; E =E;6.=E c^E F o_oLLE f () L- -ob N sa =(5=oE tooo = 5F_ b6 H* obs s - o:l x er eg# tE sEg O o(L #LOO =(!oE =o o c oEo-cfo .08coTE cL -c. .gl b +e b_>o o_ LJ .9a EE =Noo =0)i> G) E E- :. o- o) =o(DeE :26t=ulI =x. b9 (g a c)P:o#a.-- -o o.! ==(trrLr+ co 'E@ =or()ELa= -oo =a(/)o I L o oq) a Lpb GLEPoli3E;Yoo =+o+o Eo8E OE OE ^-otoOc: OE9 o.rEC oN o) 'E s^aOE-C:oo)o o';i9> E Ood oz o (o ok E -oY a )r(g (/)?5.= _c'6* g? Eco 6 C o E EE3 e6 -o tro) s- frFo(l)(/)cz A U)-ra c (5-i EE;(o 0).=EE; .g oE(r(5 itr: t= O-8 5e +EEO o o) .E EC(tr -c Lo_ooo-= trF =.Y @ o .Yo(5J I oEc(E .r-' 6ru0) c ot -\Of1o-c b Lo PoI cOcos o.) Ec 6l- =e .EI",o H'.E c -.=!fl 6A-EE ,E 8F ET(rY E ! =(tr9a o Uo o rF o o-(r') -!too-Y <9 C o Eiag)o- >zr t-oxo o) F'E _c(trb or.N oE f EE2. .Nb9 = ?,E?tuT o -o(5 '(5 (! o c .9aEo .N .9 E6T> (I, a 13 o =E E oO I Eod o .os H' '.8 bS=Bo)ooo o o oE or 7- a. E EE$ L(tr .N oo od o) .=> .h< Es)FI ooC(! Cir E oooc(tr LF oc9oi FO -AfE G) o,(! L o o C) rO @ f- @ O) o r t- rr, (u o F o E o oEoc>oo9 -Yo5o- =c-<)otr+ o,c .E (5 o- :0E(! o- =(!ga lJr zd) (!() -OgE ba):=o! o v(Ur EE E ,odo- (E oottsE*Qb o:3€ P -y. a t) *c.lD;;8 o E o- -=do o-(trfN .ugs> E o .c(trL rF o c .9-E669Eo<o- ot (5 o o ;\ (J f L- oov EE*-(s 6 - }:r (559 r1J r (5 CA '- o)o=OE3rOE -o(U'<=aa .f _o otsta ,E abFc 'Eo L9 -r([ C o E 3q o- X' ==t OE o o .E tr o CL CL o E([OE _:oIs o )v =(5 -oSc 9 Ea T g_a L -o(U'.=bsfiog6 A:.PU =o rz RE 'o -;o1,I sg EintroE() ,:E o(o E(5 G) -EL\ or.9 oo a-c'< (o E 6Eo o: o--a= E .q E- o_ =>\ad o- J>(, ooO-E LLp) ,E Eo --of,u o c)oC G)At+oH(Li; o o)oL =oa5sEoJ_ .= -oti(o _o'=de (! L o oq) a 0)rE6 _E o o -o(5 o rO a>\ o.-= co=OE tr fr8c oEtnBo<EE ,60) c o c=or98 -V)L -=dso d.9) aE ob -otro .= J a oootrJ oo = q) -c c E9E a9o 6.1U' ,=Pt 6EzeL otrF o .N .E o .E o_ o9sE (tr9b,f;eEO)p l! o -c. o c o L o Gla E(/)([ at(1)o,J9 rO- Eo E o E JooEbt -vEc)-([EJO otiF a :laE oo o o L =Oo:qH- >9 <!U)c IO Eo9c oEO cT)OE9 9- O .E OrE I-L(JEeBFo(, o(,)o) -oPc oPOE#L(ULoo) _eB(uc,, troo -LZ* o o)L oEc qo) -qr8 oC)(E = >rC69s)COC -o(! , EE 6 C o ,EE_tro EE (/2G EEJ6 ,3 o oE o_ J @ =aC oo o o(L -c E"c oL @l cu)(U= o-b(trtr )JcocO(trE .-E .=O U#o'EoB f,tts oo .E> o=c-o '= (! oi o_< lt -g =([ o -c L o -c^ o_ o- -c LIJ =sg(II .OoeE(tr r- LVOa -OE -gE ^L -Y olUO L rQ (o a oE(U o (5 EoL o E o_ o)u)> EoC(5(I,)zi EP =6<ts o JEo .(/)t @to9 o.rtP o)oc.a 'aE lo- tt o- o- Ja oELL ,+Ioo (r) LE.oa=oi u.E9)_ i,I qo3k >. YN(1)E.E '->Ll- o*LtFt 3 ofEE(o rr(tr a -c =(t o)I o g) .E L '(g LF I oL) .=Ec o o-xo +j o o) o o (5N_ =oJO ,E 6o vC.bo.-, B q,6c .L _rz(olz (JE(E(tr o Bo =rEf(5F ^€.=*O cb* 6 -o--Xo E.? -(u:= CN o:= =< E)(5 tt# Fo O c o c oaL oEo-(U_68 E€ =!' OLcE(,, ^Ad{ tt Ec(! og)(5 L o oo a6, .S2 (tr t o.l re6;uooO'Eo)= -o)xo-Egoe o o G .9 E' s o) L .E c G IL c o (U (,, o c .g -ca c)1'(5 oJ e5 .= 'aBZ '=o =E O_2(5(/, c(5 .N 2' Ug>d Ec(! (,, C- '=2 sdF- ooE(5 E ir- EC(5 l- U) EL.o 3 u.E(o-c (E FOE oc9oi FO =afE o(,, oL c) o O r N CO $ rr) (o f- @ o, or (g o,g o B o(! LrJ o .9, oi o F o =o E oJ oE' o o E ci@lr) z oat. ut o- lrJ m az o o-o UJ tr. 9o- ^(Jl-EE gi6oP EES!.= o{ tr ll(o(!(6E!OoooLLLooo === ttl (! C)o oo o !P (o o ! o .E .!! ao6L o rO- Io_Olr -o == L o (o L .9. .= EE(5 -€o-O g)lIr 60 === -o_Ol! -oB= E L(6 E oaa .L .=oco_c-o(! *€ b -Pg:(tr o (5-=I9)iiD S ;:E >.E E 5ftgEIO! o ==E= ooc(5 C c(EYgE Egtrao(5 c,)-cq9 sdE oOIl.l- troo; ==g E(! 0) -obe EA o'F or'=([h EEEo aEieoo == z o trf o U' b E o c.9o= zOo6q5 gt -O6Eq o,9oc E,.i 'a gt! E OEo-c (!t 8= c> O)aA o .>E o)! E8E EC-CE O j (,) EB€sU' O (tr'E ^'E A EE--= E .= E o=ro(5;t= o .gE(UcF([E C. o)L (6Oo =tL#ls EIol ol)tElclol o Lo lzo(5 _oc -.9Vt.e.o> oEO-C=(!(, OE -co -o-oLLo o o-'E =B tsbo (J# -o €(! .9= P>u) <n a EA E85-(5 6.! 3 o,9(/)E(Ec'F -o 8'E + O? rr3e;gB,P af,E rro _l .q.1 -Yoo -g,.o(,) C =.o E (tr (5 E c[u o E EEa_(EI -o G'A -oE(5 Pa\<U)-: E'b 'E ro EEFC o) .=0)6 -e)ogEo g.B oo)LC C(Eooo 3R oi .axoii Po I(!0 _E(5ts =oo)eg C al 9 =o ^E =(trL Ea4oEgLL '.= O o o)c .g = .c Ec(5 o) .EE LSr6gE.!2 'I oo lrJzY lrJ = L o E o o .C c o -oE ooAC Xo) .q zd C >.C -(tr(EuooE or9ot- fobc =ccr ts a.9E-o F co QH o o:VsL--(g o]- >L.-v1 nE -+g'bE .= EO- EoorF Ec(5 o .:ltr .9. =>o;ioa e= a o5 rz -Yoo(E(!J-O o (5 E o- =a :laC o(J o([ L =oo(g c Eo o)c c o Eoa(5 -o o= -I- (D orEc.a '=a .!o s3Fo EoL o o)E = _o Eo(tri- 8p -L(UCL o) L) a! z.a oE o L o o E o(,, 0)EEorr() 9E ar5 -aF EE]E (5E oL o ts (5 c o E .= E E oo 9agE t. o o o = o, L .E c(5(L E o .F (5 L o)o c c9o .=:0 o> 'FO6g >d EC(t ->o-o J a C GN o o a -E L(U (,, .cE '(5 LF ooC G' .E LL oE. o EE(! l-a H_.q 9-.,.(!(tr FE ooLf oa oL C(tr Ef, - .oU'Z r N (f) s lO f- @ O) G cDg E =oG ul o o o oootr .Y Go; E' o) L oo -oo o o tr o 6o 6 o G!,o o =uiO)Io (! o OEo)(u(EN L?86b> =c o o. o. o oo a) o) L P -3ErQ .Fb; PELL>(5 O.(E !+- -cgrt >a o.)E'= .. -c O)= * or i r!J€F r- v r(UrO aL olz o 3 EL(! E EE(5! o =Ei,Do(/)= 809eAs E:(tr O= -LEo)OE(J I- -c eE o:9 .E- := .=BBt J'P8-o'- I u(tr(tr aa -;o) .=(Egp Eo 6g Ec (5 P'* '-= o oGL>F(! o U' o) tr .Y G o = C o L ts o L ro (U l- LL - C) L-Oo 'E b.c Ee o_.=E6o cE o LhU = E'E '= (g e_E gt o-(! o E oo oL Lo_ -t- =.eEaaz o o* z'a bE -o-FF =98'€6 €* E,3; FfrE o- oo:lo >EE io o O-orx !!6'o) OG ao L 9.a c2 '(E< E@ 'I E')c o U' o 0) -ooJoI .O E.C(E C=:=(tr4(!c- >r-tsoi >,o o --= a>l"o *oEHg r- .E or U.:-Y -fobo=sE c Orr g(o'= 8 .9oa6g o)e -,g O', 'i .E C= I- -tr (E i'-So- i rrr(! L o o j Eg c -c (tr =trE@a .qE =L>o) 'E o)o=(5.= F(5Oc(o'; oEro(o L sU)E9 6) o- lJ- =T(5 -or-C Fooa .oEaE o'= :l (/)Rr- H.i O EU PL_= E qboEo EC(,ir(oo6e 5tr-.<o hI.= YJ.s d* >-o oR FFI rll Fo o:ECEooog-6 o)gr:(59s(! EEg-c 'Eq LLAHEiU(!o rE sb €- (U r-E otrErr (D =" 6a= , h, X .c ao '5 .E o(! Fo O oa E o o- -L6a >,E o(ECC ,-L =E'99 s(Ei,a- L(50)69 lJ-.gIc r(U Pt'L .!FA9 ,ri6, :8(n o(5 .c .>6a oEf, 8qE -!!) 6c = oo € f',oE Eg =G,L'j c-o .e $a U_ L.rJS ;E !E8 8z QcJ o ^rArt -C o) :, oL ! -c .9 .C ,a aa oE oL(! = (tr .=C E CI-sdZ,d6 ,- .C(oE oo -o (,, .c .c(5 L E o o- o_) o Raya oo -oEooYE ts r(U Er 5E PH -!o!>(!(!(I) L_#I EE -F>oF-ot := EE:sEE _Po-e Hel o-l(J Fl<ol ,Ol o (E .9 E' E o) .E LEs(L E .o(! L o)o C .g oL oE(o oJ (,, C -Lbe '-a'e '=o I-E oLO-o )?(/)6 c o L-oH c t'L-o oriiN L.- o o-ooy > o-(E EE G o) .E= ESFI 9oKE o =(tro-EOtcOO-c FO o o o (L 6 IJJ o G E)g o =oG lrJ o .9, oi G F o =o6 oJ lr Jlr lro(o (!Eo L o 3 -oE .qa o- U) ro c(o o>, ^=(!= P)A- E OEEg ts> 6E -c.E -As* I(5 -o ro C) C 'o o_o(o Ec o o -o(! a o (5 F(5 a>, = -cc(!tr o=It o r(J .9, o(,,(5 L o oo o J c) o-(o Lo- -cF+d -a =>,E(/) a =(5uo- '= o) 9.o)'-o-coo)o= o rO6=E5 -t/)fe G)(,,(o o oO r(o o oE o -o o o = o roN b(5) L G -) o ON .(U =C(I, -) o oN -(5)L -oo I,L o roN a of,L -oo U- o oN -(6 J C(! -) o roN -(o fE(tr -) L(! =L(! -) q)io ocl coN (\. =o E' '= o E tl-(5 a l!IJ LL EC(5 o_ lJ- o = o_ LL o = o- LL o = a L o .:< o 3 -c =(6 o -c TL - (L LL o = (L LL o = o_ TL o =oo a =-J LL tr o 6 U' ooo .Y Go =o oo t, E'o o j: o EE(5 c o E G) o,(! E(! E (5 (5E c o (,,g :E oL op 2or LL(L'E c(5 rO- b o 9'E E .===([c) .= ooE.E: 9 E: -tsE5U'O ol-Ec(,iD'= cL ='{=E-Eg6 0E P"g P .c* 6 rU CIEEc>, sEe([(l)> HEop-3 3 aE8$,9 Jo(! -oEoor+ !E(! o)c '= 3 E o o-o E o o)C '=, '(5 L op oL o_ I (trll NEgg€ o- -xbo o) .E9E .=oco Ee*o) o.E EEKE E-.s ao-= -oSE E P'9.: 'Eog : FSE tr! ,qt h ".9o r AC E o Eaaq) .t)a(tr '1, o oc c o(,) .gE '6 L oE '= oL(L I c o Eaa c)oa(5 E oo L E o <,) L '=L '(5 op o o- I +)E o Eaaooa(5 Eooc c o o,E .E '(o L op oL(L I o o otr .Y Go = E o c(! o- -,O:= =o' (!3: =^(o2P EE' *Obeo-E oL C o .F oro = c-o>'E (! " E'Q z. a- 1s L o Eo o- oL Z"oc E.9c.a =>(/)d ogZ-o Eqi NJ O-EXoo o<<Lz ao *! EbEG fEE eL E=gotro ^-L€l2(!roz.-a c- oc E o Eaaooo(! Epo^zi oo o o)L(! o L .a o) .cc '(E LF E o o o)L o oE ,o U)I C' o .6. (L G E')g E =o G UJ o o oo ootr .Y G o B !,o o o -oo o o c o 6a o oJ IL q oN(o IGo t- o L o Be 0)r .Y.ioa7 -cF .90IC) Eoo .YL o = >l o) o= -o a -cO .=# -v 3'5L'EOz O >6(6 o--c)-oooo r O-C III .= L Jo oa o .Y .ot o) o)EC . 'E '6'ru LL afq PsI- .T IJEJ-r = o- o cL' wOg -i EB ao o O o L o o LL) F- @ E o -ooL o- L f,o c) U) oo '= o CL CL o € 'E ho = >.Y r-) Eg8.so o{- ooE = I : E6 3 ; E9 E =o:cE E E-9f, a O-E L\va- JJO.CCDT o G) _oq2a -Y,= UJr oq P.a X grEo Hro _3EE rrJ t< tr Fap pb q- 6O av Ec €s€ o_.NOco>7i = €ft; EOIO.E ) *'7 a H [= E = oE (Ua- a \ -8 Eg! E ,\ L,L I FL/e c o x -# a Lil E AEEE .-^F-gJ -4-rLY=o.txtao@u s) 'E trFtrFO- Xocx()- e i':6Hb- -Lo (tr'E8 : H,e ^+l&r!LJ.=Pd(J=ci; o ! .c .C .= B 9>t-..J '5'E ^E[JcOFO5 oa) o .C Io I o)af o - oootr -Y(E o = oo,Pc or'F LF E3tra -cO E o.= EE 3 - o-o.@2.Zc a lzoEr (J:O'-pE s*(!r =oo)*(1)E -o2t o (5 E.D .YEos ct)C o'EE3 OE/-\ o-5e IIII ociso =€ NqE =c o;->)-p Eb 9E= s8i 3 Eo- Eto r r (!-O oc o, .E .E(5 oEc([E-oJ-nL)ial! 0) rC E o- J(/) > .=oc(!l Ll=Oc!L(!0Eo EOo -!.Y L o o- Io o E oz E" o a -cg =o) EE ..E o)c (tr(,(tr E(tr = O--Os= UDC a =(Uh 4-N E o EE =# c'i ,-dE Er o; +qde 5g E , g?: g L o a L oE(5 o o6 -oolz Eo =E tr oFOO 'O o)C (! E o_ f a c.)aYEEE o= =E-O - G).E Y=ttU ^.-r ='tlBE Ep5€ tr-O 36 .lc o3>Ec oo o) O .C.LLo=EE'(5 o .= orh E9.d b 6EO Eo EO: oPv3s, -o=cE.9 L UJ\Yo-o .!l! c (, '=i5.e = 6dlgLJit= roz= (,, ;i o o.)CX C .-EhL AOC P o f '= E :lo= (56c = E N ,9 8t+oEL -t- orOrll ,2 C .9 9', E .=oc= EE o= Ers '.E o c!OE .,o "9o(JpJ >:l95 o-(J tl E o1lia6o oOao o aA(/)(1)c, /-l 'E (!Oc!O =-EEaEoo7 , O? -o o o)(5 L o oo s ro(o o (E .9 E'tr o) .9c c.(tr (L o- ! aL oE(! oJ q5 =:0o> '=oXo- >d o)c L o =E o = o- o- =o o,c(E .N o o Ec(o (,) .=> .=< ScnF- ooL) oa oL C(5 E = - o(,) oL o o O o o o o- troo (U E')g 6 =o(! IU o .9, oI G o =U' o o .= tr E E oo c.)(o o o o0c oo'E ES 9,'tr-,[$ooOuto aLq)E(oo- >,o =.4 =o =o-L-{cGfibg -Olr- a G) .Y o =LL -J LL I a o -o E o E =cf, E E oO oL o .YL o =LLIJ U- =c f E Eoo ro E(Uui ^oUEO(!oo ot, o g o = L o L o -oo o o 9E aab.q +J o)tco- o- o)o-c@= O oN L(! c(5 - c o f, -o L .2E c(5 .N or 0)= >5 o)oc- '= llf,(uo>l o roN (! Jc(! -) .C c o E (5 L o c o -o .E .aE LoOrtrtr <.E tr o 6 o o o o .ts o E oo E o E I o>>.=([c -c= Eotr o >o .=Cg -t€E3E-ooo-c ooo) -co o)P o)o(gG) L- :fL8=es= o UJ AE , EO E o o- oL o i; -ct -oLEoo) or.-Ecr- '=O .= o-(E(l) LL E 6O .c> o(U l()l:l ol ^L o_l -9 (/)l(J ol o,cl .E L 3 .c. .9.a @(! o aoo O E o(! o :f .If o 6E o EE G o)c o o) oa IEE(6 a =ac oo CoE a9OErrOUP AE P; .=o .c 9lg<n c o(!EI .N aE o-> o) .cL fE Eao P:O .lc > EE iitnoi ooEs.>E -E c';oo ;*=E=€ -abAc) I ooF)(/)Ea n o.l!o()ti?,9 oootrJ(E o = o o) .c ocnoc tsC c -SGO_ =Co)o9ofoooZ.E .Y C)8g o E9OF 'o ,t LOLL 3I EetE o nO-ze o)C =3 o (5 oc oE ao_ ,-1 :U# E(! .N a8(5= fUs oofACEOo'Eo(uo LLL'ooco> (!o-- ?6 rr-O a oo C oa -oG L o o- I o- o E oz o o 'd o. (U E')g 6 B oG lU o oo ooo .Y G o =E' o o o -oo o o o E U' o o =tr E E oI o -c o c o 'EO(t= Lslo)O =8_ c .o fo L .oE U) o o c o) U) -o(! o E c o .o C o) -o L ,ao !t(o I t t 65 ANNEX-s : Key Persons MeUlnterviewed in the East Wellega CDTI Evaluation Zona| LEVEU PARTNERS Wereda LEVEL HF COMMUNITY a No Name Address Position 1 Mr. Ensermu Jeldu East WolegaZone Zonal Oncho Coord inator 2 Mr. Deressa Knoo East Wollega Zone Zonal Health Desk Head 3 It/r. Kitessa Debello East WollegaZone Zonal Pharmacist 4 Mrs. Asfaw Abebech East WollegaZone Zonal Finance Desk Accountant 5 Mr. Shiferaw Bekele East WollegaZone Zonal Finance Desk Accountant 6 Mr Kitesa Debelo East WollegaZone Zonal Pharmacist 7 Deribie Mekonnen Liqht for the World Country Representative, EthioPia No Name Address Position 1 Mr. Endale Geleta Guto Gida Woreda Woreda CDC Team Leader 2 Mr. Busho Kumbi Guto Gida Woreda Woreda Oncho Coordinator 3 Mr. Tilahun Desissa Guto Gida Woreda Woreda Health Office Head 4 Mr Tesfa Tesema Leka Dulecha Woreda Vice Woreda Administrator 5 Mr. Kitessa Mosisa Leka Dulecha Woreda Woreda Health Office Head 6 Mr. Korsa Eba Leka Dulecha Woreda Woreda Oncho Coordinator No Name Address Position 1 Ms.Meskerem Wakjira Kitessa Kebele HEW 2 Ms. Alganesh Degago Loko Kebele HEW 3 Ms.Fasikie Debebe Loko Kebele HEW 4 Ms.Wubalem Bedassa Kewissa Kebele HEW 5 Ms. Askale Taye Kewissa Kebele HEW 6 Ms.Tsehay Taraegn Bedo Kebele HEW 7 Ms.Warkinsh Wakeshum Bedo Kebele HEW No Name Address Position 1 Mr, Wolda Deriessa Haro Village CDD 2 Mr Wakjira Fufa Haro village Village leader 3 Mr. Feyisa Getahum Kitessa Kebele Kebele leader 4 Mr. Temesgen Derartu Village CDD tt 66 a Terefe 5 Mr. Kebede Dembel Derartu Village CDD 6 Mr. Taye Tafesse Kambi village Village Leader and CDD 6 Mr. Fikadu Leta Mayibasi Village Village Leader and CDD 7 Mrs. Alemu Alganesh Lafto village Village Leader and CDD 8 Mr. Eba Mideksa Leqetsebela Village CDD 9 Mr. Mulatu Etana Legetsebela Village Village Leader and CDD 10 Mr. Merga Geleta Burka Woligelti Village Village Leader 11 Mr. Birhanu Negasa Burka Woligelti Village Village Leader and CDD 12 Mr. Tesfaye Keiela Boa Oda Village CDD 13 Mr. Wagari Mideksa Tume village CDD 14 Mr Alemu Derge Wanebo village Village Leader and CDD I I e \ i 67 Annex 6: Sustainability Evatuation of astt Wollega CDTI Project Evaluation Schedule Date Activi 16th May 2009 Arrival of External Team Members 17th May 2009 Preliminary discussion on plans by team mem Matters bersive lBth May 2009 Briefing at the Carter Centre, Team reshuffling Departure to the project sites Arrival at Nekemt & 19th May 2009 Meeting with the scout & sub-division of team Briefin of East Wol a zone on the Evaluation 2o'n May 2oog Zonal level interviews and documents review 21't May 2009 Guto Gida Woreda interviews and document review Visit to Kitiesa Health post and its selected communities. Visit to Loko Community and its selected communities. 22nd May 2009 Leka Dulecha Woreda interviews review. Kewissa and Bedo Health Post document review and document interviews ans z3d vtay 2009 Visit to selected communities in Leka Dulecha 24tn May 2009 Team meeting on data collation, entry and analysis 25th May 2009 Team meeting on data collation, entry and ana continues. Report writing. lysis 26tn May 2009 Review of plans feedbacUsustainabi and presentations for the an deve ment works 27th May 2009 Zonal Feedback meeting/Commencement of zonal sustainability planning meeting Tdrnlay 2oo9 Woreda sustainabil feedback meeting/development of ans 29th May 2009 Sustainability plan development conti nues 30* May 2009 Final ization of draft susta inabil ity pla ns/subm ission Report writing 31't May 2009 Report Writing continues 1't June 2009 Departure to Addis Ababa a a a { 6B 2nd June 2009 Review of draft Report. NOTF Debriefing 3'o June 2009 Departure of External Evaluators t I