Assessment of Sustainability: Tunduru CDTI project, Tanzania (4tn Year): October 10-24,2008

World Heatth Organization Assessment of Sustainability: Tunduru CDTI Project, Ta nzania (4tn Year) October 10-24,2008 Joseph C. OKEIBUN0B & Harriet HHMISI lUadE KRBUKR Oscar KHITRBH lUilliam K IS0KR SebastiAN MHflGflMfl f Hfrican Programme for 0nchocerciasis Control a d POC 2 7 0CT, 2008 !*o,{ -( *,to1OE ( Acknowledgements a IWe are grateful to the following persons and organisations for their cooperation, contributions and assistance towards the successful execution of this assignment The Director, Dr, U. Amazigo, and staff at APOC Headquarters in Ouagadougou for making available the necessary financial and logistic requirements for the success of this assignment The WR, Tanzania and WHO staff provided support, which contributed to the smooth functioning of the Evaluation Team The WR, Nigeria and his staff provided support, which contributed to the travel of the externaf team member from Nigeria The Ministry of Health, Tanzania for facilitating the conduct of the evaluation providing relevant information The District Medical Officers and the teams who provided useful information Health workers and community members in the villages who provided importa information and contributed to the success of the mission Last but not least is our immense gratitude to the team of accom drivers who drove the Evaluation Team over several kilometres of demanding roads and terrain under tough conditions a fe I Tunduru CDTI Sustainability Evaluation Page ii I a Abbreviations/ Acronyms t AIDS Acquired lmmune Deficiency Syndrome APOC African Programme for Onchocerciasis Control CCHP Council Comprehensive Health Plan CDD Community-Directed Distributor (of ivermectin) CDTI Community-Directed Treatment with lvermectin CHF Council Health Fund CHMT Council Health Management Team cssc Christian Social Services Commission DED District Executive Di rector DHMT District Health Management Team DMO District Medical Officer DOT District Onchocerciasis Tea m FLHF First Line Health Facility Tunduru CDTI Sustainability Evaluation Page iii HSAM Health Education Sensitisation Advocacy and Mobilization IMA lnter-chu rch Medical Association MCH Maternal & Child Health MOH Ministry of Health MSD Medical Stores Department NGDO Non-Governmental Development Organisation NOTF National Onchocerciasis Task Force PC Project Coordinator PHC Primary Health Care RAS Regional Administrative Secretary REA Rapid Epidemiological Assessment RHMT Regional Health Management Team RMO Regional Medical Officer ssl Sight Savers I nternational Tshs , Tunduru CDTI Sustainability Evaluation Page iv Tanzanian Shillings WHO World Health Organisation WR World Health Organization Country Representative t Tunduru CDTI Sustainability Evaluation Page v Executive summary lntroduction The Tunduru CDTI project is the second in Ruvuma region. lt covers 7 Divisions/health Districts, not covered under the Ruvuma focus CDTI project in the Ruvuma Region of Tanzania. The project has been supported by APOC since 2005 and done 3 years of mass distribution of Mectizan@. A team of independent monitors, made up of scientists from NIMR Tanzania and University of Nigeria, together with members of the Tanzania NOTF, monitored the adherence of the project implementation with the philosophy of APOC and the CDTI process, after its second distribution in2007. The results of the independent monitoring identified strengths and weaknesses in the project implementation. Remedial steps were recommended. Summary of the 2007 independent monitoring findings are contained in the box 1 below' I Box 1: Pr6cis of 2007 lndependent Monitoring Results o CDTI has taken off in all the villages. e Treatment coverage is high and actually on the increase from seventy percent in 2005 to seventy-one percent in 2005 for the entire population. . Refusals and absenteeism constituted very low proportions of reasons for not taking the ivermectin in 2006. o The programme is integrated in the District health system. . Drug and reports pass through the routine health system' o Communities ore however, currently ignoront of their responsibilities in CDTI. This is attributabte to the poor community mobilizotion, health educotion ond empowerment, o Community members do not know why they should toke ivermedin beyond the loct thot it prevents blindness and skin diseases, implying poor health education o District Heatth Officers dictate timing ond mode ol distribution to communities. ln some FLHF staff selects CDDs lor the communities. c Senior members ol the CHMT lack knowledge of the APOC philosophy o Some CDDs were not fulty conversont with the CDTI concept signifying poor quality of training and superuision. o The troining and retraining of CDDs ore inodequate both in duration ond content. ln oll the coses encountered, CDDs were trained for only 2'6 hours. o New CDDs were not trained. o Treotment was lote in coming to the communities in 2006. t Some communities experienced shortage ol drugs. Tunduru CDTI Sustainability Evaluation Page vi ,Constitution and Task of 2008 Evaluation Team ln line with the recommendations of the NOTFs in Abuja 2002, the project was presented for the mid-term evaluation in 2008. APOC constituted a team of evaluators from Nigeria and Tanzania with the mandate of evaluating the sustainability performance of the project and supporting the implementers of the Tunduru CDTI project in developing sustainability plans' The tasks were undertaken between the 10 and 24 October 2008' The evaluation in the field was carried out over a period of one week. During this period, information were gathered from desk review of relevant documents and reports, interviews with implementers and key stakeholders as well as field observation of CDTI implementation activities in sampled sites at the Regional, District/project, Divisional, First Line Health Facility and sub-village levels. Community meetings were also held with community members. This was followed by a one day feedback session and another day workshop for the development of a five-year sustainability plan by those responsible for the project implementation at the Project and Village Levels. The collation and analysis of information garnered from the field evaluation of CDTI activities in the Tanzania project area and planning of the feedback and sustainability planning workshops took three days of intense work by the evaluators. At the end of the feedback and sustainability planning workshop, the teams came up with draft plans which the District level staff were asked to take home, edit, make necessary corrections and forward copies duly signed by the appropriate authorities to APOC management in Ouagadougou. Meanwhile the evaluation team took the remaining days to revisit its evaluation findings employing the new facts from the feedback session to fine tune its conclusions' It must be noted at this point that the feed back session provided a valuable opportunity to meet a wider spectrum of participants in the implementation processes of the Tunduru focus CDTI project. For instance, the RMO and other stakeholders could not be reached during data collection. These were met during the field back sessions and their view points taken. These additional view points were reflected on the findings. These new pieces of information, consider vital in getting broader pictures of issues in the project were duly recognized and integrated into the conclusions on issues within the Tunduru focus project area. Findings from 2008 Evaluation Every village that was identified as qualified for mass treatment with Mectizann is receiving it. Therapeutic coverage for the project for the last three treatment periods (2005-2007) is >65 percent. Therapeutic coverage for 2005, 2006 and 2007 was 70.0 percent,77.O percent and 77.0 percent respectively. lt showed that coverage has been on o steody inueose. Tunduru CDTI Sustainab ility Evaluatio n Page vii There is convincing evidence that Mectizano treatment is becoming part of the culture of people of the different villages and they expressed willingness to continue with treatment for as long as it is offered. The people are beginning to demand Mectizan@. Community members associated Mectizano with a number of other correct health and social benefits. One striking benefit mentioned by a community leader is that "Mectizon@ makes them see well ond alleviote the itching problem they usually suffered.,. ", lt was also a common belief that it gives vitality and strength that enables the men to satisfy their wives. ln addition to that, a community member said "..,my scrotum wos swollen, I wos unoble to ride o bicycle ond unoble to meet my wife. But now the problems hove been solved and I con do oll that. I om ready to meet my wife onytime provided she is ready''. Some communities asked to be treated twice in a year. Community leaders have organized to counter refusals and made more people to accept Mectizan@. Communities are empowered to make all the major decisions such as deciding when and how to be treated with Mectizan@. ln all the comrnunities visited, the people indicated their decision to be treated during the dry season of the year when they are off farming and that CDDs should go house to house to ensure everyone is reached. "We decided thot distribution should be in the dry seoson when every body will be around. However, the octuol month depends on avoilability of the druq from the District ond heolth focility". fhey selected two (1 male and l female) CDDs per sub village. However, communities have not been supporting or caring for the CDDs and have no convincing reason for not doing so. They simply pleaded poverty and inability to support CDDs in spite of their appreciation the vital role the CDDs are playing in their lives. According to a village leader in Nandukutuku, "we hove not storted paying cosh but we oppreciote their contribution to our health and we support by mobilizing the people". All the same, the current community satisfaction with CDTI has the potential for creating demand for continuation of the programme, community ownership and support for distribution of Mectizano and thus enhancing sustainability. This only needs to be enhanced and sustained. CDDs were willing to continue because the tablet is helping them and their people. Some were so enthusiastic to serve as CDDs that they insisted on being made CDDs even against the wish of other members of the community. lt was noted that in Songambele sub village, the community selected a CDD but the community leader insisted on being the CDD himself against the wish of the people. However, some CDDs complained of absence of incentives. All the same all the CDDs interviewed pledged their willingness to continue to distribute Mectizan@ for as long as it is necessary. There is high-level political commitment as evidenced in the funding contributions of the CHMT. The leadership at the Provincial level and District levels is very much aware of the problems and progress in the project and steps are taken to address the problems such as relatively low coverage in some villages and the fear of side effects. The staff attitude, commitment and level of supervision are satisfactory though training and supervision are routine activities with no objective need to be addressed at the FLHF and sub I Tund uru C DT I S u sta i n ab ility Ev al u otio n Page viii Ivillage levels. According to one of the health workers interviewed , "...1 moke three supervisory visits to the CDDs before, during and ofter distribution to oscertain census update, distribution ond collection of treotment records. During supervision t olso do other octivities like sonitotion ond heolth educotion". Another health staff argued that supervision is undertaken so frequently because CDTI is not the only programme in the village. According to her, "l supervise other healthissues os well ond when I go for them I toke time to look ot whot is happening with respect to CDT|..." The Tunduru CDTI implementation team conducts HSAM when it is deemed n"."tirry. All key stakeholders demonstrated fair knowledge of CDTI implementation at the Regional and Council levels. Even the newly appointed DMO, who is just five days old in office showed knowledge on the workings of the CDTI project in Tunduru. ln the communities, leaders are targeted with information and involved in resolving problems facing the implementation of CDTI in their domain. A village leader said, "l follow up to see whot the CDDs ore doing ond to moke sure everything is going on well. At the initiol stoge we hod refusols. We educoted thot people ond everything is going on well now...". Government funding for CDTI at project level is on the increase though slight. Between 2005 and2OOT Government spent various amounts in support of CDTI implementation in the project area. The MoH spent Tshs 33,L97,883 in respect of the Tunduru CDTI project for supervision and review meetings, which the NOTF organized. The Tunduru Local Council (District Government) spent Tshs 6,6L5,O0 from the basket funds for the implementation of CDTI at the community level. Within the period, the CHMT also provided furnishings such as Air conditioner and window and door protectors to safe guard the office equipment (computers and accessories, photocopier, printer and internet facilities) the CDTI programme received from APOC, SSI and malaria control programme. All the same the major funding for CDTI is from external sources, specifically APOC and SSl. Tunduru CDTI project is in its mid term. Yet government funding is yet to approximate 50% of the funding needs of the project. Between 2005 and 2007, SSI provided technical, material and financial support of about Tshs 76,000,000. APOC, on the other hand, has committed Ishs 95,734,04L along with other technical assistance, 4WD double cabin vehicle and other material requirements to the project since inception. The evaluators were of the view that although government is disbursing funds for CDTI activities, the amounts are not increasing proportionately to the age of the project and expected decline in external funding. Nothing is done to bridge shortfalls in funding for CDTI implementation. ln some cases, project staff members use personal resources to finance activities. One of the health workers explained that, ".... I tronsport myself becouse we go to Tunduru for solaries during which time we submit our reports...". The implication is that when there is delay in payment of salaries the submission of the reports is also delayed. Tunduru CDTI Sustainability Evaluation Page ix All the same there are sufficient funds to carry out target activities and the funds are efficiently managed. APOC and SSI provided a large piece of this. Though the Government at the District level complains of limited resources in the face of competing demands, it however, pledged preparedness to take over when APOC funds stop coming. As an indication of this, the Council lent the programme Ishs 8 million in 2006 to avert a delay in the distribution of Mecitzan@ when APOC funds failed to arrived at the expected time. This money was however repaid in two installments. All the same, it is indicative of the political will of the government to sustain the programme. The evaluation team noted the availability of transport for the implementation of CDTI activities. The 4WD has so far made a distance of over 120,000 Km in three years. The vehicles and other equipment are poorly maintained. Maintenance of vehicles is undertaken only when they break down. There is no planned preventive maintenance of vehicles. Worse still, there is no plan for replacement of these vehicles when they become dysfunctional. At the lower levels, transport facilities are lacking. There are only two motor cycles for the seven Divisions/health zones covered. Some of the FLHFs still lack bicycles or have bicycles that are in a state of disrepair. The health system does not meet the maintenance cost for the vehicles. With respect to Mectizan@, there is sufficient Mectizan@ every year. However, in 2008 Mectizano arrived late. The project coordinator goes to Dar Es Salaam, with APOc fund for travels, to collect Mectizan@, while there is a drug programme that takes drugs for the health system to the Region. The Project and NOTF are thinking seriously on ways of averting a reoccurrence of such lateness in future and also getting MSD to deliver Mectizan@ for the project at the zonal drug store in Songea, administrative headquarter of the Ruvuma region, where Tunduru belongs. That way the DMO will collect Mectizan@ along with other drugs, using CHMT resources, from the zonal medical store and stop the verticalization of Mectizan@ supply chain. ln evaluating the project on the basis of the seven aspects and five critical elements of sustainability, the Evaluation Team concludes that the Tunduru CDTI is MAKING SAT,SFACIORY P ROG RESS TOW AR D S SU STAI N AB' LITY. With regard to the five critical elements the Evaluation Team found that four (supervision, Mectizan@ and political commitment) were present in the project. The elements of money and tronsport were not sotisfactory in the project. The resources were mainly from outside though there ore opportunities for increosed govemment funding. With respect to the seven aspects of sustainability, the Evaluation Team found that six, namely integration, efficiency, simplicity, attitude of staff, and effectiveness were very much helping sustainability of the project. However, resources were lound to be blocking sustainobility becouse government funding and provision of both capital ond consumoble moteriols for the progrdmme ore very minimal. The project is, indeed, over dependent on non-Government sources for resources. t T un d uru C DT I S u stai nab ility Ev a I ua tio n Page x IAtl the some, the quontitative score of 2.95 shows that the proiect is MAKING SATISFACIORY PROGRESS TOWARDS SIISTAINABILITY and would continue to progress if the remedialoctions recommended by the Evaluotion Teom ore token, There were also clear indications of the implementation of the recommendations of the 2007 independent monitoring team. One of such indications is the intensification of Training and HSAM, which brought noticeable improvement in the people's knowledge of the CDTI programme and Mectizan@ treatment. Refusals are almost eliminated. The communities know their role and take all decisions affecting distribution. This has improved demand for treatment and ownership, at least in large part. What is left is for communities to consider the issue of CDD incentives. lt must also be noted that the team still needs to work hard at getting Mectizan@ promptly. lt was observed that Mectizan@ arrived late in the project for 2008. During the feed back meeting, the Regional Administrative Secretary (RAS), who was represented by Dr H.A. Tarimo, Regional Medical Officer (RMO), Dr Daniel Malakela, the District Executive Officer, District Planning Officer, the District Medical Officer and other members of the Tunduru CDTI project lauded the findings. The participants proceeded to do a SWOT analysis of the project bearing in mind the findings of the evaluation team. They all highlighted the need to address the issues raised in the evaluation report in preparation for the phasing out years of APOC funding support. From this point and mindset the team proceeded to develop a five year sustainability plan with the evaluation finding and recommendations, taking into consideration the weaknesses observed in the project. The team also discussed ways of moving the project forward. Some of the ideas on the way forward are listed hereunder. Way Forward As the Tunduru CDTI project in Tanzania moves to the close of the period for APOC-guaranteed supporttheteam of evaluators and the'programme managers'made a critical appraisalof the issues that need to be addressed in the short- and medium-terms to ensure the sustainability of the project post-APOC. The following is a summary of the highlights of the seven critical components of the "way forward" outlined at the joint final session between the external evaluators and the operators of the Tunduru CDTI project. L. Documentation: An important area of deficiency that needs to be tackled in order to enhance the sustainability of CDTI is the relative lack of handling the programme for reporting results. This made it impossible for evaluators to find hard copies of reports in place. A series of workshops at the District level that address this shortcoming is highly desirable. Tunduru CDTI Sustainability Evaluation Page xi 2. Resource Mobilization: By mutual agreement the contribution of the various stakeholders to the sustenance of the Tunduru project post-APOC is a major challenge. It was agreed that the Regional and District Teams should take advantage of the Council budgeting to plan and provide resources, both financial and material for the sustainable implementation of the project. The commitment of all to this element of the programme was emphasized and would be given appropriate prominence in the post- APOC plan of operations. Further to this, the SSI is requested to participate in planning at the project level before the plans are sent to the Council for appropriation. This way the Council will be abreast of what is coming from the NGDO partner and of course take steps to abridge the gap in funding. lt was also agreed that it will be desirable for the Councils and CHMTs to make allocations, in the form of running credits or imprest, to the FLHFs. This way their needs, for instance the submission of reports will be facilitated. 3. HSAM: The level of ownership of the project at the community level is limited to acceptance of Mectizan@. This has, however, not translated into actually supporting the process by providing assistance to the CDDs in the form of transport to collect Mectizano and submit treatment results. lt was thus resolved that high powered HSAM activities supported by staff from the Regional and National levels should be undertaken to sensitize the community members on their roles in CDTI. 4. CDD Compensation: This was also covered under the point on HSAM. There is need to further sensitize the communities to appreciate their role in CDTI. 5. Operations Research: The evaluation team also found the new enthusiasm to distribute and take Mectizan@ very interesting in many respects and the same time awesome. The low CDD dropout rate, contradicts what is seen elsewhere. Many CDDs expressed willingness to continue to distribute Mectizan@. ln one community, a village leader, going against the wish of his community members, insisted on being the CDDs. These need to be systematically documented and to serve as reference materials for promoting community ownership. b. lt is reasoned that if these are real there are lessons to be learn from them, Beyond that however, the evaluation team observed some instability in trends of treatment coverage in the communities and health sub districts (Divisions). The team noted three categories of coverage levels over the past three years. ln one there is a consistent rise in coverage in many communities. ln the second category, there is a consistent decrease in coverage in a few communities while in the third there is no clear pattern. This is a source of concern. One of the reasons adduced is the unreliability of the population. ln Luwawa, for example, there are Mozambiquans who migrate in and out of the communities. These affect the population parameters and may affect coverage' a Tunduru CDTI Sustainability Evaluqtion Page xii :c. Thus both the evaluation and TUNDURU CDTI implementation teams discussed and agreed that it will be rewarding to conduct operational researches to ascertain the factors driving the zeal in the CDDs to continue to distribute Mectizan@ and the factors responsible for the treatment coverage in the different categories of communities listed above. The questions for the CDD study will include i. To what extent is their willingness to distribute Mectizan@ driven by altruistic motives? ii. What are the other factors that could explain the zeal to distribute Mectizan@? iii. What other benefits do they derive from distributing Mectizano? iv. The CDDs are automatically enlisted as members of the village health committees and by virtue of this participate in immunization campaigns where they are paid. They also participate in the distribution of Zithromax for Trachoma lnitiative. What is the influence of these realities to the CDDs'zeal to distribute Mectizano. For the community members the questions may include a. What is the level of awareness of Mectizan@ and the control of onchocerciasis? b. What are their perceived social and health benefits of taking Mectizan@? c. To what extent is their interest in Mectizan@ driven by the perceived social and health benefits of taking Mectizan@? d. To what extent is the interest in taking Mectizan@ driven by the perceived and actual benefits of Mectizan@? e. Will the people continue to want Mectizano if they are made to play their roles fully and support the CDDs? Tund uru CDTI Sustainability Evaluation Page xiii TABLE OF CONTENTS ACKNOWLEDGEMENTS ABBREV!ATIONS/ACRONYMS EXECUTIVE SUMMARY TNTRODUCTTON ............ 1.1 Background to the Evaluation Exercise. 1.2 Tunduru District CDTI Background....... 1.3 Evaluation Questions........... 1.4 Evaluation Objectives METHODOLOGY 2.1 Design 2.2 Population .................... 2.3 Samp1in9....................... 2.4 Sources of lnformation. 2.5 Analysis.. EVALUATION FINDINGS 3.1 Sustainability at Project Level (Tunduru District Project Level) 3.2 Sustainability at the District Level (Divisional Level for Tunduru) ...... 3.3 First Line Health Facility (Village Level in Tunduru) 3.4 Sustainability at the Village Level (Sub-Village Level for Tunduru).'... coNcrusloN ....... APPENDIX ...1t ilt 4 4 4 4 5 6 a .7 4.1 Grading the Overall Sustainability of Tunduru Focus CDTI Project. 4.2 Feedback/Planning Meetings........... 4.4Way Forward I PROJECT AND DISTRICT LEVEL WORKSHOP PROGRAMME..... ll SWOT Analysis Tunduru CDTI Project Tanzania lll Names of people interviewed during the Evaluation of Tunduru CDTI Project . IV ADDRESSES FOR EVALUATION TEAM MEMBERS.. ..39 ..39 ..45 ,,46 16 22 28 49 51 66 69 ....49 Tunduru CDTI Sustainability Evaluation Page xiv ..vi tIST OF TABLES a Table 1 Distribution of Samples in Health Districts/Division, Villages and Sub Villages Table 2 Average Sustainability Score of the Different Groups of Indicators by Levels of CDTI Implementation in Tunduru CDTI Project Table 3 Actua! Contribution of Paftners to Tunduru CDTI Implementation Tunduru CDTI Sustainability Evaluation Page xv LIST OF FIGURES Figure 1: Tunduru CDTI: Sustainability at Project Level Figure 2: Tunduru CDTI: Sustainability at District level Figure 3: Tunduru CDT!: Sustainability at FLHF Level Figure 4: Tunduru CDTI: Sustainability at Community Level Figure 5: Average Performance of the Different Levels of Tunduru CDTI Pooled Groups of lndicators Figure 6: Sustainability Score of the Groups of lndicators and Level of CDTI lmplementation in Tunduru CDT! Project Average Performance of Each Group of lndicators in the Entire Proiect Tunduru CDTI Project: Performance of Group of lndicators Trend in Treatment Coverage for the 2O05-2OO7 by Level of lmplementation Funding Contributions of Different Partners in Tunduru CDT! Project Figure 7: Figure 8: Figure 9: Figure 10 t Tunduru CDTI Sustain ability Evaluatio n Page xvi 1.0 INTRODUCTION I 1.1 Background to the Evaluation Exercise The African Programme for Onchocerciasis Control (APOC) adopted the Community Directed Treatment with lvermectin (CDTI) approach, where communities themselves play major roles in the annual mass distribution of ivermection, as its major strategy in control of onchocerciasis, a disease with proven socioeconomic and demographic impacts in Africa. APOC, which was established with the mandate of instituting structure for sustainable distribution of ivermectin, supports CDTI project proposals that are based on a partnership of WHO/APOC, the National Programmes and several NGDOs. The first grants were awarded in 1997. Each project is funded for a maximum of five years. During this period, it is expected that APOC support will decrease proportionately to that of the other partners in the partnership as the years go by and at the fifth year the cost of treatment per individual will be reduced by 90 per cent. ln addition, it is expected that the National Onchocerciasis Control Programme (NOCP) will continue to manage their CDTI activities for up to 15 years after APOC exit, in order to guarantee effective control of Onchocerciasis. APOC's philosophy for CDTI is to ensure that the recipient communities own the programme in order to sustain it. The communities themselves determine the activities. They determine where to obtain their Mectizan@, when and how to distribute it. They select their community directed distributors (CDDs), collect information about coverage that help the determination of programrne success. The CDTI is a process building up based solely on the experience of the community members, and consequently, enhancing the decision making and problem solving capacity of the communities. As part of the building up process, all APOC supported projects go through independent monitoring of adherence to the APOC CDTI philosophy in their second year and after the third year are evaluated for sustainability. Tunduru CDTI project, which has entered its fourth year is thus being evaluated to establish its progress towards sustainability' 1.2 Tunduru District CDTI Background Tunduru District is the extreme southeastern district of Ruvuma Region. lt is located between L0el.5' and LLe45' south of the equator and longitudes 36s30' and 38e east of Greenwich. Tunduru District borders with Namtumbo District to the west in Ruvuma Region, Liwale District to the north in Lindi Region, Nachingwea District in Lindi Region and Masasi District in Mtwara 1 Region to the east, and the Republic of Mozambique to the far south. lts population of 219,000 lives in a land area of L8,778 sq. km, resulting in a population density of L7.7 people per sq.km. Tunduru town, the administrative centre of the district, is 264 km from Songea, the regional capital of Ruvuma and approximately L,150 km from Dar es Salaam. The road from Dar es Salaam to Songea is a well-maintained tarmac and suitable for driving at all times of the year. However, the road from Songea to Tunduru is an earth road that is not well maintained and not easily passable during periodic heavy rains. Administratively, Tunduru District is comprised of seven divisions, 24 wards, and 107 villages averaging 2,000 persons each. There are two hospitals, five rural health centres and 35 dispensaries. The Tunduru Focus CDTI Project covers the district of Tunduru in Ruvuma Region in the far southeastern part of the region. The total population of the district is approximately 2L9,000 people, and it is estimated that approximately 52,757 people are in hyper- and meso-endemic villages. The region has an area of 78,778 square kilometers. lt was not included in the Ruvuma Focus CDTI Project because of its great distance and isolation from the rest of the region. Administratively, Tunduru is the most difficult district in the region where communication is poor and where the road network deteriorates periodically in the rainy season to a point where road traffic comes to a standstill. lt was for these reasons that Tunduru CDTI was established as a separate project for APOC support. Distribution of lvermectin in Tunduru District commenced since 1994 with support from the River Blindness Foundation (RBF). ln 1998, the Christian Social Services Commission (CSSC) assumed support for this activity in Tunduru District. The highest number of people receiving treatment was recorded in the year 1997 when 27,663 received tablets. The CDTI project was approved for support from the APOC Trust Fund in 2005 with SSI as the NGDO partner. The project has had three rounds of distribution in 2005, 2006 and 2007 with 74,236; 83,740 and 92,957 persons treated respectively. This shows a steady increase in people treated since adoption of the CDTI approach. The project underwent independent monitoringin 2OO7, just before its third distribution. Ome of the key finding of the team of monitors was that CDTI programme implementation in Tunduru District was commendable, particularly with respect to the high and rising coverage as well as the attitude of the Council Health Team and the level of integration. However, it was also noted that the level and nature of implementation at the community level needed improvement in the form of community mobilization and health education as well as training of CDDs, else the current coverage may not be sustained for a long time. Moreover, some of the leaders in the Council Health Management Team lacked knowledge of the APOC philosophy and principles behind the CDTI strategy. The team of monitors was particularly concerned with the progress of the project towards sustainability. ln line with the 2002 decision of the NOTFs in Abuja, Nigeria, the project, which is now in the third year of funding from APOC, is undergoing a mid-term evaluation for its potentials in sustaining CDTI implementation post-APOC. The Evaluation Team drawn from Nigeria and Tanzania was therefore charged with the tasks of: a Tu nd u ru C D T I S ustain ab il tty E v aluatio n Page2 ao Evaluating the sustainability potentials of Tunduru focus CDTI project o Discussing the findings and conclusions of the evaluation with the Regional, projects and supporting NGDO partners as well as with the NOCP o Facilitating the development of post-APOC sustainability plans to be prepared by the project leadership 1.3 Evaluation Questions t. How sustainable is the Tunduru Focus CDTI project? 2. What are the structures now in place to sustain Tunduru CDTI programme as APOC pulls out it support for the implementation? 3. Have the CDTI process become part of the routine processes of health delivery in the District? 4. How integrated are the support activities of CDTI into the health systems? 5. How are the Mectizan@ procurement and delivery mechanisms performing? G. What is the financing mechanism put in place to ensure the availability of local and dependable source of funding of Tunduru CDTI project when APOC pulls out? 7. What is the state of preparedness of the Local Government to maintain, replace and ensure the availability of transport and capital equipment for the continued delivery of Mectizan@ to the people for long term treatment? 8. How committed are the human resources for CDTI implementation in Tunduru CDTI focus? 9. What are the results of the CDTI project in the last three years of implementation of the project? a. Are all communities identified by REMO for treatment receiving treatment? b. ls treatment coverage > 55 per cent? c. What are the trends in both geographical and therapeutic coverage rates? 1.4 Evaluation Objectives The general objectives for the evaluation exercise are to determine the sustainability potentials of the Tunduru focus CDTI project by its mid-term of operation and assist in developing a plan for sustaining the project post-APOC a) b) c) d) The specific objectives therefore are: To assess the performance of the different groups of indicators of sustainability of CDTI projects in the Tunduru focus CDTI project To identify the factors that may block or help the sustainability of the project Discuss the outcomes of the evaluation exercise with the relevant stakeholders in the Tunduru focus CDTI project Develop plans for sustaining the Tunduru Focus CDTI project post APOC T unduru CDTI Sustainability Ev aluation Page 3 2.0 METHODOLOGY 2.1Design The design for the study is functionally evaluative however the cross-sectional descriptive design was employed as the process for data collection. This design ensured the one-time collection of data that permitted the description of a phenomenon. ln doing this with respect to the evaluation objectives and questions that need to be answered, data were collected in order to provide the analysis required 2.2 Population The Tunduru Focus CDTI has seven divisions with a total population of 277,O0O people in over five hundred (5271communities. The REA that supported the establishment of the Tunduru CDTI project put the population at risk of onchocerciasis at over one hundred and twenty thousand, seven hundred people (L20,7L41 in hyper- and meso-endemic communities. This implies that at least one in every two persons in the District is at risk of onchocerciasis. The population for the evaluation in the Tunduru Focus CDTI project, however, includes the key players in the process of ensuring long term annual treatment with Mectizan@ of the people living in the onchocerciasis endemic areas. These were the members of the Regional Medical Teams in Songea; Tunduru District Council Health Management Team (CHMT); the government at all levels, the Rural Health Management Teams; the project villages and sub-villages, their CDDs and the project's finance officer, as well as the NGDO partner (SSl) in the effort to control onchocerciasis in the area through the CDTI strategy. 2.3 Sampling Since this is a one District project, there was no sampling of District, which was covered by the project level evaluation instrument. However, to fulfil the District level obligation three out of the seven health zones/divisions involved in CDTI implementation were selected by balloting. Details of the sampling of FLHFs/villages and sub-villages are contained in Table 1. A multi stage sampling techniques was adopted. This entailed the selection of health zones within the Tunduru CDTI focus, FLHFs and sub-villages for the evaluation. First three health zones were randomly selected out of the existing five health zones in the Tunduru focus CDTI project and were included in the evaluation exercise. Two FLHFs were selected by balloting from the list of FLHFs in each sampled health zone, giving a total of six FLHF areas. The simple random sampling approach was also adopted in selecting villages from the FLHFs areas prior to , Tunduru CDTI Sustainability Evaluation Page 4 ithe evaluation visits. While in the FLHF selected villages, two sub-villages were selected by balloting in the presence of the village authorities, from the list of sub-villages produced by the same authorities. Table 1: Distribution of Samples in Tunduru Health Zones/Divisions, FLHF and Communities: 2"4 Sources of Information lnformation was collected from interviews, verbal reports and documents. Various categories of people were interviewed in the Region. These included the Regional Administrative Secretaries (RAS) Regional Medical Officers (RMO), members of the Regional Health Management Team (RHMT) and the Project Coordinator as well as the Country Representative of one of the supporting NGDOs (SSl). At the District level, the District Executive Director (DED), Council Chairman, District Treasurer, District Pharmacist and the District Health Management Teams (DHMT), were interviewed. Other persons interviewed were at the Frontline Health Facility (FLHF - Dispenser in charge) and the sub villages (village executives, sub-village leaders, CDDs, and community members). lnformation was recorded on the evaluation instruments and discussed extensively before the Evaluation Team undertook the grading of the level of performance on the indicator by level of CDTI implementation. S/No. Division Ave. R, Coverage Rate (in last Rx periods) Village/FtHF Area Sub-Villages (R* Coverage Rate) 1 Lukumbule 71.6% Kazamoyo (Lukumbule HF) Maearibi (47%l Mashariki(78%) Makande Mapunga (86%) Umoja (82%l 2 Namasakata 79.5% Makoteni Nandukutuku A (76%l Nandukutuku B (81%) ChemChem Mikoroshoni (72%l Jangwani(78%) 3 Nalasi 63.7% Chiwana Ujamaa (79%l Songambele (66%) Mbesa Luwingu (49%) Nairobi (72%l Tunduru CDTI Sustainability Evaluation Page 5 2.5 Analysis Based on the information collected, each indicator was graded on a scale of 0-4 (worst to best), in terms of its contribution to sustainability. The average 'sustainability score' for each group of indicators was calculated, for each level, and a graph was plotted. Summary statistics for the scores were calculated for each level, and for each group of indicators, and tables and graphics of these results were presented at feedback workshop. The quality of the overall project was also assessed using the different aspects and critical elements of sustainability present in the project. The five critical elements and the seven aspects of sustainability in the project were qualitatively discussed and results agreed to by the team in open debate. The project was graded using these aspects and elements in accord with APOC guidelines. The evaluators discussed qualitative description of problems, and deliberated on likely suggestions for solving the problems identified. Thus judgment about the sustainability potentials of the project was based on the quantitative assessment of the average sustainability scores of the groups of indicators as well as the qualitative assessment of the critical elements and aspects of sustainability of the project. Recommendations were generated in the format recommended by APOC. , Tunduru CDTI Sustainability Evaluation Page 6 3.0 EVALUATION FINDINGS a 3.1 Sustainability at Proiect Level (Tunduru District Proiect Level) Planning (Moderately; 1.6) There is an overall written year plan of the health service at the project level, including the regional level. This plan contained onchocerciasis, which is also listed as a priority problem in the area of focus. There is also a more detail plan, which contains all key elements of CDTI. The detailed plan did not vary for the three years of the project life. Training in the project was routinely planned. The same applied to other key activities in CDTI implementation such supervision and monitoring. ln 2OO7 and 2008 the NGDO partner was not involved in drawing the detail plan for CDTI implementation in the District. The Onchocerciasis control team drew the plan alone and submits same to the Council Health Management Team (CHMT) for incorporation into the Tunduru CDTI Sustainability Evaluatio n Page 7 CHMT plan. ln the words of the project coordinator, The NGDOs were only involved in the initial planning for CDTI in the District. According to the Project Coordinator, ...the CDTI teom members develop the plon. This is given to the DMO. So when develop in the council heolth plon it is incorporated. Then we submit it to the Notionol Office in Dar es Soloam. He further stated that, For the previous years we plonned with SSI when Mr. Mobubo wos there. Then we ogreed on what is to be done ond whot will be the contribution of SSl. With the new odministration we are left to do our plans alone and send to S5/. They then uniloterally odjust the plans to suit their resources. The implication is that there is no dialogue among partners and when the NGDO partner unilaterally drops some items there is no opportunity of getting the CHMT to take on those items dropped and they are not duly informed. All the same, the plans contained sufficient evidence of the level of integration that exist in the health system. Onchocerciasis control is planned for as an integral part and routine responsibility of the health system. Activities' timeline in the health system incorporate CDTI activities as well as other health programmes at this level. The project has also developed plan for the sustainability of CDTI when APOC funding supports ends. However this was in soft copy. There was no documented evidence of a plan for sustainability either at the project level. No reason was given for the absence of hard copy sustainability plan. Furthermore, it was not clear from the plan that was shown to the evaluation team how the project plan to fill the resource gaps that will come with the exit of APOC as it continued to allocate funding responsibility to APOC. According to the project coordinator, ...the former Notional Coordinator informed us thot APOC will continue to support the team for ot leost two more yeors after the five yeor support. We ore just in our fourth yeor. So we think thot by our eight yeor APOC will still give us support for supervision, troining ond other octivities. tntegration (Fully; 4.0) There is ample evidence of integration of activities. There is a written work plan, which shows how activities are implemented in an integrated manner. For instance, staff combined tasks during routine health monitoring/supervision. Staff combines CDTI activities with those of other programmes, where this is relevant. a Tunduru CDTI Sustainability Evaluation Page 8 !. Leadership (FuW;4.0) The leaders are fully aware of the progress, successes and problems of the project. The RAS, for instance, showed good understanding of the ramifications of the disease and added voice to the effort to control the disease in the area. According to him, ...health problems ore so mony; from HIV/AIDS to problems with the sight. We ore trying to solve some of these problems. We ore trying to improve the Regional Hospitat. As you can see we ore tying with the hardware. ln Tunduru we hove hospitol housed in old structure. Lost yedr we ogreed with the donors to put money in the basket to refurbish the hospitals. But they did not come up ofter the drowings. lf they ore operating there, it is with difficulty. Stressing his points on the problematique of onchocerciasis further, he said, ...in other diseoses like H|V/A|DS, molario and tuberculosis we hove mony interventions. But people seem to ignore diseoses of the eye. There is need to creote oworeness in the communities on the problem of the eye. Though the impact moy not be felt like HIV/AIDS, moloria and tuberculosis. lt tokes time to get blind but when it hoppens it becomes o big problem. On his part, the project coordinator said, .... The big problem is inodequote funding from the Council. We are trying to sensitize the District leoders. The problem ot the community levelwos refusols ot the initial stage. Now they ore demanding the drug. The other problem is with the CDDs. They demand more incentives. Some of the communities give different forms of incentives like selecting CDDs os Villoge Choirpersons, villoge executives etc. The leadership delegates responsibilities to colleagues and theris sound collegial working relationship. Monitoring and Supervision (Highly; 3.3) Relevant records were readily available to show treatment summaries, financial records, and inventory of equipment, among others. The records are of good quality, the contents clear, detail and convincing. Tun d uru C D T I S u sta in ab ility Ev al uatio n Page 9 Supervision was however not based any objective needs. The Project Coordinator supervises the DOTs. But according to him, "in some cases I go down to the sub villoges to see whot the CDDs ore doing". Staff member at this levelgoes beyond the level immediately below, even to the community (sub village) to resolve problem thus failing to empower the level below them to supervise activities at their own level. Monitoring/supervision is not planned to make for cost effectiveness. Officers at this level go beyond the level immediately below them thus vitiating the positive impacts of the plans for efficiency. All the same evidence show that resources for supervision are efficiently used as there exist a supervisory plan for the health system, which makes for shared use of resources. Supervisory visits are thorough. Two checklists exist at this level. One is used for strictly APOC supported CDTI activities. However, staff members routinely go for supervision of CDTI activities four times in one distribution period strictly for CDTI. According to the members of staff at this level, The teom goes out often to supervise during distribution. However, without the distribution octivities we normolly go once o month to get the community to mobilize themselves for next distribution ond other tosks needed by the project or to seek community odvice. For instonce, last yeor they advised us thot they need the drug in the dry seoson. lf we do not visit them they will not be able to tell us their preference. The project coordinator tried the justify the frequent supervisory visits when he said that, We go often to check on mistokes. Anything moy go wrong. Sometimes the drugs may not be sufficient. lf the drugs ore not ovailoble the theropeutic coveroge may drop. So we hove to go to find out whot is wrong during distribution becouse Mectizon@ distribution is the moin activity. And when you visit the community it motivotes them rother thon iust throwing in the drug ond leoving them on their own. Problems and successes identified in the process of monitoring and supervision were addressed in a very deliberate and systematic manner. As soon as problems are identified they are addressed. According to the project coordinator, For minor problems which we con solve we con discuss with the community and solve the problem. Sometimes we educote them. For instonce, they thought government should provide measuring stick. We told them thot it was wrong becouse it the responsibility of the community. We give feedbock on the performonce. We encouroge them to continue if the community is doing well. When we find oreo with poor coveroge we visit the community. This hoppened lost year ond the DMO, CHMT went to sensitize the community. a Tunduru CDTI Sustainability Evaluation Page 10 aMectizan' Procurement and Distribution (Moderately; 2.0) Mectizan@ supply is controlled within the government system only to the extent that when the drugs get to the District they are kept in the custody of the District Pharmacist. However, the drugs are collected from Dar es Salaam while other drugs used in the District are collected from the Regional Medical Store in Songea. "The medicol officer collects drug from the zonol medicol office. But Mectizan@ is usually kept in the medicol stores in Dor es Sqloom" said the project coordinator. Demonstrating the process further he said, We apply to the Notional Coordinator for Mectizon@. We ordered 544 bottles and got oll. We get o note from the NOCP ond then go to the Central Medicol Store to collect the drug. The drug is kept in the District Phormocy. Tronsportotion is provided by Government, NGDO or APOC for trovelling. other drugs come through MSD. These ore taken to the rurol oreos. For District Hospital, the District Phormacists or Medicol Officer collects the drugs from the medicol stores deportment. We hove been promised thot in future the drugs will be sent to the zonol medical stores so thot the medical officer or District Pharmacist could collect the drugs olong with others Training & HSAM (Moderately; 2.0) Staff members at his level routinely train FLHF staff every year. lt was argued that the retraining of already trained FLHF staff would ensure accurate recall. No clear evidence for the need for retraining, though this is a young project and may need more regular training for the process to be adequately internalized. The project coordinator argued that they trained twice because the independent monitors complained of inadequate training and APOC said there were lots of absenteeism. lt might be necessary to reiterate the findings of the monitoring team. The team noted that the independent monitoring team noted the poor level of details given to training and not the frequency. ln the words of the report, The training ond retraining of CDDs ore inadequote both in durotion ond content. ln olt the coses encountered, CDDs were troined for only 2-6 hours. This period is just enough to give them instructions to go house to house and administer the drug. They are not health educoted ond not told why they ore involved in the progromme. This did not justify training twice in a year Health education, Sensitization Advocacy and Mobilization (HSAM) was properly planned and carried out based on need. The project coordinator argued for instance that, "there are Tunduru CDTI Sustainability Evaluation Page 11 frequent changes in leadership of the District administration. At this moment there is a DED that is only one month old in office and a DMO that is five days old in office". With the change of leadership in the project area the team planned a sensitization of the new leadership and advocate for her support for the programme. Finance Resources (Moderately; 2.3) The costs for each onchocerciasis control activity were clearly spelt out in a budget. There was evidence of approval of expenditure, and funds for expenditures were allocated according to the approved plan of action. The DMO approved both the proposed expenditure and funds for the implementation of CDTI in line with the work plan drawn by the CHMT. The accountant had insight into each budget line and advised on the propositions for expenditures on the different activities. The Council Health Management contributes financial to the funding of activities though this is often very minimal as the Council complains that it lack funds and has other programmes to cater for. ln 2005, 2006 and 2007 the District spent 500,000 Tshs, 2,515,000 Tshs and 3,500,000 Tshs respectively from the basket funds for onchocerciasis control activities in the communities. ln addition to this, the council sometimes provided other supports to the project. According to the project coordinator, Sometimes they give funds for other expenditure such os fuelling for distribution of Mectizan@, purchose of Air conditionol, sometimes allowances for implementotion of CDTI. Molorio project donoted the internet facility However, the project Coordinator, at this level, did not have clear estimates of the funds that will be available for onchocerciasis control in the coming year as well as the expected sources and made budgets for CDTI implementation to fall within this expected and estimated income. The project management is aware of shortfalls but only rationalizes the amount available. The project planned that government will take up the responsibilities when APOC funding ends. According to the project coordinator,"the government is being sensitized. The promised they will increose budgeting when APOC funding stops". ln the meantime, all the operations for the control of onchocerciasis at this level were funded largely by APOC and SSl. There was not sufficient evidence of a cost reduction strategy, except for the integration of activities. There are though sufficient funds from APOC and the NGDO partners for the implementation of CDTI. ln 2007 the Council gave the project a loan of 8,515,000 Tshs for the implementation of activities when APOC funds delayed in coming. The project had to repay this in two installments. The evaluation team found this a concern because it means that the Council may not have filled in the gap if there was no assurance that it will be repaid from APOC funds. a Tunduru CDTI Sustainability Evaluation Page 12 aTransport and other Materia! Resources (Highly; 3.3) Transport and materials are available and functional for CDTI activities. The project has one 4WD Hilux double cabin vehicle supplied by APOC. The project has two motorcycles from SSl. APOC also supplied a Desk Top Computer and accessories as well as a laptop to the project. Other capital equipment supplied by APOC include photocopier and fax machine. Capital equipment and supplies, though functional and sufficient now, will definitely need replacement considering the work still to be done in the coming 5-L0 years. The vehicle has done over 120,500 kilometers in the last three years. There is also urgent need for more bicycles so as to ensure one bicycle per health facility. As can be noticed, all the supplies necessary for the implementation of CDTI in the area are from non-government sources. Even the maintenance of vehicle and other equipment is met by APOC and SSI and on rare occasions by Council funds. lt was also noted that the maintenance is not regular as there were no maintenance schedule. Maintenance is undertaken when equipment breaks down. There is no plan for replace of equipment and vehicle. lt was actually emphasized that, There is no plon for reploce. Replocement from government stondpoint; no hope.... Government is not in o position to purchase new vehicle.... The government needs to rethink its place in the supply of capital equipment and office requirement for the successful implementation of CDTI. The RMO demonstrated that, Human Resources (Fully; 4.0) Staff members at this level are very stable and committed to their CDTI implementation work. The project coordinator has been in the District since 1980 and has been in the project from its inception in 2005. Before the introduction of CDTI he was involved in distribution of Mectizano supported by Christian Social Services Commission (CSSC), lnterchurch Medical Association (tMA), SSland the cHMT. There is also a high level of commitment among the staff. According to the DMO the Project Coordinator is very dependable and committed to his CDTI work. ln his words, The project coordinotor is efficient. He is alwoys up ond doing, busy with the progromme. Thot is how I got my feeling thot the progromme is very orgonized ond quite impressive. Other officials at the District level corroborated this information. The NOCP staff noted that, Tunduru CDTI Sustainab ility Evaluation Page 13 ...the stoff members are committed. And becouse there was no verticol distribution before CDTI they took time to leorn ond are now very skilled Coverage (Fully;4.0) Geographic has always been L00% since inception of the programme. All the areas identified by REMO for mass treatment are under treatment with a treatment coverage range of 40% to 94% in year 2005; 55% to 88% in 2005 and 45% to 88% in 2007. This gives the impression of a decrease taking the upper limit for the three year respectively. However, average therapeutic coverage shows an increase from 7OYo,77% and 77% for 2005, 2005 and 2007. People are beginning to show more interest in the drug. Tunduru CDTI Sustainab ility Evaluation Page L4 Recommendation for the Project Level a Recommendation !mplementation Planni ng There should a written plan of action for CDTI, in which all partners are involved in drawing up. Develop a short term and long term sustainability plan showing strategies for cost reduction and dependence on reliable sources of funding Priority: HIGH lndicotors of success: a) Existence of action plan endorsed by all partners b) Minutes of panning meeting Who to toke action: cHMr/DMO Deodline lor completion : December 2008 Monitoring and Supervision supervision in order to train the staff at FLHF and devolve monitoring and supervision to them supervision from the reports on drug distribution Priority: HIGH lndicators ol success: a) Minutes of workshop on monitoring and supervision. b) Comrnunities able to conduct Commun ity self monitoring Who to toke oction CHMT/DMO Deodline for completion: End of current distribution Training & HSAM focus future plans for training on these needs sensitisation and sensitise them to support CDDs Priority: HIGH lndicators of success: a) Training targeted at areas of need b) Reports of communities sensitised Who to toke actionz DMO Deadline for completion: End of next distribution Financial Resources activities on essentials for reduction in cost funding CDTI activities by increasing its budgetary allocations and releases for CDTI implementation PrioritytH|GH lndicators of success: a) Developed plan of action targeting only the essentials b) lncreased amount budgeted and released for CDTI activities Who to take action: DMO/CHMT Deadline for completion; End of next distribution period Tunduru CDTI Sustainability Evaluation Page 15 3.2 Sustainability at the District Level (Divisional Level for Tunduru) Planning (Hiehly; 3.0) There is a general plan of action for all health activities at this level and CDTI activities were included in all the three divisional/health zones covered in this evaluation. The written work plans had all the elements of CDTI, but these were routine and not targeted on need' tntegration of Support Activities (FulU; 4.0) Support activities are planned and carried out in an integrated manner. Staff members combine activities of CDTI and sometimes, different programme activities in one trip. CDTI is integrated with eye care. teadership (Fully; a.0) The Council Health Management Team (CHMT) takes full responsibilities for CDTI activities in all the divisional/health zones of the District. There is a focal person, called district onchocerciasis taskforce member (DOT) for CDTI in all the divisional/health zones and in the District. lt is the management team at this level that initiates activities. a Tunduru CDTI Sustainability Evaluation Page 16 aThe DOTs are aware of the problems of the project. According to one of the DOTs, ...the mojor problem is the movement of health workers. Sometimes the trqined heotth workers leaves for further troining or tronsfer then we have to start oll over to train new persons.... Another problem is that this place is o mining ploce' Sometimes the young men move out to the new mining oreos ond miss treotment. The CDDs do not bother to go back to trace ond treot them. So you find treotment declining because of obsenteeism.... Another, DOT noted that, ... in the post we hod probtems of refusols. People took the drug the first time ond experienced such side effects like heodoche ond fever and decided not to toke the drug the following yeor. We ore oddressing that with heolth education ond sensitizotion ond the people ore beginning to demond the drug even for other benefits like strength.... The peopte qre even involved in CSM ond they give us reports of the CSM. There is a focal person for the health zones/division. Monitoring and Supervision (Highly; 3.3) There is routine supervision at this level, using a checklist. Sometimes there is spot check in communities. Staff at this level routinely supervises the FLHF. According to a DOT interviewed, "we supervise the FLHF. Sometimes we do spot check to sub villages closest to the FLHF to see if there ore problems". The evaluation team was however worried that the supervision was not targeted on need or information collected from previous monitoring or records. There is also no justification for doing the spot check on sub villages closest to the FLHFS. The submission of reports is within the government system. Reports are sent to the DMO's office for his information. The same reports are sent to the RMO for his information, while the Project Coordinator (PC) sends the reports to the NOCP during a trip supported with APOC funds. The same resources used for supervision and monitoring are used for transmitting reports to the project coordinator because the DOTs stay in the same office with the project coordinator. According to the DOT for Namasakata, ...we do not incur ony cost in submitting reports to the proiect coordinotor. The tronsportation costs to collect informotion in the villoges ore sometimes met borne by the District. At other times it is covered under APOC or even SSI Tu n d u ru C DT I Su stai n a b ility Ev al u atio n Page 17 There is a feedback system in place to manage problems and successes associated with onchocerciasis control at this level. Problems identified from monitoring exercises are discussed with the community leaders. For instance, a DOT said, ...when we find people who do not take drugs we educote them. You con see that initiolly, there were lorge cases of refusals. These people hoving token the drug once comploin of side effects like heodache. We educate them ond the rote of refusol is dropping.... Success stories are made open. For instance, CDDs and sub-village leaders, with success stories, are invited to review meetings to share their good experiences for others to learn from. According to the DOTs interviewed, ...in coses of good performing we proise them. We toke the CDDs to sub villoges where the coveroge is poor to discuss the drug. Sometimes if I have o pen on me I will give it to the CDD However, DOTs go beyond the level immediately below theirs and move into the communities to handle problems of CDTI implementation in the villages. Mectizan' Supply and Distribution (Fully; 3.5) The DOTs used Mectizano forms for ordering Mectizano from the District Pharmacy. Request for drugs were based on the FLHF and community data. Mectizan@ was available for the last distribution, and has been since the inception of the programme. Mectizan@ is sufficient every year. However, Mectizan@ came in late this year due to logistics of ordering the drugs. Mectizano is being collected and stored within the government system, which is effective, uncomplicated and efficient. Mectizan@ is stored in the District Pharmacy. The DOTs fetch Mectizano from the District pharmacy and deliver them to the FLHFs. The transportation of Mectizan@ to the FLHFs is done with APOC motorcycles and are fueled with APOC Trust funds. Training and HSAM (Moderatelyt t.7l DOTs only train FLHF staff, and this is done routinely every year with no objective or targeted need for the training. However, the DOTs indicated that for 2008 they trained 4 new and retrained 31 old FLHF staff whereas last year they trained and retrained 2 new and 31 old FLHF staff respectively, pointing to efforts to target training. The DOTs justified the retraining on need to ensure that the health staff recall all they were taught on how the previous year as they are involved in other programmes. On how they differentiate between training and retraining the DOT for Namasakata said, "troining tokes a longer time...". , Tunduru CDTI Sustainability Evaluation Page 18 tTraining is integrated in the different zones. Resources for training are not efficiently used as all the DOTs trains the rural health workers in the different zones, irrespective of the zones they are responsible for. Staff members identify situations where HSAM is necessary and undertake it. The HSAM activities were properly planned and have led to increased uptake of Mectizan@ by hitherto reluctant persons. However, the CDD incentive has not been successfully addressed. Financial Resources (Moderately:. t,7l Costs for each CDTI activity were clearly spelt out in a budget but there was no evidence any of cost containment strategies except for the integration of activities in the health system. Training, for instance is routine to remind people of what they were taught previously. There is no evidence of cost reduction. The DOTs have no clear estimate of the expected funds and the budget falls within the estimated income. This level of operations happens to co-exist with the project level at the District Authority level. The District Authority budgeted and released over 600,000 Tshs for the sensitization activities in 2005 and gave only 2.5 million Tshs and 3.5 million for 2005 and 2007 respectively. However, the government funding is not increasing commensurate to the needs of the project as APOC funding decreases. The CDTI implementers at this level did not even demonstrate confidence of government taking over the implementation of Mectizano to the endemic communities. According to a DOT, ...the Council promised to sustoin the progromme if APOC funding support ceoses. However, for now, they olwoys reduce our budget on the comploint that there is no money.... Nothing is done about shortfalls. The project team still expects APOC to fund the routine activities like training and supervision even after the five year agreement of APOC funding support. There is, however, a financial management process, which instills financial discipline in the system. Funds released for CDTI activities, as in the case of other health programmes in the Council area, get approval from the DMO. Transport and Other Material Resources (Moderately; 2.0) SSI provided the team with two motorcycles. These motorcycles are however ran and maintained with APOC Trust funds. Motorcycles are used in an integrated manner. However, the evaluation team did not find any trip authorization for the use of motor cycles. Further Tunduru CDTI Sustainability Evaluation Page 19 more, the vehicles are not suitable for the amount of work left to be done in the next 5-10 years. No plan is made for replacement of motor cycles by the District authorities. According to a DOT, ...we con try to request from the Council. The problem is that even when we budget for distribution octivities the omount they releose is not enough.... They complain thot they do not hove money ond they have other progrqmmes to coter for.... Human Resources (Fully; 3.5) The longest serving staff has stayed for 3 years, though staff members are not transferred frequently. The government has legislated against the transfer of its staff except on very serious grounds. Provision for training of new unskilled staff members during the next distribution period has been made. There is in-service training and two of the old DOTs have gone for further studies The staff members at this level are committed to their CDTI work. They are stable and have been in place for a long time. One of the DOTs has been in the District Health Service for 29 years. The second has been there for 15 years. Both have been in the CDTI programme since its inception in the District and have undergone in-service trainings. However, one of the Health Districts, Lukumbule Division, has no DOT and there is plan to share the responsibility of that Health District between the two available DOTs Staff members express satisfaction with their present responsibilities. According to the DOT for Namasakata "we ore happy doing the job.... We ore helping our people." Coverage (Fully,4.0) Geographical coverage has remained at L00 per cent since inception of the programmes in the Tunduru Focus. The range for the 2005 round was79.3% - 88.O% and the year before,65% - 76%.ln 2005 it ranged from58%to7L%. The average therapeutic coverage rate for 2005,2006 and 2007 were 65.5% 7O.3% and 82.2% respectively. There is a consistent rise in the therapeutic coverage for the past three years. More importantly, therapeutic coverage for the sampled health zones has remained above 65%for the three years under review. ' Tunduru CDTI Sustainability Evaluation Page 20 Recommendation for the District Leve! t District Recommendations Implementation Monitoring & Superuision target on weak areas Pnbrity: HIGH fndicators ofSuccess: a) Repofts of monitoring exercise b) Inclusion of CDTI in the government HIMS Who to take action'NOCP, DMO Deadline for ompletion: End of next distribution in 2009 Mectizan@ Supply and Distribution the supply of Mectizan@ to the health zones PrioriS: MEDIUM fndiators of Success: a) Payment vouchers for the fuelling of motorcycles for the supply of Mectizan@ to the health zones Who to take action'DMO Deadline for ampletion : End of next distribution in 2009 Training & HSAM of FLHF staff next level only on areas, where they are weak or lack skills should be conducted by the DOT for the zone/Division advocacy team from NOCP and Regional Level to sensitize the people to support CDDs Priority: MEDIUM fndiatorc of Success: a) List of training needs of FLHF staff b) Report of traininq Who to take action: Project Coordinator Deadline for completion : End of next distribution in 2009 Financial Resources essentials for CDTI implementation be increased PrioriU: HIGH fndiators of Success: a) Report on activities for 2005 b) Vouchers of government fund releases for CDTI implementation Who to take action: DMO & PC Deadline for ampletion: End of next distribution in 2009 Transport & Other Material Resources maintenance and running of vehicles and equipment control of motorcycles PrioriU: HIGH fndicators ofSuccess: a) Existence of logbook b) Vouchers of government fund releases for running transport and maintaining equipment Who to take action; DMO & PC Deadline for ompletrbn : End of next distribution in 2009 Tunduru CDTI Sustainability Evaluation Page27 3.3 First Line Health Facility (Village Level in Tunduru) Planning (Not at all; 0.0) There were no written work plans for weekly and yearly activities for all health programmes in most of the health facilities. One of the FLHF staff argued that, ... though I do not hove o work plon for CDTI I know my responsibilities ond they include troining of CDDs, issuing them with Mectizon@ and following up on the process of distribution of Mectizon@ to community members. lntegration (Fully, 4.0) FLHF staff combines supervision tasks on a single trip when supervising the Village Health Workers. Other programmes FLHF staff members supervise along with CDTI include the child and maternal health (MCH), HIV/AIDS, lMCl, Environmental/Health Servicing and bed nets for women, and compilation of data for all health programmes. , t Tunduru CDTI Sustainability Evaluation Page22 |! According to one of the health staff interviewed, "l look out for child heolth; do heolth educotion on HIV/AIDS becouse t om the coordinator os well. t olso work on Fomily Plonning." Another health worker said, I do environmentol sanitotion.... Sometimes when I go out for supervision during distribution I give heolth educotion on why the people should occept the drug and encouroge the CDDs on the distribution of Mectizon@. Furthermore, two or more CDTI tasks are combined on a single trip. And sometimes this involves a combination of CDTI tasks with those of other health programmes. Overall, integration is good but record keeping is poor. Leadership (Highly; 3.0) The FLHF staff members take CDTI as routine health service duties and take full responsibility for CDTI activities as they do for other health programmes and activities. They initiate activities, which are however dictated by the availability of Mectizan@. ln one of the facilities however staff waits for instruction from the project coordinator and DOTs. According to her, ... the project coordinotor and the DOTs coordinotor give us the instruction on when to undertake octivities like troining becouse they first troin us give us the drugs ond ask us to go ond troin the CDDs. I decide on when to go for supervision.... Monitoring and Supervision (Highly; 3.0) CDDs submit reports of the distribution of Mectizan@ to the officer in charge of their corresponding FLHF. The officer in turn submits to the District Onchocerciasis Team member in charge of the zone. The reporting is integrated within the government system. However staff uses personal fund for the submission of reports. All the same, one of the health workers explained that, ...there ore different types of reports from the heolth focilities to the District level. Some ore monthly, others ore quorterly. Whenever CDTI report is ready ond falls within o reporting period I submit it along with others. I tronsport myself because we go to Tunduru for solaries during which time we submit our reports.... The implication of this, however, is that any delay in payment of salaries will mean a delay in the submission of the reports. t Tunduru CDTI Sustainability Evaluation Page 23 The FLHF staff members supervise CDDs routinely before, during and after drug distribution, and in an integrated manner with other health programmes. Unfortunately, checklists are not used for these supervisory activities. Worse still, supervision is not targeted at need. The staff in some health facilities supervised all the CDDs routinely and in some cases 3-4 times in a year. According to one of the health workers interviewe{ ...1 moke three supervisory visits to the CDDs before, during and ofter distribution to oscertain census update, distribution ond collection of tredtment records. During supervision I also do other octivities like sonitotion and health education. Another health staff argued that supervision is undertaken so frequently because CDTI is not the only programme in the village. According to her, "l supervise other heolth issues as wellond when I go for them I toke time to look ot whot is hoppening with respect to CDT|...". The implication is that when there is delay in payment of salaries the submission of the reports is also delayed. As soon as problems are identified through supervisory visits or coverage data health staff at this level take steps to address them. For example, one of the health staff said, ...sometimes I experience resistonce to taking the drug. At other times CDDs comploin to me. ... I go to the villoge leaders to resolve the issues and health educote the community on the benefits of the drug. Another health worker said, ...when it hoppens thot there ore refusols I go with the CDDs to educote the people. But that is rore here. This year people were demonding more.... lf records ore not well kept, I correct it together with the CDDs. On successes, the health workers give the necessary feedback and encourage such performances. According to one of the health workers, "...when things go well I go to the sub villoge leoders to give them feedbock by congratuloting them for doing well...". Mectizan' Ordering, Procurement and Supply (Fully; 4.0) FLHF staff members used order forms for the ordering and procurement of drugs and request is based on requisitions from the sub villages. ln this case, Mectizan@ tablets were received as requested. Mectizano is stored with other drugs in the health facility. Obtaining the drug follows a government system for procurement and management of drugs, which is effective, uncomplicated and efficient. The DOTs collect the drugs from the District Pharmacy and supply it to the Health Facilities. The CDDs, especially those living close to the health facilities collect the drugs for their communities. a Tunduru CDTI Sustainability Evaluation Page24 Training and HSAM (Highly; 3.0) There is retraining before every distribution, but this is not targeted at any specific deficiency in the skills of the CDDs. However, some of the health workers argued that some CDDs have dropped out due to relocation and marriage making training necessary. There is also the argument of possible new instructions. Staff members identify situations where village leaders lack information and conduct sensitization and health education. Such HSAM activities are properly conducted and an evidence of the effectiveness of these activities is the new demand for Mectizan@ in the villages. However, the issue of incentive for CDDs has not been fully addressed. Only the ln-charge of the FLHF does the training, which takes place within the communities, though under the supervision of the CHMT. The trainee-trainer ratios are satisfactory; in most cases not more than 20 CDDs are trained in one session by one FLHF staff. All village leaders interviewed are aware of CDTI and help in sensitization and mobilization. Advocacy is only done at the beginning of a distribution period to remind the community leaders of the need to support the mobilization of community members to make themselves available for treatment. Finance (Not at all;0.0) Budgets are not normally made at this level. According to one of the health workers, ...before I can spend from the community heolth funds I need to get the villoge leodership to coll o villoge meeting during which they will debote ond in mony cases they comploin thot they have not money. Transport and other Material Resources (ModeratelYr 2.Ol Some staff members make use of personal transport for the implementation of CDTI activities. The government did not provide transport for staff. Those without any private transport pay their way to the communities, irrespective of the distance, to supervise distribution of Mectizan@. Officers repair bicycle with personal funds. ln one of the facilities, the bicycle is in state of disrepair and neither the village committee nor government provided spare parts for the needed repairs. The officer is still waiting on the village leader to repair the bicycle a I A bicycle used by a rural health worker needing repairs Tunduru CDTI Sustainability Evaluation Page25 . .-....-,- l 1, tr "v because "it helps during voccinotion". When interviewed, the village leader said, "l om thinking of presenting it before the village executives...". However, in Mbesa, the officer indicated that the District gives them money for running transportation Human Resources (FulU; 4.0) As is the case with the District staffing, there is stability of staff in the FLHF. The ln-charges of FLHFs visited have spent up to twelve years in their present postings. There is in-service training. According to one of the health workers, ...1 was here os o Moternol ond Child Health (MCH) Aid. I went to upgrode myself and I om now a Public Heolth Nurse (PHN) in chorge of this facility The Evaluation Team noted that in most places the staff members had enough skills and knowledge to undertake CDTI activities. Coverage (Fully,4.0) Geographical coverage has remained stable at L00% since the inception of the programme in the health areas visited. ln2OO7, therapeutic coverage rates ranged from 66.0% to 81.4% with an average of 74.4% for those FLHF areas sampled. ln 2005, therapeutic coverage rates ranged from57.3% to 98.0%with an average of 74.7% and in 2005 therapeutic coverage ranged from 58%to73.5% with an average of 66.6% for the FLHFs visited. This shows a consistent increase in the therapeutic coverage since inception of the project. More still, all the FLHFs visited scored above 65% in 2007 compared with the situation in 2005 and 2005 when one and two FLHFs visited scored below the required coverage (<65%1. ) Evaluation team interviewing a rural health worker Tunduru CDTI Sustainability Evaluation Page26 ,' ill; t,,* ; Recommendation for the FLHF Level F[HF Recommendations lmplementation Planning integrate all health programmes at this level should be documented Priority: MEDIUM lndicators of Success: a) Minutes of planning meeting b) Existence of integrated Plan Who takes action: FLHF staff Deodline for com p leti on : End of next distribution in 2009 Monitoring and Supervision areas the FLHF Priority: MEDIUM lndicators of Success: a) Report of monitoring exercise b) Copies of monitoring reports Who to take action: National Coordinator and Project Coordinator Deadline lor completion: End of next distribution in 2008 Training assessment Priority: HIGH lndicators of Success: a) List of training needs b) Report of training Who to take oction: Project Coordinator Deadline for completion: End of next distribution in 2009 Financial Resources implementation activities at this level Priority: HIGH lndicators of Success: Payment vouchers for CDTI activities at this level Who to take oction DED/DMO Deadline for completion: End of next distribution in 2009 Transport & other Material Resources transport for CDTI implementation Priority: HIGH lndicotors of Success: a) Existence of bicycles at FLHFs Who to toke action: DED/DMO De ad li n e lor completi o n : End of next distribution in 2009 T u nd u ru CDT I S u sta ina b il ity Ev aluatio n Page27 3.4 Sustainability at the Village Level {Sub-Village Level for Tunduru} Planning (Highly; 3.0) CDDs are planning and managing their CDTI work efficiently. However, census update is undertaken differently from Mectizan@ distribution. Flexibility exists for the CDDs to choose route and strategies that make his CDTI work easier to execute, though. in some areas the Rural Health Worker determines the period of treatment. The sub villages use house-to-house mode of distribution, which they found preferable to central place distribution (not all people come to central place or are at home at the time of mass distribution of Mectizan@ and coverage is better finding people at home). Leadership and Ownership (Fully; 4.0) There is communication between leadership and the CDDs, and follow-up of problems areas in a combined fashion. Community leadership takes responsibility. They try to educate those who refuse to take treatment to do so. They follow up on the distribution. According to a village leader, ...t follow up to see what the cDDs ore doing ond to make sure everything is going on well. At the initiol stoge we hod refusols. We educated that people ond everything is going on well now...". Tunduru CDTI Sustainab ility Evaluation Page 28 tFollow-up of problems by the community leadership is immediate and problems are dealt with as they arise. Help is also obtained from community members. There is also community self monitoring meant to follow up the distribution process. Some of the reports on community self monitoring from previous distributions were found in the office of the project coordinator. Community decides on the mode of distribution and selected CDDs. "The CDDs are copable ond community trusts them" says one of the leaders. The communities also decided on the period of distribution, though not the exact time. According to one of the village leaders interviewed, -.we decided thot distribution should be in the dry seoson when every body will be oround. However, the actual month depends on ovailobility of the drug from the District and heolth focility The project coordinator noted that, ...the drug orrived eorly in June but he went to MSD twice without being able to collect the drug becouse of the procedure. I only got the drug in lote September. There is deloy in getting the customs to opprove exemption. ln Songambele, the community selected a CDD but the community head insisted he wil be the CDD. Though this is not going down well with the community members, the fact remains that the CDDs came from the community and not from the health workers. Community members were able to mention a number of reasons why they take Mectizano. They also mentioned that they were told to take the drug for 15 years and indicated willingness to take it for even long periods for the benefits. According to an elderly woman in Mikoroshoni, ...we toke the drug to prevent blindness.... The benefits ore mony. lf we toke the drug, different worms ore taken core of. Those thot do not see begin to see.... A young man in Nandukutuku said, "ltoke it becouse the drug is ovoiloble....u. A village leader in Nandukutuku said, ...we are toking it to prevent blindness. Those who were affected with skin lesion ond itching get relief ofter toking Mectizan@. Those with poor sight get improved.... A young female CDD in Nandukutuku said, "the men get hyper libido ofter taking Mectizon@.... I hove experienced it". ln Umoja, a community member asked, " is it possible give us the drug twice?" Tunduru CDTI Sustainability Evaluation Page29 ln Songambele, a community member said, "this drug has olso treoted scabies. tt used to be o problem but now it is no longer o problem in our community''. Another said, ...my scrotum wos swollen, I wos unoble to ride o bicycle ond unable to meet my wife. But now the problems hove been solved ond I con do oll thot. I om reody to meet my wife onytime provided she is reody Another man in Songambele said, ... I have been sent by my community members to give o vote of thonks to this progromme. Fifteen women ond twenty men osked me to give thanks to the progromme for bringing the drug becouse the drug helps them to do better sexuolly. ln Luwawa, a man said, "l hod itching for o long time. As I took this drug the itching stopped", Monitoring (Fully; 4.0) Records are kept in notebooks with family (household) information in relation to treatment, along with a summary of the particular treatment rounds. They are also using APOC reporting forms. lnformation is transmitted to the FLHF in a timely fashion basically by the turning in of the notebooks. The CDDs compiled two registers, leaving one at the FLHF while the other is kept in the community. Obtaining and Managing Mectizano (Highly; 3.0) Treatment records, and interviews, indicate that all eligible persons who wanted treatment got treatment at the time of distribution. Appropriate amounts of Mectizan have been ordered and no problems associated with miscalculation were mentioned in any of the communities. CDDs collect the medication from the health facility and take the responsibility to arrange their transportation for the collection of medication, be this walking, bicycle, etc. The community does not arrange for transportation, but this does not seem to be a problem for these communities. HSAM (Fully;4.0) Both CDDs and community authorities are sensitive to situations that require more information and provide it as much as possible. They encourage community members to take Mectizan@. According to one of the village leaders, "...initially we hod problems of refusols. We educoted the people ond they now toke the drug". I Turtduru CDTI Sustainability Evaluation Page 30 Financing (Moderately t 2.Ol The CDDs are not given any support in cash or kind. However, the village leader in Nandukutuku said, ...we hove not storted poying cosh but we oppreciote their contribution to our heolth ond we support by mobilizing the people. We do not give ony financiol support becouse the villoge hos nothing.... We ore plonning to buy o bicycle in the future for the CDDs to trovel to the heolth focility where they collect the drugs... The village leader in Jangwani said, ...whot I do is to sensitize the community to toke the drug. We do not give ony financiol support to the CDDs. We recognize them ond we hove o plon to exempt them from community self help proiects. Human Resources (Fully; 4.0) There are two CDDs (1 male and 1 female) in each sub village. The distances covered by each CDD are short. The CDDs showed skill in accomplishing their duties within CDTI. One of the CDDs said, '...we ore trained every yeor before distribution". CDDs are willing to continue distributing Mectizan@ in the communities. This was expressed in many ways by the CDDs visited. According to CDDs in Mikoroshoni, ...we hove been distributing the drug for yeors now (since the inception of the progromme). We ore reody to distribute it for 75 yeors becouse we wont to control the diseose in our community. We are very hoppy doing the work becouse we hove been given knowledge.... ln Nandukutuku, a CDD said, ...we are willing to continue the job becouse we understand the octivities ond we ore interested to volunteer to serve the community.... We ore willing to volunteer becouse we ore serving our relations. ,fr CDDs in a community Tunduru CDTI Sustainability Evaluation Page 31 \. i" il qgr *'" Another CDDs here, said, "we ore encouroged by the willingness of the community members to take the drug.... We feel proud to serve our community members" A young male member of Nandukutuku 'B' said, "we hove no money to poy them but we respect their spirit to serve the community". Tunduru CDTI Sustainability Evaluation Page32 Coverage (FulU;4.0) ln 2OO7 therapeutic coverage ranged from 47% to 86% with an average 74.5% for communities sampled, while the 2005 and 2005 therapeutic coverage ranged from 53% to 87% and 40% to 80% respective. The average treatment coverage for the periods,2006 and 2005, were 67.3% and 64.8% respectively for the sampled communities. All the same Coverage is dropping in Mikoroshoni because of absenteeism. lt was also noticed that coverage was very low in Magaribi $7%l for 2007. CDDs do not seem to go back to those absent because of mining activities during distribution. Another reason given was the fear of side effect. Tun du ru CDTI S u stai n ab il ity Ev al u a tio n Page 33 Recommendation for the Village Level Recommendations !mplementation Planning undertake census update at the same time with distribution of Mectizan@ to reduce work burden Priority: MEDIUM lndicators ol Success: a) Distribution report Who tokes action: FLHF staff and DOTs Deadli n e lor com p letio n : End of next distribution in 2009 Monitoring arrange transport for CDDs to submit reports Priority: MEDIUM lndicotors of Success: Transport provided for CDDs where necessary Who to toke action: Village leader and FLHF staff Deadline for completion: End of next distribution in 2009 Obtaining and Managing Mectizan@ transport for CDDs in distant location to collect Mectizan@ for their communities Priority: HIGH lndicators ol Success: L. Transport provided for CDDs in distant locations Who to take action: Community leaders and FLHF staff Deadli ne for comp leti on : End of next distribution in 2009 Financing in CDTI implementation, especially on financial support for the programme Priority: HIGA lndicators of Success: Evidence of community financial support for CDTI implementation Who to take oction: DOTs/FLHF staff Deodli ne for comp leti on : End of next distribution in 2009 Coverage coverage to educate the peoPle on the benefit of taking Mectizan@ revisit home with absentees during treatment Priority: HIGH lndicators of Success: lncreased coverage in every village Who to toke action: DOTs/FLHF staff Deodline lor completion: End of next distribution in 2008 Tunduru CDTI Sustainability Evaluation Page 34 3.5 Comparative Analysis of the Sustainability of the Four Levels All scores awarded during the evaluation to the Tunduru CDTI project for the various sustainability indicators groups are shown in Table 2. The overall score was 2.95 points. The FLHF had the lowest average score for CDTI implementation (Figure 5). The FLHF scored 'Moderate' (average 2.7) compared with other levels which clustered around the "High' sustainability potential rating (2.9-3,1). Tabte 2: Average Sustainability Score of the Different Groups of Indicators by Levels of CDTI in Tunduru Focus CDTI The weakest groups of indicators for the FLHF level included finance (0.0), planning (0.0) and transport (2.0); and highest score (4.0) was awarded for coverage in the FLHF level. lt is also important to note that coverage scored fully at all levels. Fig. 5: Average Perforrnance of Different Levels in Tunduru CDTI Pooled Groups of lndicators <l s .99o =oJGO IE (t, 3.L 3 2.9 2.4 2.7 2.6 2.5 -."€ ""--" **- """* on*""' GROUPS OF INDICATORS eg (g N (J o = E vtE o rJ tr .!g t o CLth tr lE tr lE E J o oo IE o oI Alltevels ErO .gtrg l! E c .9 (E u0o E CL E .a oEl!o ooE =o .Eo'Ec,ooo =a 2.92.O 2.O 2.3 1.8 4.0 4.0Project 1.5 4.0 4.0 3.3 3.5 4.0 3.1District 3.0 4.O 4.O 3.3 3.5 1.7 1.8 2.0 4.O 4.0 2.7FtHF 0.0 4.0 3.0 3.0 4.O 3.0 0.0 2.0 4.0 4.0 3.1Community 3.0 4.O 4.O 3.0 4.0 0.0 2.7 1.0 1.9 3.9 4.0 2.95At! 1.9 4.0 3.8 3.3 3.1 Tunduru CDTI Sustainability Evaluatio n Levels Page 35 The performance of the groups of indicators is further presented graphically in Figure 6. The frontline health facility (FLHF) leave is still shown as weakest in planning, with no points at all. The weakest point for the District was in finance. The community level's weakest point was finance. Generally, finance was the weakest of all the groups of indicators. Transport and other material resources, then of course planning follow this. Fig. 6: Sustainability Score of Group of lndicators and Levels of lmplementation in Tunduru CDTI Project 4.5 4 3.5 3 2.5 2 r.5 1 0.5 0 r Project I District I I.LHI. r Cummunity I AII -s.$.{-"..*-{rt''*-.1$""i"S Mean sustainability scores for the groups of indicators are shown graphically in Figure 7 below. The weakest groups of indicators, overall, were the provision of Finance (1.0), followed by Transport/Material resources (1.9), planning (1.9). These findings are good reflections of the performance of the groups of indicators in the three years under review. The Evaluation Team made important recommendations in all of these areas. These concerns were addressed in the post-APOC sustainability plans prepared in order to move the project towards being fully sustainable as APOC support ceases finally. Tunduru CDTI Sustainability Evaluation Page 36 FigT: Average Performance of Each Group of lndicators in Tunduru CDTI Proiect 4.5 3 2.s z 1.5 1 0.s 0 3.8 3.3 3.1 2.95 The box plot below shows the actual performance range of the different groups of indicators across the four levels of Tunduru project administration. lt gives the actual performance range, which cannot be discerned from the bar charts. lntegration and coverage were constant and lacked variability in the performance of these groups of indicators across levels. For the other groups of indicators, there were wide ranges of performance, hence the need to highlight this in the box plot. The box plot shows that finance and transport did not only performs poorly, but the performance across the four levels for these groups of indicators range from a low 0.0 and peak at approximately 2.3 in the case of finance and 1.8 to 2.0 points in the case of transport and other material resources. The median score for these groups of indicators is <2.5 for finance and transport across levels. The median score for human resources, coverage and monitoring groups of indicators were >2.5 points. All the level of implementation scored between 0 and 3.0 points on planning. The community members expressed satisfaction with Mectizan@. ln many cases they demonstrated the strength they gained from taking Mectizan@. ln some cases, the people wondered why they would not be allowed to take the drug twice in a year for greater and sustained effects ofthe drug in the body. Tunduru CDTI Sustainability Evaluatio n Page37 3-9 .+ Similarly, the personnel involved in the handling of Mectizano at all levels are happy with the programme. For most of them, satisfaction with the programme derives from the fact that it has provided them with the tool to do more work. At the community level, the satisfaction derives from the joy they observe on the faces of those who take the drug. They are happy to be associated with that which brings joy to their people. Fig. 8: Tunduru CDTI Prolect:Performance of Group of Performance on Planning and finance varied greatly among the different levels of implementation of CDTI. For instance, sustainability performance on planning ranged from as low as 0.00 points in the Health Area (FLHF) level and 3.00 points at the district and community levels with a median score of 1.8 points. This shows that while CDTI is planned for at the Ditrict and community levels no planning takes place at the FLHF level. Similarly, the sustainability performance on Training and HSAM varied from as low as 1.7 points at the District level to as high as 3 2 o ,+% nu^ ?e" 4.0 at the community level with a median score of 2.50 points in the entire project areas. The District level staff still need to intensify HSAM and target training for greater results in the implementation of CDTI in the project area. T ? + ;I Tunduru CDTI Sustainability Evaluation Page 38 4.0 CONCLUSION 4.1 Grading the Overall Sustainahility of Tunduru Focus CDTI Project. Making a judgment of the project in terms of the seven aspects of sustainability (a) Make a judgment of the project, in terms of each of the seven 'aspects' ot sustainability: Judgment: to what extent is this aspect helping or blocking sustainability in , this project? ; HETPING Resources BTOCKING {EtflltG HETPING HETPING Effectiveness HELPING a lntegration There was an adequate amount of integration CDTI into the health systems at all levels of the project implementation. Staff at the project, District and FLHF levels combined diverse programme tasks on each single trip to the levels immediately below. For instance, at the FLHF level, staff on a monitoring visit for CDTI do give health education and sensitization talks and in some cases check on other programmes like environmental sanitation, child health to mention a few. The evaluation team thus concludes that the current state of integration in the project is helping it towards sustainability. ln ration Aspect Efficiency si Health Staff Acceptance of Commun ownershi HETPING Tunduru CDTI Sustainability Evaluation Page 39 a Resources (Human, Financial and Material) The government contribution has so far been minimal and limited to the provision of manpower and a little amount for fueling and training of health workers, which is even on the decline. Government does not meet the running costs for transport. Over reliance on support from APOC and SSI could block sustainability. a Efficiency: There is a rational use of resources. Programme support activities are being planned and shared, which results in lower cost and efficient management of resources. Examples are sharing of logistics. The integration of activities also makes for efficient use of resources and hence helping sustainability o Simplicity: The project uses simple and uncomplicated procedures for the implementation of CDTI activities. Both processes of accessing funds and Mectizan@ are largely within government system and uncomplicated. Mectizan@ supply and funding have never been found wanting due to any complex process. This is thus judged to be helping the progress of the project towards sustainability. a Attitude of Staff: At all levels, there is ample evidence that the staff members are positively disposed to continue CDTI implementation. The Evaluation Team rated the attitude of staff at all levels towards CDTI to be very positive. The health workers at all levels take CDTI as their routine responsibility. To a large extent they initiate activities within the project. The evaluation team considered the present attitude of the health staff towards CDTI as helping the Tunduru CDTI project towards sustainability. a Community Ownership: Communities make all the decisions in the implementation of CDTI and gave sound rationale for their decisions on CDTI activities within their respective domains. They are quick to mention the social and health benefits of taking Mectizano. The people are increasingly demanding for Mectizan@ and in some they want to take it twice in a year. Tunduru C DTI S ustainab ility Evaluatio n Page 40 a Effectiveness: Geographical coverage rate has remained 100 per cent since inception of the project in 2005. Therapeutic coverage rate has been 55 per cent and is on a consistent increase Fig.9: Trend in Treatment Coverage for2005-2007 by levels of 90 82.2 80 74,t - ppslsct -Di(trict*FLHF -(9nnl1plty 74,5 70 .3 64.8 50 --1- 2005 2006 za07 (b) Next, the Evaluation Team examined the five key aspects of the project - 'uitical elements' of sustainability. lf these are not present it is unlikely that the project will be sustainable: Money: ls there sufficient money available to undertake strictly necessary tasks, which have been carefully thought through and planned? (Absolute minimum residual activities). Tronsport: Has provision been made for the replacement and repair of vehicles? ls there a reasonable assurance that vehicles will continue to be available for minimum essential activities? (Note that 'vehicle' does not necessarily imply '4x4' or even 'car'), Superuision: Has provision been made for continued targeted supportive , supervision? (The project will not be NO NO T und u ru C DT I Susta in ab il ity Ev alu ati o n Yes Page 41 Isustained without it). ; Mectizon@ supply: ls the supply system , dependable? (The bottom line is that enough drugs must arrive in villages at the time selected by the villagers). Politicol commitment'. Effectively demonstrated by awareness of the CDTI process among policy makers (resulting in tangible support); and a sense of community ownership of the programme. Fig, 10: Funding Contributions of Different Partners in Tunduru CDTI Project YES YES Money The project lacks sufficient money to undertake necessary CDTI activities, hence it scored 1.0 point. The CHMT and MoH is not making increasing contribution commensurate to the age of the project. Contributions of the NGDO are on a steady decline. APOC contribution nose- deeped in 2006 and rose again. The Government failed to meet the gap created by this decline in APOC funding. 50,000,000 , 45,000,000 -' 40,000,000 35,000,000 30,000,000 25,000,000 20,000,000 15,000,000 10,000,000 5,000,000 0 I I I I -APOC -s5l*MoH -GIMT Worse still funding from government sources, CHMT and MoH is rather crawling even as the financial contributions of the NGDO were decreasing. The actual amounts of the different partners are contained in Table 3 below. 200s 2006 2407 Tunduru CDTI Sustainability Evoluation Page 42 Table 3: Actua! Contribution of Partners to Tunduru CDTI Project lm ementation in since I n to2OOTI Though government funding for CDTI at project and district levels is increasing the increase is not in the proportion expected within the APOC philosophy. Consequently, the project relies largely on APOC and NGDO for the financing of programme activities. Between 2003 and 2007 the MoH spent Tshs33,197,883 on the Tunduru Focus CDTI project. However this money went mainly for operations at the National level. The CHMT also spent Tshs5,615,000 for the same period under review. This again was from the basket fund and the monies were set aside for the financing of programme activities at the community level. No budgetary provision was made from the Local Council for the funding of CDTI activities at the project or divisional/district level. Hence the project had to depend almost solely on the provisions from SSI (Tshs76,000,000) and APOC (Tshs95,734,O4Llfor its routine activities. Transport Provision has not been made for the replacement of vehicles. Every person interviewed declared the inability of government to replace the vehicles used for implementation of CDTI at all levels. Routine presumptive maintenance of the vehicles was absent in the project. Vehicles are only repaired when they break down. The cost of maintenance is met with APOC or SSI funds in rnost cases. The project was thus scored 1.9 on transport for the foregoing reasons. Supervision Adequate provision has been made for supportive targeted supervision in the project at all levels. With the integration of CDTI and Eye Care programme the project is well positioned to conduct supportive supervision of CDTI in areas of weakness as the Eye programme is undertaken. Moreover, the health service has also a unit of supervision and monitoring which provides support for CDTI supervision. The project was scored 3.3 points on supervision. 'APOC also provided capital equipment and other technical (non-financial) support 'SSI also provided capital equipment. The financial expenditure in year 1 was for the construction of office complex for both the CDTI and Eye Care programmes 3 NOTF support was mostly in the form of technical assistance (review meetings and supervision by NOCP staff) a This fund from the CHMT were from the basket fund spent at the community level AMOUNT RECEIVED TOTALYear APOC (Tshs)1 SSI (Tshslz NOTF/MoH3 (Tshs) CHMT (Tshs) 11,515,990 600,000 89,754,143YEAR 1: 2005 37,538,363 40,000,000 YEAR 2:2006 9,970,474 19,000,000 11,340,893 2,515,000 42,766,167 YEAR 3: 2007 48,285,204 17,000,000 70,24L,000 3,50o,ooo 79,026,2U TOTAT:3 Years 95,734,04L 75,000,000 33,197,883 5,615,000 2L1,546,924 Tun d u ru CDT I S u stai n ab ility Ev al u a tio n Page 43 Mectizan@ There is always sufficient amount of Mectizan@ in the project, which also arrives the communities early. The process of ordering, procuring and supplying Mectizano is uncomplicated and very efficient. lt was thus scored 3.1 points. Political Commitment The policy makers in the health service are committed to CDTI implementation. CDTI is considered part of the minimum package for health in the country. lt is the communities have not started supporting the CDDs. All the same, the communities appreciate CDTI and Mectizano and for various reasons demand treatment. The policy makers in the health service recognize this weakness at the community level and are planning sensitization activities to exploit the present high demand for Mectizan@ and shift the responsibility of supporting CDDs to the communities and make the communities own the programme. The project was scored 3.8 points on leadership, which approximates to political commitment amonB the groups of indicators. ln line with the guideline for grading the whole project using the seven aspects and five critical elements of sustainability the Evaluation Team concludes that the Tunduru focus CDTI project is MAKING SATISFACTORY PROGRESS TOWARDS SUSTAINABII.ffY. AII the same the team found one of the element, resources are not dependable and blocking sustainability because they come largely form outside the government systems. Two of the aspects, money and transport were also not in the project in a quantity that could be considered dependable for sustainability. The quantitative score of 2.95 also supports the qualitative decision which holds that the project is making satisfactory progress towards sustainability. Evaluation team grading Tunduru CDTI proiect Tunduru CDTI Sustainability Evaluation Page 44 *, d* .16'i , .x: l. iI \ F 4.2 Feedback/Planning Meetings One joint feedback/planning meeting, for Provincial and District level teams was successfully organized. The objectives of the workshop were as follows: To give feedback on the evaluation findings by the Team of Evaluators Discuss the findings among the implementers, the policy makers and the Team of Evaluators Discuss the concept of sustainability in relation to the Tunduru CDTI project Develop plans for the sustainable implementation of Tunduru CDTI for the coming five years The workshop lasted for two days. The programme is attached The Ruvuma Regional Administrative Officer (RAS) declared the meeting open. ln his opening remarks he noted that onchocerciasis got recognized as a disease of public health importance in Tunduru District in 1994, following a REMO which identified 14 hyper endemic villages. The Sight Savers lnternational, Christian Social Service Commission, lnter-church Medical Assistance (lMA) and government made efforts to control the disease using the CBIT approach. ln 2002 REMO was conducted in Namasakata, Nampungu and Lukumbule where more villages were identified to be hyper endemic for onchocerciasis. Community directed treatment with lvermectin approach was officially launched on 26th January 2005. Since the introduction of the approach treatment has remained high (>65o/ol every year. This effort covered. The RAS thus congratulated and thanked all those who made the project such a success, especially the communities who are the beneficiaries of the project. He also called upon those involved in one way or the other to play their roles in order to sustain the success achieved so far. At this point he pledged the support of the regional administration and requested the RMO to do all in his power to ensure adequate support for the project. Other participants, the DED, Tunduru Council Chairman and RMO, in particular pledged their support for the project. They made meaningful suggestion on ways of addressing the concerns raised in the evaluation report. The team also went to the office of the NGDO partner (Sight Savers' lnternatlonal) to debrief the staff on the findings of the evaluation team. ln response to the findings, one of which was the non-participation of the SSI staff in the planning of the project team and of course the implications of creating resource gaps which could not be filled, the SSI team pleaded understanding. According to the Senior Programme 1. 2. 3. 4. Regional Officers at the debriefing meeting Tunduru CDTI Sustainabiliy Evaluation Page 45 m -*? Officer, the office is very thin and the staff is over stretched. The Country Representation also expressed his regrets for his inability to attend the debriefing, They however, showed appreciation for the evaluators' concerns about the non-involvement of the SSI team in the yearly planning of the project team. They suggested that it might be better, in future, to first plan in the SSI office and show the projects what the NGDO will support. These can be included in the project plans and that will avoid a situation where gaps are created if the projects and Districts conclude plan before coming to the NGDO level. On the other issues, such are the different coverage rates noticed and the issues surrounding CDD incentive, the SSI team noted that these are very important findings and also bought into the need for operations' research. According to the Country Representative, the organization will like to support such operation research in Mbeya and Kilosa Districts where they plan to try out some new ideas. lt is however, noted that the findings are with reference to Tunduru District CDTI. Beyond the debriefing, the District team proceeded to develop a five year sustainability plan. To do this, the District team, which included the District Medical Officer (DMO), District Executive Director(DED), District Planning Officer(DPLO) and the Chairman of the District Legislative Council as well as the CDTI project coordinator and District Onchocerciasis Team (DOT) members, first proceeded to do a SWOT analysis of the project. They identified opportunities that exist in the project area to enhance CDTI sustainability. Based on this, they made judicious cost sharing and allocation of responsibilities to the NGDO partner, District Council as well as the CHMT. ln line with the conditions for further APOC support after the five year arrangement, the team mainly requested APOC to assist it with the replacement of some capital equipment in view of the work that needs to be done in the next 5-10 years post APOC. They also requested APOC support for operations research and the sensitization of new policy makers at the District level. 4.4 Way Forward As the Tunduru CDTI project in Tanzania moves to the close of the period for APOC-guaranteed support the team of evaluators and the 'programme managers' made a critical appraisal of the issues that need to be addressed in the short- and medium-terms to ensure the sustainability of the project post-APOC. The following is a summary of the highlights of the seven critical ':i. District team developing sustainability plan .:i I t t V", .^$* h*" h; j i -, -.* ,r {e * t Tunduru CDTI Sustainability Evaluatio n Page 46 i components of the "way forward" outlined at the joint final session between the external evaluators and the operators ofthe Tunduru CDTI project. 1. Documentation: An important area of deficiency that needs to be tackled in order to enhance the sustainability of CDTI is the relative lack of handling the programme for reporting results. This made it impossible for evaluators to find hard copies of reports in place. A series of workshops at the District level that address this shortcoming is highly desirable. 2. Resource Mobilization: By mutual agreement the contribution of the various stakeholders to the sustenance of the Tunduru project post-APOC is a major challenge. It was agreed that the Regional and District Teams should take advantage of the Council budgeting to plan and provide resources, both financial and material for the sustainable implementation of the project. The commitment of all to this element of the programme was emphasized and would be given appropriate prominence in the post- APOC plan of operations. Further to this, the SSI is requested to participate in planning at the project level before the plans are sent to the Council for appropriation. This way the Councilwill be abreast of what is coming from the NGDO partner and of course take steps to abridge the gap in funding. lt was also agreed that it will be desirable for the Councils and CHMTs to make allocations, in the form of running credits or imprest, to the FLHFs. This way their needs, for instance the submission of reports will be facilitated. 3. HSAM: The level of ownership of the project at the community level is limited to acceptance of Mectizan@. This has, however, not translated into actually supporting the process by providing assistance to the CDDs in the form of transport to collect Mectizano and submit treatment results. lt was thus resolved that high powered HSAM activities supported by staff from the Regional and National levels should be undertaken to sensitize the community members on their roles in CDTI. 4. CDD Compensation: This was also covered under the point on HSAM. There is need to further sensitize the communities to appreciate their role in CDTI. 5. Operations Research: The evaluation team also found the new enthusiasm to distribute and take Mectizan@ very interesting in many respects and the same time awesome. a. The low CDD dropout rate, contradicts what is seen elsewhere. Many CDDs expressed willingness to continue to distribute Mectizan@. ln one community, a village leader, going against the wish of his community members, insisted on being the CDDs. These need to be systematically documented and to serve as reference materials for promoting community ownership. T u n d u ru CDT I S u stain ab ility Ev aluati o n Page 47 b. lt is reasoned that if these are real there are lessons to be learn from them. Beyond that however, the evaluation team observed some instability in trends of treatment coverage in the communities and health sub districts (Divisions). The team noted three categories of coverage levels over the past three years. ln one there is a consistent rise in coverage in many communities. ln the second category, there is a consistent decrease in coverage in a few communities while in the third there is no clear pattern. This is a source of concern. One of the reasons adduced is the unreliability of the population. ln Luwawa, for example, there are Mozambiqueans who migrate in and out of the communities. These affect the population parameters and may affect coverage. c. Thus both the evaluation and TUNDURU CDTI implementation teams discussed and agreed that it will be rewarding to conduct operational researches to ascertain the factors driving the zeal in the CDDs to continue to distribute Mectizan@ and the factors responsible for the treatment coverage in the different categories of communities listed above. The questions for the CDD study will include i. To what extent is their willingness to distribute Mectizano driven by altruistic motives? ii. What are the other factors that could explain the zeal to distribute Mectizano? iii. What other benefits do they derive from distributing Mectizan@? iv. The CDDs are automatically enlisted as members of the village health committees and by virtue of this participate in immunization campaigns where they are paid. They also participate in the distribution of Zithromax for Trachoma lnitiative. What is the influence of these realities to the CDDs'zealto distribute Mectizan@. For the community members the questions may include a. What is the level of awareness of Mectizano and the control of onchocerciasis? b. What are their perceived social and health benefits of taking Mectizano? c. To what extent is their interest in Mectizan@ driven by the perceived social and health benefits of taking Mectizano? d. To what extent is the interest in taking Mectizan@ driven by the perceived and actual benefits of Mectizan@? e. Will the people continue to want Mectizan@ if they are made to play their roles fully and support the CDDs? Tunduru CDTI Sustainability Evaluation Page 48 APPENDIX DAY ONE PROJECT AND DISTRICT LEVEL WORKSHOP PROGRAMME Sustainability of Tunduru Focus CDTI in Tanzania "Feedback"/Planning Meeting AGENDA t Registration of Participants 8.30-9.00 Secretariat 2 lntroduction of Participants 9:00 - 9:05 All 3 Welcome and Opening Remarks 9:50 - 9:15 RAS 4 lntroduction to the workshop; What are the objectives What is sustainability Methodology for Evaluation 9:15 - 10:00 Dr. Okeibunor 5 Tea Break 10:00 - 10:30 All 6 "Feedback" on achievements, issues and lessons from the evaluation on sustainability of CDTI of Tunduru Focus CDTI Project Level (Tunduru District) District Level (Health Zones) FLHF Level (Village) Community Level (Sub Villages) i 10:30 - 11:15 William Kisoka Oscar Kaitaba 7 SWOT Analysis 11.15 - 11.30 Dr. Kabuka 8 Group Work Discussions on problems identified and the solutions to these problems using SWOT analysis in groups: 1. Planning/lntegration/L eadersh ip/Monitoring & Supervision 2. Mectizan/Finances/Tr aining & HSAM 3. Transport/Human/Cov erage 11.30 - 12.15 9 Report from Groups & Discussions 12.15 - 13.00 Dr. Kabuka 10 LUNCH 13.00 - 14.00 All Activity Time FacilitatorItem T un d u ru CDT I S ustai n ab il ity E v al uatio n Page 49 ) 1,L Roles of the different levels and partners 14.00 - 14,30 Harrieth Hamisi 72 Steps in Planning for sustainability in this project and grouping 14.30 - 15:00 Sebastian Mhagama 13 Tea Break 16.00 - 16.15 All L4 Group work L5:15 - L5:45 All 15 Closing Remarks L6.45 - L7.00 Chairman for Day 1- 16 General Matters/Announcements L7.O0 - L7.L5 DAY TWO 8:30 - 9:00 Secretariat1 Registration of Participant 9:00 - 9:10 ChairReview of previous day's activities and lntroduction to the day's activities 2 9:10 - 10:00 All4 Resumption of Group Work All10:00 - 10:305 Tea Break 10:30 - 12:30 All5 Resumption of Group work 12.30 - 13:30 All7 LUNCH 1.3:30 - L4:30 Sebastian Mhagama 8 Presentation of Group Work All14:30 - 15:009 Group work to incorporate corrections 15:00 - 16:30 All10 Tea Break Evaluation Team NGDO Rep NationalCoord. RAS/DED/PC 15:30 - 17:007L Way Forward: APOC NGDO Partners MoH Regional Administration & Local Govt. Secretariatt7:0O - t7:L5L2 GeneralComments t7:I5 - 77:30 RMOClosing remarks13 Ite m Time FacilitatorActivity Tundu ru CDTI S ustainability Evoluatio n Page 50 II SWOT Analysis Tunduru CDTI Proiect Tanzania: Group A E o o G .9E' s Strength Weakness Opportunity Threat Suggestions lmprovement o '6' o. o0 .cc C(E CL . There is an overall health plan in the delegation . The plan has a section on CDTI . There is detailed plan for CDTI . There is integration in the system . There is a three year sustainability plan (2009- 20t1) o The plan did not vary for the last three years o The NGDO partner (SSl) does participate in the planning r Plan does not show how resource gap would be filled Availability of plarrning expertise. Availabilitii of fLrnd. Attitude of NGDO staff (SSl) to worl< together duri ng plarrning Unallocated of fund in tirne. lnvolve NGDO partncr (SSl) d uring planning activities. Funds to be given on time c .9(t L oo(I) E . Support activities are planned and carried out in an integrated manner . This is the practice every year a All stakeholders arc awarc of the CDTI project. Presence of commurrity hased programs which are willing to integrate. Donor's decision on running the p rog ra rn. To continue in intcgration rvith other stakcholders T u n d uru C DT I S u stain ab il ity Ev al u atio n Page 51 E o o l! !, E Strength Weakness Opportunity Threat Suggestions lmprovement .9 ! o 'o(E oJ . Leaders are aware of the progress and problems in the project . Some of the leaders demonstrated deep thinking about the problems and ways of solving them . Coordinatio n meetings in the CHMT Freguen I changc o{ Government and Political leadc,rs, Training to enhancc sustainability of pr ojcct. t, IA i5 o0 .Ecc(E E . CDTI is integrated into the overall written plan for the health district . The plans make provision for all key CDTI activities . Plans are routine and not targeted Availability of working equipments. Availability of human rcs{) u rce. Availability of planning gLridcline. Availability of funds. lnadequate fu nd. t-ack of training in planning. Proper allocatir:n of fLrnds to division level. Proper trarning to Fl-l-lWs. Tunduru CDTI Sustainability Evaluation Page 52 E o o G .9 =,s Strength Weakness Opportunity Threat Suggestions lmprovement c o P ro o0q, c . The various support activities are planned and implemented in an integrated manner . CDTI is also integrated with Eye care Ail stakeholders arc irwar c ol CDTI projcct at this levol. Presence comrnunity based proSrams which willing i ntegrate ol are to Donor's decisiorr ru nrring progr0m or"l the To continue with intcgration with other stakeholders at this levcl. .9 .C, oE(o oJ . Manageme nt team at this level initiate the key CDTI activities . The leadership at this level is aware of the problems and progress in the project Adequate project staff Availability training leadcrship skills. of in Freq u ent change of Government and Political Icadcrs. Training of leadership s kills to enhance the s Lr s ta ina b ility o{ the project. I b0 .ccc(! o- . No written work plans exist in the FLHFs visited Availability of human resource for planning activity. Lack of planning sl<ills and knolvledge. Lack of fr-rnds for planning sessi o ns " Errough budget t0 conduct planning sesslons to F LH\,Vs. T und u ru C DT I S u stai n ab il ity Ev al u atio n Page 53 E o o t!I =,c Strength Weakness Opportunity Threat Suggestions lmprovement c .9P(E o0 o s . CDTI activities with other programmes All stakeholders arc aware o{ CDTI projcct at this level Prescnce cornmunity bascd programs which willing i ntegrate of are to Dot-tor'S decision running proSram on thc To continue with integratiGn with other stakeholders at this lcvel, .9 ! (u T'(! oJ . Staff at this level initiate some activities . Health management team consider CDTI as theirs . ln one of the FLHFs visited staff wait for instruction from PC & DOTs Availability of committed staffattlHl-. Availability training leadership skills. of in Soine staffs are not confident with decision making,. Frequcnt change of Government and Political Icaders. Continues HSAM learning education should be e rn ph asized. .E) E E o{J bo .s C Csd . CDDs are planning and managing their CDTI work efficiently CDDs update the registers differently from distribution Presence of rornnritted CDDs . Norms and vnlues of the corrtnrunity. Gender imbalance d uring planning. {-'*mmunity sens itiza tion shciuld be em phasizcd(HSAlvl) in plannin5l CDDs should he taught on planning. Tunduru CDTI Sustainability Evaluation Page 54 E o o (! .9E' =q Strength Weakness Opportunity Threat Suggestions lmprovement .9 -c o C 3o oU .9 ! L oE(l, o -) . Community the leadership takes responsibility . Communitie s make all the decisions for CDTI implementatio n . People value Mectizan@ Availabiiity o{ rarnmittcd Lea d crs. Availability of corrr rr ittcc{ commu nity merrrbers on CDTI. Nornts .tfid valrrrs of thc conrmunily Fr eq ucnt changc sf GovernrnEnt arrd Political lcadcrs, 5ensitization 0n CDTI activitics to corrmunity lcaders and ntembers should be em ph asized. Tunduru CDTI Sustainability Evaluotion Page 55 SWOT Analysis Tunduru GDTI Project Tanzania Group B 6 (l, o (E .9E'E Strength Weakness Opportuni$ Threat Suggestions lmprovement (, o o o- c .otn .E o)o- =a o6 o)c Lo =co = lmportant records are readily available are of good quality Staff members only supervise staff at FLHF level and sometimes goes to community level Supervision is integrated Resources are efficiently used for supervision. As soon as problems are identified arc addressed. Team goes for supervision without justification. The Project Coordinat or being a Co-opted member of the CHMT. lntegrated supervisio n The new data entry format provided by APOC is difficult to print. APOC managemen t should revise the data sheet and design as user friendly one. Create targeted supervision. @E(U N (J o) = . The system used is effective and uncomplicated . ltis dependable and sustainable . There is sufficient supply of Mectizan -Mectizan@ arrived late in 2008 -Mectizan collection is outside the govt system. -Other drugs come through MSD to the ZonalMedical stores Existence of Zonal drug system, which receives drugs from MSD APOC funds are used for collection of drugs Mectizan should be collected using Govt system of drug procurement Tunduru CDTI Sustainability Evaluotion Page 56 E o o G .9E' Strength Weakness 0pportunity Threat Suggestions lmprovement = U) - oO O) .g .gg F . Training and supervision takes place in the project. . The leadership uses every opportunity to sensitize stakeholders on CDTI. . Training is not targeted . Team conducts kaining twice a year. . Team failed to differentiate between duration/content of training mentionedin frequency of traininq . Conduct targeted kaining at all levels E .9 ..2 ct o L ..!o C, .9 a/, .E oo- =CJ) od o,c o '- o -Reporting is within the govt system -As soon as problems are noticed staffs at this level take steps to resolve them . Supervision is not targeted . Resources of the District Health system are used to transmit reports . The district team does not empower the FLHF staff to handle the problems at their level in some Health zone - lntegrate CDTIinto other programs in the district -Empower FLHF staff to undertake fully their responsibilities @c C'N oo . Mectizan@ is sufficient in the divisions. . Mectizan is controlled within govt system. . The system for collection of Mectizan is uncomplicated and effective . The drug came in late this year - 2008 - Make drug available at Project level on time Tunduru CDTI Sustainability Evaluation Page 57 E o o Gl .9 !,s Strength Weakness Opportunity Threat Suggestions lmprovement =a - od o) .E .sg F . HSAM is planned and carried out in efficient manner . Training is conducted routinely every year . There is no objective need for training in most CASES . DOT's do not train FLHF staff under their jurisdiction alone . Training resources not efficiently managed . CDD' S incentive issue has not been successfully resolved lnvolve religious leaders, influential people ect. when conducting HSAM Lack incentives CDD's of to Sensitization to Leaders at village and sub village levels on the impodance of CDD's motivation both in cash and kind. lJ- -JlJ- LL C .o .t) .E oo- =U) od o, C, o =Co = . Reporting is within government system . Problems are promptly managed . ln FLHF reports are submitted using personal funds when they go for their salaries. . Staff in some FLHF supervise CDD' . Supervision is routine - lntegration into other health programs at that level -There is no planning and budgeting at this level -All of the above is done at higher level Staffs at this level should be empowered to implement fully CDTI Conduct targeted supervision related to needs @ c.(U N oo = . ln most cases is managed within the same system as other drugs. . Handling of Mectizan is efficient and uncomplicated Maintain the current situation Tunduru CDTI Sustainability Evaluation Page 58 E(, oG,o Es Strength Weakness 0pportunig Threat Suggestions lmprovement a - o6 o, .s .E(E F . Sensitization is carried out effectively . HSAM is effective . Training of CCDs is planned and conducted routinely. . No objective need for kaining. . CDD incentive issue remains a problem. lnvolve religious leaders, influential people ect. when conducting HSAM Lack incentives CDD's of to -Sensitization to Leaders at village and sub village levels on the importance of CDD's motivation both in cash and kind. -Conduct targeted kaining related to needs o)c Lo .=co . Report gets to the FLHF promptly . Communities conduct self m0n o) la) o) (t,(5N c'ESE -> oO o, .s .g(x, -oo . There is enough Mectizan every year in many communities . CDDs fetch Mectizan for their communities . No transport is arranged for CDDs to collect Mectizan@ Sensitize community leaders to provide kansport to CDDs 6 (D .E =E Eo() =a - . People are sensitized to take Mectizan@ . Community leaders encourage members to take Mectizan and this has reduced refusal rates -Maintain cunent situation -Sensitization to Leaders at village and sub village levels on the importance of CDD's motivation both in cash and kind. -Conduct targeted training related to needs T und u ru C D TI S usta in ab il ity Ev al u atio n Page 59 SWOT Analysis Tunduru CDTI Project Tanzaniar Group C E oJ L oa) GI t,ct{ Strength Weakness Oppoftunity Threat Suggestions Improvement to oL o. o(J c(o c iE . Cost of each activity is clearly spelt out . Project management is of for aware shortfall . Funds disbursed oncho control from the budget at this level are efficiently managed . Some members of the leadership are not clear about what is available for CDTI . Funding is mostly from external sources . No specific and realistic plan to bridge the shortfall . Council willingness to contribute more funds . Possibility to integrate with NTD Program Lack of funds from Council due to other competing commitments The Council should be sensitized to allocate more funds to CDTI oo(J L =ol'tod. llt 'tr oP(o E Lo -cPo ato o- U}Eg F . Vehicles and other equipment are readily available for CDTI work . Drivers use log books . No copy of previous travelling authorization for use of vehicle was seen . There is noplan for replacement of vehicle and other equipment . Maintenanc e costs are met with APOC & SSI funds . Integra tion with other Program . Council to allocate funds for regular maintenance . Delay on other program implementati on . Staff should staft using travel authorization . Council should allocate money for maintenance of vehicles Tunduru CDTI Sustainability Evaluation Page 60 E oJ L o ,L' TE .9E E!{ Strength Weakness Oppoftunity Threat Suggestions Improvement UI(l)(J L) ovl(l) d. c(o E)E . There is sufficient human resources . Staff members at this level are stable . Staff members are skilled enough to undertake their responsibility at this level . Staff members are highly committed (u(fl(o Lo oU . Geographical coverage is 100o/o for the past three years . Therapeutic coverage is above 650lo and is on the increase, ranging from 70.o/o in 2005 to 77.0o/o in 2007 High demand for Mectizan@ Delay on procurement of the drug NOCP should follow up MSD to ensure that drugs are made available in time ,L,U LPo a (u(,c(o c iE . The costs of each CDTI activity in theyear plan is clearly spelt out in a budget . The funds are mainly from APOC and SSI . Nothing is done about shortfalls . Possibility of using Community Health Fundsfor CDTI activities . Drop on cash crop production that may affect peoples income Tunduru CDTI Sustainability Evaluation Page 61 E oJ L o ,L) G .9t,Et{ Strength Weakness Oppoftunity Threat Suggestions Improvement L !r)oo) -cuPLr^l :,:odE Edo-o.(oU' 'EcolgH F= . The DOTs maintains the available transport and other equipment . Transpoft is used and managed in an integrated manner . The vehicles are not adequate . No plans for replacement of vehicles and equipment Integration with Eye Care Program Competing priorities of other pro9rams Multi-sectoral approach att o(J L =otn(ud c(I, E fI . Staff is stable . Staff members express satisfaction with their responsibilities . Staff is committed . There is no focal person in one of the health districts visited All divisions are allocated to the DOTs availlable Staff movement (upgrading, transfers etc). Project expansion to cover more villages causing more work load to staff Train more health staff on DOTs at division level o C7l(, L(l) o(J . Geographical coverage is 100o/o in the last three years . Therapeutic coverage is >650/o and is rising generally . Therapeutic coverage range from an average of 65.70/o in 2005 to 70.3o/o and 82.2o/o in 2006 and 2007 respectively among the Districts visited Delay on Mectizan arrival is a potential threat on therapeutic and geographical coverage Full intergration of Mectizan delivery in MSD logistic system c Tunduru CDTI Sustainability Evaluatio n Page 62 II IJ LL L o ,L) GI t,c!-{ . Strength . Weakness Oppoftunity Threat Suggestion (u(,, c(o CiI . No budget in at this level a Possibility of using Community Health Funds for CDTI activities Drop on cash crop production that may affect peoples income Leadership to be sensitized to allocate funds from CHF to FLHF EEbsts=rd besp o ato o- IJ,cg F . Transport is maintained . Transport is not adequate for running CDTI activities at this level . Cost of maintaining vehicles and equipment is not met with health centre funds . No plan for replacement Possibility of using Community Health Funds for CDTI activities including vehicle maintenance and replacement Drop on cash crop production that may affect peoples income Poor sensitization of community members tojoin and contribute to CHF Leadership and communities to be sensitized to contribute and allocate funds from CHF to FLHF tn(u(JL fo vl Q)d. c(o E3I . Staff at this level is stable . Staff is skilled Staff willing to do CDTI activities Staff movement (upgrading, marriage, transfers etc). Project expansion to cover more villages causing more work load to staff Tundu ru C D T I S u stainab ility Ev a I u a tio n Page 63 (u C,l(o Lo o(J . Geographical coverage is 100o/o for the past three years . Therapeutic coverage is high and rising . Average Therapeutic coverage range from 66.60lo in 2005 to 74.0o/o and 74.4o/o in 2006 and 2007 respectively Coverage is unstable in Mbesa Village Availability of skilled and stable human resource Delay in drug delivery from higher levels Unstable settlement of community members Continuous HSAM .E E E E oI olc 'o c(o c lJ- Nothing done support now is to CDDs CHF has been instituted Drop on cash crop production that may affect peoples income Poor sensitization of community members tojoin and contribute to CHF Continuou s sensitization of community members to contribute to CHF Communit y leaders to be sensitized to allocate some of CHF funds to CDTI Tunduru CDTI Sustainabiltty Evaluation Page 64 UI(u(J L) o at(u d. c(E E)I . There are two CDDs in many communities . The ratio of CDD to population is good . CDDs have been there since inception of the pro9ramme . CDDs indicated willingness to continue the job for the benefit of their communities even without pay Communities can decide to select as many CDDs as they desire Lack of incentive to CDDs by community members HSAM tO communities to include their roles in suppoting their CDDs G)ol(o Lo oU . Average coverage of the communities visited is 74.5o/o (i.e. >650lo) . Average coverage for the communities visited has been consistently >650/o in the last 3 years. The trend in coverage is not steady increase in some communities The demand for drugs is high as a result of the perceived benefits by community members Delays in drug delivery may interfere with community activities and lead to high absenteeism Drug procurement and delivery should take into consideration the time decided by communities for distribution Tunduru CDTI Sustainability Evaluation Page 65 |'.! No. Names Title Address Regional level L. Dr Malekela Regional Medical Officer Ruvuma Region 2. Salehe Pambaa Regional Secretary Administrative Ruvuma Region Project level 3. Ephraim Ole Nguyayi District Executive Director Tunduru District 4. Dr Alex Kazula District Medical Officer Tunduru District 5. Nurdin Maloya Project Coordinator- Tunduru CDTI Project Tunduru Project District Onchocerciasis Focal person (DOTs) 6. Godfrey Nvile DOT- Tunduru CDTI Project Namasakata Division 7 Mkulia Dastan DOT- Tunduru CDTI Project Nalasi Division FIHF, Village and Community level 8. Cesilia Fussi FLHF Staff- ln charge Namasakata FLHF 9 Amina Matumila FLHF Staff Namasakata FLHF 10 Ausi Ndago Village Chairperson Mbesa 1.1 Hassan Ngolowi Village Executive Secretary Mbesa 72 Mohamedi Ngolanga Sub-village Chairperson Luwawa 13 Katias Mkunde Sub-village CDD Chairperson/ Nairobi 74. Mussa Mzee Community Member Luwawa 15. Hema Mshana Community Member Nairobi 16. Mussa Mzee CDD-Male Luwawa 77 Mwanahawa Mhaji CDD- Female Luwawa III Names of people interviewed during the Evaluation of Tunduru CDTI Proiect I Tunduru CDTI Sustainability Evaluation Page 66 18 Omary Kadewele FLHF Staff- ln charge Mbesa Mission Hospital 79 Zuwena Mchenjeuka FLHF Staff Mbesa Mission Hospital 20 Yenje lsmail Village Chairperson Chiwawa 21. Adam Selemanl Village Executive Secretary Chiwawa 22. Swalehe Salehe Sub-village Chairperson Ujamaa Sub-village 23. Ally Athuman Sub-village Chairperson Songambele Sub-village 24. Jelas Ally Community member Ujamaa Sub-village 25. Mwajibu lsmail FLHF Staff- ln charge Chiwana Sub-village 26. Hamidu Kaziya FLHF Staff Chiwana Sub-village 27 Rashid Chiwanga CDD- Male Ujamaa Sub-village 28. Cheusi Mpota CDD- Female Ujamaa Sub-village 29 Fatu Thabit CDD Female Songambele Sub-village 30 Chipoka Musambola Village Chairperson Chemchem 31 Litami Kange Village leader Nandukutuku A& B 32 Sijawa Said Waziri CDD - Female Mikoroshoni Sub-Village 33. Sakina Musa CDD - Female Jangwani Sub-Village 34. lddi Mukwanda CDD - Male Jangwani Sub-Village 35. Musa Rajabu CDD- Male Mikoroshoni Sub-Village 36. Zuberi Kazembe Village Executive Secretary Nandukutuku "A" Sub-village 37 Matanda Bonamali CDD- Male/ Sub-village Chairperson Nandukutuku "A" Sub-village 38 Zuhura Buruhani CDD - Female Nandukutuku "A" Sub-village 39 Rashid Namtunga CDD- Male Nandukutu ku "B" Sub-village 40 Zainabu Hashim CDD - Female Nandukutuku "B" Sub-village 47. Hamad Bonomali Community member Nandukutuku "A" Sub-village 42. Bwanado Amlima Village Chairperson Kazamoyo 43. Ndemanga Saidi Village Executive Secretary Kazamoyo 44. Mohamedi Mchopa Sub-village Chairperson Mashariki Sub-village 45 Zabibu Hussein CDD - Female Mashariki Sub-village 46 Matembezi Anusa CDD - Male Mashariki Sub-village 47 Amina Matumla FLHF Staff Ligoma Health facility 48 Salumu Kahala Community Member Mashariki Sub-village 49. HajiSiani Sub-village Chairperson Umoja Sub-village 50. Said Amiri CDD-Male Umoja Sub-village a Tunduru CDTI Sustainability Evaluation Page 67 51. Zainabu Shaibu CDD-Female Umoja Sub-village 52. Mohamed Jalas Community Member Umoja Sub-village 53. Mbika Jafari Sub-village Chairperson Magharibi Sub-village 54. Mwanahawa Salumu CDD-Female Magharibi Sub -village 55. Daudi Husseni CDD- Male Magharibi Sub-village 56. Salumu Mohamedi Community Member Magharibi Sub-village 57. Mzee Wasia Village Chairperson Makande 58. Said Tikatika Village Executive Secretary Makande 59 Twalibu Binamu Sub-village Chairperson Mapuga 50. Kassimu Suedi CDD-Male Mapuga 61. Asha Mapunda CDD-Female Mapuga 62. Mwenye Makande Community Member Mapuga 63. Abdalah Halifa FLHF Staff Makande Community Member NGDO 64. Dr. Kabore lbrahim Country Representative (SSl) Dar Es Salaam 55. Mr. Nyangusi Programme Officer (SSl) Dar Es Salaam 55. Sixbert Mzee Mbaya Senior Managers (SSl) Programme Dar Es Salaam i Tunduru CDTI Sustainability Evaluation Page 68 ry ADDRESSES FOR EVALUATION TEAM MEMBERS a NAME ADDRESS Harrieth HAMISI P.O Box 7785, DAR ES SALAAM. TANZANIA Email : lvimoharrieth@vahoo.com Cell: +255 78438L52O Dr. Wade KABUKA Ruvuma Regional Hospital, P.O. Box 5, Songea. Ruvuma Region. TANZANIA e-mail : wka bu ka @va heg-cqm Fax: +255 25 2602048 Cell: + 255 754899947 Oscar KAITABA Deputy National Coordinator MoHSW P.O. Box 9083 DAR ES SALAAM TANZANIA FaxlPhone: +255 22 213 000 9 Cell: +255 754 889390 e-mail : ockaitaba@vahoo.com William KTSOKA Research Scientist, National lnstitute for Medical Research, P O Box 9553 Dar es Salaam Tanzania Tel +255 222L2L400 Fax +255 22 272L36O Mob 255 755 096 080 e-mail : wkisoka@yahoo.com Sebastian MHAGAMA District Onchocerciasis Coordinator P.O Box 42, Mbinga Ruvuma Region Tel +255 2640762 Cell + 255 756 236020 Fax + 255 2640L81 Email : s _mhagama@yahoo.com Dr. Joseph C. Okeibunor Depa rtment of Sociology/Anthropology University of Nigeria, Nsukka Enugu State Nigeria. t T und u ru CDT I S u stai n ab il i ty E v al uatio n Page 69 Cell: +2348043180351 e-mail : iokeibunor@vahoo.com t '4 I , Tunduru CDTI Sustainability Evaluation Page 70