Report of the Tanga sustainability evaluation: March 8-11, 2003

REPORT OF THE TANGA REGION SUSTAI NABI LITY EVALUATION MARCH 8 -21 ,2003 TEAM MEMBERS: 1. Dr. J.Y. Jiya 2. Mrs Chinyere U. Maduka 3. Mr. James Mugisha 4 Ms. Maria Nakabiito 5 Mr. Karoli D. Mallry 6 Mr. George Pallangyo Team Leader Member aa a) a) F ft u r l i ? :r rutt, lSLij H,n4('1 !ndex Page Abbreviation/Acronyms and Acknowledgement Introduction and Methodology Findings and Recommendations l. RegionaL/Project level 2. District level 3. First line health facility level 4. Community level 5. Overall Grading/Sustainability Status of the Project Feedback MeetinglPlanning Workshop 2 8 3-7 8-l I t2-14 t5-17 r8-20 2l 2t-22 22-63 64 65 65-66 67 68-76 Appendix l. 'o 2. aaa J. ,, 4. ') 5. "6. Detailed Findings Schedule of Activities List of document sighted Persons interviewed Agenda for Planning workshop/Feedback meeting Minutes of feedback meeting/planning workshop 2 List of A bbreviations/AcronymsAPOC African Program for Onchocerciasis ControlCDDs Community Directed DistributorsCDTI Community Directed Treatment with lvermectinDDH Designated District HospitalDMO District Medical OfficerDOTS District Onchocerciasis TeamDHMT District Health Management TeamHKI Helen Keller lnternationalHSAM Health education, Sensitisation, Advocacy and MobilisationlEC lnformation Education and CommunicationNGDOs Non Governmenta! Development OrganisationsNIMR National lnstitute for Medica! ResearchNOCP National Onchocerciasis Control ProgramNOTF National Onchocerciasis Task ForceMOH Ministry of HealthR/PC Regional/Project coordinatorRMO Regional Medica! OfficerROTF Regional Onchocerciasis Task ForceRHMT Regional Health Management Team.RHW Rural Health Worker \ /l-lO World Health OrganisationFLHF First Line Health Facility Acknowledgement We wish to express our thanks and gratitude to the following people who were immensely helpful and contributed positively to the success of the exercise. Mr. Charles Franzen of IMA Dares-Salaam who picked us up at the airport on arrival. Dr. Amri and Miss Eva Muro both of WHO Country office Tanzaniawho assisted with all administrative problems. Dr. Maggid and Mrs Harrieth Hamisi the Project Coordinator for Tanga Region and the HKI Representative respectively. These two ladies were extremely helpful to us and made our stay in Tanga memorable. To the drivers, Kulwa lssa and George who took us on those very perilous roads up and down the mountain of Muheza, Lushoto and Korogwe. Finally we thank the Almighty God who granted us journey mercies and good health, before, during and after the mission. 3 Executive Summary The Tanga Region Community Directed Treatment with Ivermectin (CDTI) was launched in the year 2000. Prior to this mass distribution of mectizan in Hyper and meso endemic communities of the region, treatment had been given in Designated District Hospitals and National Institute for Medical Research to Patients Suffering from Onchocerciasis. Rapid assessment Surveys carried out in 1999 indicated the prevalence of Onchocerciasis in three districts of Loshoto Korogwe and Muheza which led to the introduction of CDTI Strategy for control in year 2000. Helen Keller international is the NGDO supporting this project. Onchocerciasis in the three districts is hyper and meso endemic in only I 19 Communities. These are the Communities where CDTI is currently being implemented. In the first year of implementation, I 18 Communities were covered while all the I 19 Communities received mectizan in the 2"d year. Currently, the project is in the 5d year of implementing CDTI, coverage in all the Communities was impressive in the first year with a 74.9% Coverage rate; this dropped to 69.3%o in the td year. The drop was attributed to a number of refusals which came about because of reaction to the drugs in the first year of mass distribution. The district Onchocerciasis team members and the first line health facility staffhave gone out to reeducate the Communities and to further give health education to enable those at risk to take the medicine in the third year, which treatment was in progress at the time of evaluation. The regional government does not have vote for Onchocerciasis control. This is because of the peculiar administrative system in the Country where project implementation is at the District level. Funds are voted for and released to project implementers at the district level. All the 3 districts where Onchocerciasis is endemic in the Tanga Region are mountainous areas. Accessibility to the Communities can only be during the dry session; the roads are winding, slopping and dangerous. It is to the credit of the District control team members, the first line health facility staff and all the community distributors that mectizan reaches all those who need the drug and require to take it annually. The Community leaders, distributors and first lindtealth facility personnel all show total Commitment to the programme. The team is convinced that the project is working steady towards sustainability moreso that the project areas and the communities at risk is not large. We are positive that giving the commitment shown, and the active participation of persons at the district, FLHF and community levels, the project will be sustained after APOC withdraws support. 4 Tanga- Introduction 1.1- Background information A rapid Epidemiological Assessment (REA) for onchocerciasis was carried out in 4 districts of Tanga Region in February 2000. The 4 districts were: Lushoto, Korogwe, Muheza, and Handeni Out of these, only three districts were found to be onchocerciasis endemic with prevalence rates of 48o/o and above. The onchocerciasis endemic districts are: Lushoto Korogwe and Muheza districts, while Handeni had prevalence rates of 09.9%. Prior to this exercise, treatment for onchocerciasis was given in some clinics in Muheza district, which has thehighest prevalence rate in the 3 districts. A Rapid Assessment Method (RAM) was carried out and an epidemiological map for the disease developed. This led to the invitation of APOC to support control activities in these 3 districts in the year 2000 whenthe project first presented proposal to APOC to assist in the implementation of the Community Directed Treatment with lvermectin (CDT!). The Tanga regiona! government in partnership with the \ ,I-|O/APOC and Helen Keller lnternationa! (HKl) taunched the Tanga Focus CDTI project officially on 29 April 2000. No communities in Korogwe and Lushoto districts had received Mectizan tablets before the launch of the mass treatment exercise. Patients who manifested with signs of onchocerciasis in: Lushoto and Korogrue districts were referred to Designated District Hospitals (DDH) and the National lnstitute for Medical Research (NIMR), both in Muheza district for treatment. Meanwhile, patients diagnosed with onchocerciasis in Muheza district had received Mectizan tabbts as curative therapy since the 1990s. Mobile teams through DDH gave treatment to these cases and in some other seventeen identified endemic villages, which have now been included in the CDTI strategy. 1.2 Methodology Taking into consideration the guidelines for evaluation pf projects for sustainability, and especially the primary criterion for sampling, the team together with the Deputy National Coordinator, the Regional/ Project coordinator and the NGDO representative of Helen Keller lnternational took into consideration 3 critical factors before selecting evaluation sites. These include: Coverage (Geographic and therapeutic,), the endemicity levels based on REMO results available and geographical spread. The Tanga Region CDT! Project is composed of 3 districts. !t was therefore decided based on the factors listed above and the geographical terrain of the communities, to evaluate 2 representative sample districts of Korogwe and Muheza 5 ln each of the districts selected 4 health facilities were tobe visited 2 in high coverage and 2 in low coverage areas if available; For each health facility 2 villages were selected .the selection criteria followed the sample principle used for the health facilities. ln the course of our work in Tanga Region, we eceived instruction from APOC that the three districts be covered in the sampling. The sub-team one was dispatched to Lushoto district immediately to carry out the exercise in two FLHF with two villages each in the district. We also took the privilege of interviewing the officers from Lushoto who were attending the FeedbacU Planning meeting at Tanga. 1 District Muheza 2. Korogwe FLHF Amani(HC) Zirai(LC) Kwatango(HC) Misozwe(LC) Vugiri (HC) Bungu(LC) Mnyuzi(HC) Mbelei Villages Mbomole Mashewa Zerai Kwelumbizi Kizerui Kwatango Misozwe Mkwajuni Kwemhanyi Vugiri Kwenshai Bungu Lusanga Kwalei Mbelei 3 Lushoto Kwekitui Tota! 3 16 1.3: Objectives . The primary assignment of this exercise was to evaluate how sustainable the Tanga Region CDTI project is. o Design of the methodology was to have a cross-sectiona! and descriptive information from the sampled sites. o The population of Tanga region project includes the project coordinators and members of the HMT, the District Onchocerciasis Team (DOT) together with the DMTs, the NGDO partners, the FLHF staff, the project villages, their chairmen and CDDs. 9 6 lnstrument o Questionnaire was developed based on the instrument . The instrument assesses the sustainability at 4 levels of operation. o The instrument guided the evaluator to collect the relevant information about each indicator. . Source of information was vebal from those interviewed, supplemented by documentary evidence where relevant and available. Analysis . Based on the information obtained both verbal and documentary each indicator is graded on a scale of 04 in terms of contribution to sustainability. . A summary finding for each group of indicators is given at the end of each group. o Average sustainability score for each group of indicators is calculated for each level. . A narrative description of the problem areas is given. o Recommendations are given with time frame and priority level. 1.4 Team Composition: The team was composed of the following members 1. Dr. Jonathan Y. Jiya, Director Disease Control, National Coordinator NOCP Nigeria -Team Leader 2. Mrs. Chinyere Maduka, Project Administrator Enugu/ Anamba/ Ebonyi, 8B First Avenue lndependence Lay Out Enugu, Nigeria 3. Mr. James Mugisha, Socia! Scientist, Kampala, Uganda 4. Ms. Maria Nakabiito, Social Scientist, Population Secretariat, Kampala, Uganda 5. Mr. Malley Karoli. D. Research Scientist, National lnstitute for Medical Research Tukuyu Research Station P.O. Box 538, Tukuyu, Tanzania 6. Mr. George Pallangyo, District Onchocerciasis Coordinator, Box 745 Songea District, Tanzania The team was divided into two sub teams to evaluate activities at the various levels in the 2 districts. The team met with the project officers and the deputy national coordinator, to select evaluation sites and to discuss instruments to be used in order to harmonize their approach. The two teams departed for Korogwe and Muheza each wth the team leader engaged the project coordinator and the NGDO partner in the evaluation of regional activities using instrument No.1 The team leader later joined the team assigned to Muheza district to continue the evaluation of the remaining 3 levels in that district. 7 1.5 Planning / feedback workshops A planning and feedback workshop was conducted for all the relevant officers at the Regional and the 3 District levels. During these workshops, the officers were briefed on the field findings and issues raised were discussed. Efforts were made to develop sustainability plans for the remaining part of year 2003 and a new plan for year 2004. 1.6 Limitations Quite a number of documents requested for at the district and FLHF were not available. The team met with the Acting RMO, but was unable to meet with the Regina! Commissioner who had just been appointed and was on an orientation tour of the region Selected villages did not receive advance notice of the evaluators visit. ln many of the villages, the community leaders (Village Chairman) and most of the villagers had gone to their farms.ln al! villages, at least one CDD was found and interviewed. The team could not arrive at the evaluation site of Tanga on Saturday because it had to sort out some logistics ard administrative problems with the \ /l-lO country office in Dar-es-Salaam on Monday morning. a a a 8 r] Figurc l: Grading susteinability indicators at Regional Levcl -c -'"|*';*".*"**."-"u)""'.'"'.".od "/ {,t$" +"+ Indicators 4.5 4 3.5 s3 o E 2.so 9, ')e-o < l.) I 0.5 0 MAIN FINDINGS 1z REGIONAUPROJECT LEVEL (a) Coverage Geographical coverage was calculated to be at the maximum level because the villages to be covered have all been fully identified. ln the first year of treatment 118 villages were covered while 119 villages received treatment in the second year. The therapeutic coverage rate was calculated to be 77.4o/o and 69.3% in 2001 and 2OO2 respctively. We noticed a drop in therapeutic coverage rate and this was due to refusals by persons who got reactions in the first year of treatment. The other reasons given were the increase total number of villages covered and the populations thereof. With intensified health education, sensitisation, advocacy and mobilisation, the ongoing third year mass treatment should record higher coverage rates. (a) Planning At the regional level, there is no written plan for contro! of onchocerciasis within the government structure/system. This only exists at the district level. However, the project plan was done in a participatory manner where all key partners were involved in its development. 9 (b) Mectizan @ Mectizan is received within the governmentestablished system, that is; the Medical Stores Department (MSD). The ordering and stock control documents are available. The system is effective, simple (uncomplicated), and efficient. lt uses government resources in the processing of application and receipt of mectizan. However, the last shipment had some problems due to lack of adequate information to the government authorities responsible for the control of tax of the imported goods. As a result, the tablets were held at the port of entry for some time This resulted into the delay of the drugs reaching the end users. There were no delays in application for and receipt of the drug during the previous two years when CDTI was introduced. (c) Training and Mobilization Adequate training manuals were available at the regional /project leve!. The regiona! project office does training of District Onchocerciasis Control Team (DOT) members routinely. We noticed that the regional office does training of trainers. The trained officers thereafter train the health facility staff. However, occasionally, regiona! officers assist in the training of the FLHF staff and community directed distributors (CDD). Health education, sensitisation, advocacy and mobilisation at the district level are effectively conducted. (d) Financing There is no evidence of budget or allocation of government fund to onchocerciasis contro! activities at this level although there is an APOC approved budget for the project. !t was noted that actual budget provision were made at the district level. We were also informed on the bosket funding, which operates at the district level which supports various health activities. This in effect means that funds required for field activities can only be obtained at the district level. Most of the expenditure incurred at the project level is from APOC Trust Fund. Government pays salaries for onchocerciasis personnel at the regional level and provides accommodation and utilities free to the onchocerciasis office. (e) Transport and Materials Transport and materials were made available by AFCC. The motor vehicles and capital equipment provided are in good functional state. !t was also noted that one of the laptops has broken down while one portable printer has been stolen. The NGDO partner, Hellen Keller lnternational (HK!), has provided onepiece of desktop computer system with all the peripheral accessories. The vehicles and other equipment are routinely maintained using APOC funds. There is however, no replacement plans for vehicles and capital equipment.(0 Supervision l0 Routine supervision is carried out by three persons; namely. the project coordinator, her deputy and occasionally the regiona! medical officer. lt was noted that targeted supervision visits to handle problems and success were not being done. There is also no Regional Onchoceciasis Task Force (ROTF), which impeded empowerment of the district level teams to undertake targeted supervision and problem solving. (g) Human resources The regional office has a policy team, which is called Regiona! Health Management Team (RHMT). This bdy consists of the Regional Medical Officer (RMO) as the Chairperson and other heads of the public health divisions in the region as members. The Onchecerciasis Project office is manned by two full time technical staff that are civil servants and are wellskilled in CDTI. There is good cooperation amongst the programme. (h) Leadership A project coordinator is in place together with the project Accountant. These two officers are the technical heads at the Regional Level. The Regional Medical Officer (RMO) together with the other members of the Regional Health Management Team (RHMT) perform the duties of an advisory body to the project. Membership of the RHMT gives a very good opportunity for programme integration both at the regional and district leve!. The political leaders at the regiona! level are involved in policy matters, which seem to have positive impact on the public health issues. Active involvement in CDTI activities by political leaders is not noticed. il RECOMMENDATIONS Coverage . We recommend that the project should endeavour to have the total number of villages covered . We recommend that health, education mobilization, and sensitisation be intensified Priority: High lndicator of success . Enable accurate geographical coverage . lmproved therapeutic coverage. Action to be taken by Project coordinator by Dec.2003 Planning . We recommend that a plan for onchocerciasis control should be included in the regional health plan, this will facilitate the govt. contribution Priority: High lndicator for success . Existence of onchocerciasis in the next year's government plan-2004 Action by:- Proiect Coordinator Mectizan . Government establishments should link up with one another to ensure that mectizan delivery in not delayed due to government taxation. Priority: High Action to be taken by: Project Coordinator and National Coordinator. lndicator for success: . Timely delivery of mectizan . Mectizan removed from taxable items Transport and materials . Adequate health education sensitisation advocacy and mobilisation materials be produced urqently. Priority: High lndictor of success: HSAM materials. Action to be taken by: Project coordinator by July 2003 Supervision Targeted and problem solving supervision be initiated and carried out. We wish to discourage routine CDD and FLHF staff level supervision by the regionalteam Priority: Medium lndicator for success: . Empowerment of district level personnel to monitor and supervise the levels below them respectively Action to be taken by: Regional health staff immediately Human resources We recommend the active participation of members of the RHMT in onchocerciasis contro! The establishment of a project onchocerciasis task force to assist in information dissemination to all lower levels Priority: High lndicator of success . for success: Establishment of an regional onchocerciasis task force (ROrF). Action to be taken by: regional medical officer by Sept 2003 Leadership Technical personnel at the project office be increased so that we have a project coordinator and at least 2 team members to assist her in the execution of her duties Priority: Medium lndicator for Success: . Appointment of support staff to the project office. Action to be taken by: Regional medical officer by Dec 2003 t2 32.5 s o oo o EI)G o I .5 I 0.5 s* ndr"rtr.C f* nf *t- su o'€ C .-, 'od.--.,t.'^t-c --t c'd \- s' .s !i Indicators Ill Ll 2: DISTRICT LEVEL Figure 2: Grading of sustainability indlcetors rt District levcl (a) Planning Year plans are integrated in the overall district plans, drawn in a participatory manner but do not take into account some of the community requirements like time and mode of distribution. The plans make provision and take into account key activities like mectizan supply, targeted training, and monitoring and supervision and health education. (b) Leadership. The leadership at the district level is the full responsibility of the district health management team. They direct key CDTI activities ard DOTs are directly responsible for CDTI field activities. (c) Monitoring and supervision. The reporting process takes place within the government system and is cost effective. However staff at this level go beyond their scope and supervise CDDs at the community !eve!. lt would be beneficial if intergrated supervision is 13 conducted with immunisation, malaria. APOC funds are the main source of funding for monitoring and supervision. (d) Mectizan Procurement. Mectizan procurement runs through the existing government system. The ordering and stock control systems are sound. The supply was always on time except for the current year. The DOTs take the mectizan to the health facilities once the drugs are received. (e) Training The district health personneltrain staffs at the FLHF. Training was done at the district level. Taking into consideration the nature of the terrain and facilities availabb the approach is efficient and cost effective. lt was also noted that district health personnel go as far as trahing village level CDDs which is not cost effective. The FLHF officers have been well trained and they organise a central training for CDDs. There is no integrated training with other disease control programs. Annual retraining of CDDs is done but not targeted to address deficiencies. Advocacy, sensitisation and mobilisation are carried out prior to census and distribution. (f) Financial. There was evidence of budgetary allocation for CDTIs at the district level for the year 2003.There was evidence of relese of Tanzania Shillings 100.000 in 2001 in Muheza district and 699.984 was for motorcycle repairs in 2003, this we believe was from APOC Trust Fund. Tanzania shilling 1.700.000 provision has been made for training in the bosket fund for CDTI in Muheza for the year 2003. ln Lushoto and Korogwe, similar budgetary allocations were made in the 2003 budget provisions. The establishment of a bosket fund is a good source of funding which could be used after APOC withdraws. There was evidence of expenditures atthe district level. (g) Transport and Materials. Each district has 2 motorcycles provided by APOC. There was no health education materials seen. The vehicles available have regular maintenance. Wren vehicles break down, staffs seek alternative routes of tansport mostly public means. Available transport is used for CDTI activities at the health facility. The motorcycles are used for other public health activities. They have neither logbooks nor official authorisation books. Staffs at this level are very knowledgeable about CDTI activities. The district onchocerciasis team members have been trained in all aspects. They catry out activities such as planning, training, health education and mobilisation; monitoring and supervision and mectizan inventory. They ae committed to their jobs and stable. Their salaries are paid by government regularly. 14 (h) Goverage There is evidence that all communities identified and recommended for treatment are being treated. Over 75o/o ol persons in the identified villages who areeligible for treatment received mectizan in 2001. There was a slight drop in 2OO2 due to refusals emanating from minor side effects in the first year of treatment. With sensitisation and mobilisation which were carried out before the current year (2003) treatment it is hoped that therapeutic coverage will exceed the 2001 levels. Recommendations Planning . FLHF be empowered to assist and support villages in planning for the time and mode of distribution. Priority: High lndicator for success: . Communities are empowered to determine the time and mode of distribution. Action:- to be taken bv: DOT Monitoring an s supervision . DOT should empower the FLHF staff for the supervision of CDDs Priority: HIGH lndicator for success: . DOT do not supervise at CDD levelwith immediate effect Mectizan procurement . FLHF should collect their mectizan from the district level Priority: Medium Action to be taken by: District coordinator and FLHF staff by next treatment season. Training . Training of FLHF staff should take place at the clinics . An integrated training with other disease control programs . lmplement policy of targeted training for problem solvinq Priority: High lndicator of success: . Empowerment of FLHF Staff and the community to ensure better coverage Action By: DOT Financial . The District health system take part of the responsibility for maintenance if the motorcycles for CDT! activities . Financial expenditure profile from Govt and APOC funds be recorded for examination. Priority: High lndicator of success: . Ensure other source of funding and sustainability of programme. . New sources of funding provided. Action by: DMO and DOT in the next 6 months. Coverage . We recommend that the district level empowers the FLHF to carry out sensitisation and mobilisation at the village levelto cover minor reactions in order to improve therapeutic coveraqe Priority High !ndicator for success: . lmproved coverage rate for the next treatment season Action by: FLHF and DOTs immediately 15 43.5 J o 2.5 oo6) o- ot) o I d. rd *d -o "$"td ^---oU I "9{rts -""-- +$ Indicators 3: FIRST LINE HEALTH FACILITY LEVEL Figure 3: Grading ofsustainability indicetors et First Linc Heelth Facility Lcvcl (a) Planning The FLHF have no plans and depend on the district plans (b) Leadership The clinical officer in charge heads the FLHF but there is an Srnember health facility committee with chairman, and the secretary. The FLHF officer in charge is the secretary. The chairman of the committee is the village chairman. The politica! leadership at this level actively participate in CDTI. (c) Monitoring and Supervision Reporting system at this leve! is within the government system. Reports are submitted with census figures and treatment records to the FLHF. The staffs at the FLHF do supervise CDDs during census and at the time of drug distribution. t6 The FLHF staff are clinical officers who deal with a!! disease control programs in the area of their srpervision. They are therefore responsible for an integrated approach to problem solving at village !eve!. They manage problems like refusals and absenteeism. (d) Mectizan Procurement The FLHF staff order for mectizan from DOTs using census figures. Ordc forms were available and adequate mectizan tablets were received on time from the district office. There were no shortages reported. However, communities were not making decisions on the time of distribution. Mectizan tablets are delivered to the FLHF by the district while the communities pick up their mectizan allocation from the FLHF. (e) Mobilisation, training and health education Training of CDDs is carried out by the FLHF regularly but some times the DOTS participate in the training. However, the training is not prioritised for specific needs. The time used for training is too long and the sites used for training are not cost effective. FLHF staffs catry out sensitisation and mobilisation at the community level to solve problems of refusal. (f) Financial There is no evidence of budgeting and expenditures at this level. No transport or educational materials. (g) Human Resources. Each of the FLHF visited had adequate staff manning the unit. They were very knowledgeable and skilful to undertake CDTI except in areas of planning. The staff is very committed but need extra motivation. (h) Coverage There is full coverage of all the communities in this area. Therapeutic coverage has been good except for some refusals and some people who have left the community. The therapeutic coverage for 2001 and 2OO2 treatment cycles were 74.7o/o and 69.% respectively. \Mere as the geographic coverage for 2001and 2OO2 treatment cycles are 80.02o/o, 8O.O9o/o and expected to reach its ultimate treatment in 2003. t7 Recommendations Planning . We recommend that FLHF be empowered to plan for their CDTI activities and encourage communities to plan for time and mode of distribution PRIORITY - High lndicator of success . Drug delivery is at the time requested by the community. ACTOR - FLHF staff Monitoring and Supervision . Recommend an integrated approach in monitoring and supervision of CDD at village level PRIORITY - Medium lndicator of success: . Reduce number of visits by FLHF Staff to the communities Actor- FLHF Staff Time:- lmmediately Leadership . We recommend that village chairmen take full charge of mectizan .distribution.(CDTI) in their communities, . They should establish a village fund to support CDT! activities PRIORITY:- High lndicator of success: . CDD are motivated ACTOR: Vlllage chairmen are empowered in the next three months Mectizan Procurement & Distribution . We recommend that mectizan tablets be left with the village chairmen for at least two months after distribution period to resolve the problem of absenteeism and refusals. PRIORITY:- High lndicator of success . lmproved treatment coverage rate ACTOR:- FLHF Staff Mobilization, Training, Health education and Sensitisation . We recommend that, CDDs be trained at the community level. PRIORITY:- High lndicator of success: . lmproved health education level at the village level. Actor:- FLHF Staff Human resources . We recommend that the staff at the FLHF be motivated by awarding certificate of excellence to the those who perform creditably. PRIORITY:- High Indicator for success: . lmproved motivation and willingness to continue with CDT! work. ACTOR:- DOTs Time:- within this year, by the end of 2003. Coverage . We recommend that FLHF staff should monitor closely distribution activities by CDDs and to also ensure that census is accurately carried out. PRIORITY:- High lndicator of success: . lmproved therapeutic and geographical coverage . Accurate census figures. Actor:- FLHF Staff l8 4: Gommunity Level Figure 4: Grading susteinability indicetors at community level 0.5 0 ) Planning At the community leve! there is a village health committee with 58 members who are mainly village members. The committees participate in problem solving, supervision, monitoring, stock control and address problems such as absentees and refusals. They are also involved in motivation of CDDs like cooking and exemptions from community service. However, the communities are not involved in decision-making regarding distribution period. (b) Leadership. The village chairmen are fully involved in mectizan distribution. !n some of the communities the deputy chair was found to be a CDD. The selection of CDDs is done by the community .The people are appreciative of the mectizan treatment and are willing to continue taking the drug as required. 4.5 4 3.5 3 S B z.s 8 oB2o 1.5 -""- .,rup" **""ooo'-n'"^o*.o*o*..c' o.C "/ -.oc .; rndicators (a t9 (c) Monitoring and Supervision. The CDDs are supervised by the FLHF staff during census and distribution . No provision of transport has been made for CDDs at community level or the FLHF staff. This makes movement at this level a difficult task for these persons. (d) Mectizan Supply Mectizan tablets are collected from the FLHF immediately information is received about its arrival. Al! eligible persons are treated through the house-to-house method. Calculating from the total population multiplied by 2.2 is used to calculate the number of tablets required. No transport problems were noticed because the homesteads are close to each other and where it is far the CDDs are loaned bicycles. After distribution some tablets are reserved for those who are absent, sick or pregnant. However, the time given for keeping the mectizan at community level is too short (always less than a month) (e) Health Education Activities. Community directed distributors request the assistance of FLHF in area of community health education and mobilisation. This becomes important in handling community refusals. lt is also necessary to have FLHF personnel and in some cases the DOT to assist in community advocacy and acceptance of community ownership. (0 Financing There were variatlons in motivating CDDs at community level. $me communities provide height measuring sticks, loaned them bicycles for drug collection, pay compensation in cash, about TShs. 500/=, exemption from community service, cooking during distribution etc (g) Human Resources. There were a minimum of 2 CDDs per subvillage and depending on the number of sub-villages the numbers ranged between 1O2O CDDs per village. The ratio is around 1:100. The CDDs were found to be skilled at their work knowing all the criteria. They knew how to manage side effects, and ould also take care of attrition rates. They however had deficiencies in record keeping. The CDDs are committed and willing to continue and only very few have opted out. There are a significant number of female CDDs in each sub village. (h) Goverage The geographic coverage was full and all households treated. Therapeutic coverage rate is good although there were some refusals in the second year of 20 Planning , We recommend that the village health committee members should participate in problem solving and should determine the time and mode of distribution with consultation with the community members. . We also recommend that a community self monitoring (CSM) be implemented with the assistance of the National office. PRIORITY:- High lndicators of success . Distribution of mectizan at the time requested by the community. Actor:- Mllage chairmen and village health committee. Monitoring and supervision ' We recommend that, community self-monitoring be established, stakeholders meeting be also instituted. PRIORITY:_ High !ndicators of success . lmplementation of community self monitoring and stakeholders meeting Action :- Within six months Mectizan . We recommend that, mectizan tablets be left at community levelfor a period of two months after distribution in order to take care of allthose who were absent PRIORITY:_ High !ndicator of success . Mectizan is received at the appropriate time and CDDs motivated and committed to do their work. Action:- By village chairmen Coverage . We recommend aggressive health education, sensitisation and mobilization at the community level. To take care of those who refuse to take mectizan. PRIORITY:- High lndicator of success . lncrease number of persons taking mectizan annually. Action:- FLHF Staff, Mllage chairmen and CDD treatment. The third year treatment is currently ongoing and more people are taking tablets. Recommendations 2t Overall grading on the Tanga CDTI Project Flgure 5: Overall Gradlng of sustalnablllty of the Tanga Reglon CDTI roject 3.5 3 2.5 2 \i oo ooo o EDg o osrd "-j ""--dt t' -otra Indicaors The overall assessment of the Tanga CDTI Project is good. 'lhe team believes that the project is working satisfactorily towards sustainability. lt is hoped that once the recommendations and other issues raised are addressed the project wi!! be sustainable. The evaluators noticed that the community and FLHF levels are stronger than other levels. Based on the information received at the debriefing meeting with the Director of Preventive Services of the Ministry of Health; the United Republic of Tanzania, sustainability can be enhanced once the issue of excluding ivermectin from the taxable items is resolved. Feed back meeting/Plann ing Workshop The meeting and workshop was held on Monday the tf to Tuesday 18h March 2003 at the Bandari House Tanga. In attendance were members of the evaluation team, the National Coordinator for Tanzania NOTF Dr. Simon Katenga who came in from Morogoro. The Tanga Region Acting Regiona! Medical Officer, the Tanga Region ..'"- I TI 22 1 Project Coordinator Dr. Maggid together with the representative of the NGDO partner supporting the project (HK!) Mrs Harrieth Hamisi. Mr. Charles Franzer of IMA and the NGDO Chairman came in from Dares-Salaam together with Oscar Kaitaba who is the assistant to the Nationa! Coordinator. The three districts of Muheza, Lushoto and Korogwe where CDTI is in place were represented by their Council Chairmen, Planning officers, District Medical Officers and a District Onchocerciasis team member. The report of the feedback meeting and agenda are attached as an Appendiix 5 Following is summary of the workshop: A general introduction of participants was made The team leader made a presentation of 'what is Sustainability' Another member Mr. James Mugeshi made a presentation on the nine indicators used to evaluate project sustainability. Mrs C. Maduka discussed the methodology and approach used to select the evaluation sites though this had to be amended since all the three districts were visited. She also gave a brief on our limitations. Team members thereafter made presentations on the findings at the project, district, first line health facility and community levels. General discussions were made on the findings and explanations given for some of the findings. The meeting went into group discussions, each district formed a group and supervised by two team members each - three groups of indicators were assigned to each group. A report was presented by each group and fully discussed, questions were asked and clarification sought. Problem areas were identified and solution to each problem proferred Another group work for planning was undertaken, each district forming a group. The regional project office formed an independent group. Each group had the task of preparing a sustainability plan for the remaining part of year 2003 and a new plan for year 2004. A group presentation was given after which participants made their observation and comments. Outcome The Tanga Regiona! CDTI prepared a sustainability plan together with the three districts of Lushoto, Muheza and Korogwe, we are hopefu! that the project will implement the plans made. lt was agreed that the plan wil! need to be further revised as implementation gets underway. There may be need to change some of the activities and timings proposed in the plan for effectiveness and efficiency. There is the need for the projectoffice and the districts to further examine the recommendations put forward by the team and seek for ways to incorporate these into the future plans. 23 lnstrument 1: NATIONAU STATE LEVEL lndicators of activities and processes: planning 1.1 Check whether there is a year plan containing details of all activities needed for effective onchocerciasis control activity at this level. This indicator assesses whetherthe project isfunctioning effectively, and whether management accepts ownership of the programme - both good for sustainability. 1.2 Check whether all partners (government, UN agencies, NGDOs) are meaningfully involved in the overall planning process. This indicator assesses whether the programme is functioning effectively and efficiently - if each paftner is clear about its role, this is good for sustainability Characteristics of the indicator Findings Onchocerciasis control should be integrated into the overall written year plan of the health service at this level. Note that this plan is usually not very detailed. Describe the present situation: The region presently has a written plan for oncho control. There's an Action Plan forthe third year, which is laid out in a quaftely format and stictly followed. There's presence of minutes of planning meetings. ls this indicator for sustainability being achieved?t Not aoplicable Fully Highly Slightly Not at allIModerately Characteristics of the indicator Findings This more detailed plan should make provision for all key elements of onchocerciasis control: Mectizan supply; targeted trai ni ng ; targeted HSAM ; targeted monitori ng/ su pervision. The plan varies from year to year, showing that it is targeted to the specific needs of each year. There's a yealy plan with details of activities for Onchocerciasi s control. The plan contains defai,irs of activities such as mectizan supply, training, HSAM, monitoing and superuision, mectizan di stibution, data collection and report witing. I !s this indicator for sustainability being achieved? Fully Moderately Slightly Not at all Not applicable 24 Hi ghlv Characteristics of the indicator Findings All partners should contribute to the routine planning of a project. Partners should be clear about their own roles, and those of the other partners. All paftners who contibuted at the planning meeting prepared the plans and each partner is clear about their roles. ls this indicator for sustainability being achieved?I Not at all Not applicable ModeratelyFully SlightlyHighly 1.3 Check whether specific planning for sustainability has taken place, for the period after APOC funding is withdrawn. Ihis assesses whether the programme is functioning effectively, and that management has begun to take ownership of it and can mobilise the resources it needs. Characteristics of the indicator Sources of information Staff members have made plans for this period, which will enhance programme sustainability. This planning should include: identifying resource gaps; strategies to cut expenditure; and strategies to find dependable sources. There should be written evidence that such planning has taken place. There should be evidence that the plans are being successfully implemented. I ! There's a recent Health SecforReform, which requires the regions to superuise distict health activities. This reform also includes the setting up of a common pool of funds where programs can access such funds for the implementation of the activities. There's hope that this facility can be used when APOC leaves. Which steps are being taken to improve the situation? Working closely with HKI to supplement on the distict resources. I ls this indicator for sustainability being achieved? Fully Highly @ftftfi Slightly Not at all Not applicable 25 2. lndicators of actiyltieq andproges.sc: monitoring/ supervision 2.1 Check whether staff at this leve! is being used appropriately for monitoring/ supervision. This indicator assesses whether the ramme is functioni Characteristics of the indicator Findings There are no superuision checklists. The staffs at this level sign the visitors' books when superuisory/ monitoing visifs are canied out. lt is noted that the staff of the regional level also carry out vrsifs fo the first line health facility and the community level. Staff members at this level should routinely only supervise the Ievel immediately below them. Staff should notsupervise the FLHF or community levels. 'Spot checks' may however be done from time to time. Staff members at this level should have empowered staff members at the level below them to supervise activities at their own level, as well as levels further down. lf staff members are not being used appropriately for monitoring/ supervision: . \Mry is this? Management at the region level has not yet appreciated fhis as duplication of resources. . \Mrich steps are being taken to improve the situation? The staffs have been advised not to exceed the distict levels for their superuisory and monitoing vr'sifs. ls this indicator for sustainability being achieved?I Highly Moderately Not at all Not applicableFully Slightly 2.2Checkwhether monitoring/ supervision is being planned and carried out in an efficient and manner. This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Findings One routine superyision visit per year must be done to each project (as a separate entity, or as a district). Supervision visits for CDTI should be integrated where appropriate with supervision of other programmes. Resources for supervision (human, transport etc.) should be efficiently used: * Using as few staff members as possible. * Planning trips to cut down on distance traveled. * Not spending unnecessarily many nights out etc. Supervision visits should be thorouqh, usinq a checklist. t I No superuision checklists were available. lndications at the immediate level below this showed vlsfors books, which were filled at times of visit. There were also indications that routine visits by the regional medical officec the regional, coordinator and the HKI representative were canied out. Describe the situation the previous year: Superuision/ monitoing visitswere canied out duing census, distibution of mectizan and data collection. ls this indicator for sustainability being achieved?I Fully Highly Moderately Not at all Not applicableSlightly 26 f I 2.3 Check whether there is a routine process of management of problems and successes, which are indicated the monitori This indicator assesses whetherthe programme is running efficiently and effectively, and whether management is beginning to accept ownerchip of the programme. 3. Indicators of activities and processesr Mectizan@ procuiem-ent ind distribution 3.1 Check whether sufficient Mectizan is being ordered, stored and distributed within the government system at this level. This indicator assesses whether the programme r's functioning efficiently, ifs processes are simple, and it is becoming more integrated into the govemmenf sysfem. Characteristics of the indicator Findings As soon as problems are identified as a result of supervision visits, or from coverage data (i.e. areas with low coverage) the appropriate manager should dealwith them. Such problems should usually be passed on to the appropriate managers at the next level below to deal with, with the necessary support-thus empowering these persons. Successes should be noted and reported, and appropriate feedback given. There should be evidence of action taken based on recommendations in previous monitoring exercises. There were no minutes of planning neither meetings nor regional oncho taskforce meeting. There's evidence of one community meeting. There was also no repofts of monitoing / superuisory visifs nor /effers of commendation as feedback. I !s this indicator for sustainability being achieved? Fully Highly Moderately E il Not at all Not applicable Characteristics of the indicator The Mectizan supply should be controlled within a government system This does not have to be the system routinely used for the supply of other drugs. The system should be effective, uncomplicated and efficient. This system should use dependable, sustainable resources for its operation. lt is desirable that the government should supply these resources. The system should supply sufficient Mectizan for the needs of all the proiects concerned. All mectizan ordeing and stock control documents were available. . \Mrat has been the trend in Mectizan supply? The system is effective, simple anf uncomplicated, only that, duing the last year there has been some slight delays in removing the mectizan consignment from the poft of entry due to some newly introduced govemment regulations that demands that all the impofted items should be taxed. However, fhrs has already been solved and distibution has already been going on ls this indicator for sustainability being achieved?a Fully Moderately Slightly Not at all Not applicable 27 Highly 4. lndicators_ of activities and processes: training and HSAM 4.1 Check whether staff members at this level are bei used as trainers. This indicator assesses whether the programme is functioning efficiently. Characteristics indicator of the Training of distict personnel is done by the regionalteam using the following manuals: CDD training manual, RHW training manual, CDTI treatment and ivermectin guide fortrainers of CDD, regional training guide for onchocerctasis. Staff at the national level should routinely only train staff at the level immediately below it. Staff members at this level should have empowered the level immediately below them to train lower levels. ls this indicator for sustainability being achieved?I Slightly Not at all Not applicableFully ModeratelyHighly 4.2 Check whether training is being planned and carried out in an efficient and rated manner This indicator assesses whetherthe programme is functioning efficiently. Characteristics of the indicator I There should be an objective need for each episode of training. This means there should be evidence that staff to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Repeat training of already skilled staff should not happen. If circumstances permit training for CDTI should be integrated with other training, e.g. in in- service training programmes. Resources for training (human, transport etc.) should be efficiently used: * Using as few staff members as possible. * Using as little time as possible (without sacrificing quality) x Using the most cost-effective site etc. Training of trainers is canied out at distict level. These officers then go out to train the first line health facility staff. Targeted training is organised annually with the annual retraining of CDDs. ln the previous year trainings were done before mectizan distibution started. is this indicator for sustainability being achieved?I Futly Iiftfl Moderately Slightly Not at all Not applicable 28 I 4.3 Check whether staff at this level is planning and carrying out HSAM activities in an efficient manner. This indicator assesses whether the programme is functioning efficiently and effectively, and whether managers are taking ownership of the programme. 5. lndicators of activities and processes: integration of support aCtivlties Characteristics of the indicator Findings a Staff members identify situations where decision makers lack information abouU commitment to CDT!, and undertake activities to inform and persuade these persons. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. Such activities should only be carried out at the national level, and at times at the level immediately below (but only when staff at that level asks for help). There is evidence that these HSAM activities have been effective and have led to action. The present method of health ed ucation, sensltisafio n, advocacy and mobilisation as presently ananged are efficient and produces positive outcome. ln the previous year govemment provided support in the areas of health education by producing IEC mateials while advocacy was suppofted by the NGDO partner. I ls this indicator for sustainability being achieved? Fully Moderately Slightly Not at all Not applicable 5.1 Check whether the various programme support activities are being planned and carried out in an integrated manner. This indicator assesses whether the programme is functioning efficiently. 6. lndlcators of resources: financial Characteristics of the indicator Staff combines two or more tasks on a single trip: . Monitoring / superuision for CDTI and other projects. . Training for CDTI and other projects. . HSAM. ' Fetch records or to deliver Mectizan. There's good cooperation between the programs especially the Tuberculosis and leprosy control. ln most cases, field activities are canied out together. Before the project received it's vehicle, it used that belonging to Tuberculosis and leprosy control programme. ls this indicator for sustainability being achieved?I Fully @ Moderately Slightly Not at all Not applicable 6.1 Check whether appropriate amounts are budgeted for planned onchocerciasis control activities at this level This indicator assesses whether the programme is functioning efficiently. 29 f Characteristics of the indicator ! The costs for each onchocerciasis control activity in the year plan at this level should be clearly spelt out in a budget. There is evidence of a const reduction/ containment strategy (e.9. targeted training, HMAS and monitoring/ supervision; training conducted at the next level below etc). Project managers should have a clear estimate of the funds that will be available to them for onchocerciasis control in the coming year, and should be able to justify this belief. The total amount budgeted for in the year plan should fall within this estimated income. There were no available records to indicate budgeted amounts for onchocerciasis control at the regional level neither the regional level officers can asceftain how much will be available for control activities. lf budgeting has been inappropriate: . \Nhy is this? There's no budget provision for onchocercrasis control at the regional level. But provisions are made at the distict level. I ls this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not applicable 6.2 Check whether the government is budgeting and disbursing sufficient amounts for onchocerciasis control yearly. This indicator assesses whetherthe programme is becoming integrated, and whether management is accepting ownership of the programme and can mobilise the resourc* it needs. 6.3 Check whether in case of a deficit between estimated costs and the amount provided by the goyernment, dependable provision is being made to meet it. This indicator assesses whether management is able to mobilise the resources rt needs, as well as its commitmentto ownership. Characteristics of the indicator The relative budgetary contributions of the government and other partners to onchocerciasis control should be clearly spelt out. The amount that the government has budgeted in one or more specific onchcerciasis control budget lines (e.9. current and capital) should be increasing yearly, as a proportion of total expenses. By the end of Year 5 of APOC funding the bulk of onchocerciasis control expenses at national level should be met from government funds. The amounts actually disbursed from such budget lines should be increasing yearly, as a proportion of total expenses. (Note that actual disbursement is more important than budgeting, and is a real siqn of political commitment). There are no budget disbursements for this year 2003, nor for the previous years. Budgets are provided for only at the distict level. I ls this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not applicable 30 ! B I Not at all * Characteristics of the indicator Findings No figures available for all the years. Project management at national level should be aware of the shortfall, if one exists, and of its size. Project management should have specific and realistic plans to bridge the shortfall. lf it is planned that non-government sources of funding are to be used after APOC funding ends, written commitment for this should have been obtained at the highest level in these donor organisations. Projects in their third year of APOC funding should also be well on the way to achieving such commitment. ls this indicator for sustainability being achieved?! Slightly Not applicableFully Highly Moderately Not at all 6.4 Check whether funds disbursed for onchocerciasis contro! from the budget at this level are efficiently managed This indicator assesses whetherthe programme is functioning efficiently. 7. lndicators of resources: transport and other material resources 7.1 Check if adequate and appropriate transport and other material resources are available for CDT! activities at this leve!. This indicator assesses whether the programme is functioning effectively, and whether it is able to mobilise the resources rt needs. Characteristics of the indicator The budget holder should be using a control system with the following elements: * Approval of each item of expenditure. * Allocation of expenditure against specific budget headings. '* Regular calculation of residual amounts under budget headings. All the funds released yearly should be spent as budgeted. Approval of expenditure : Management of the APOC funds is done at the project level where approvals of expenditures are made. Allocation of expenditure: Ihrs is done as per the approved APOC budget. Regular insight into budget line balances: Balances are checked before approval Fully Highly Slightly Not at all Not applicable Characteristics of the indicator Findings There are adequate numbers of functional vehicles available for necessary CDTI activities. The vehicles are appropriate for the purpose they are There are adequate office equipment; this include Air conditioner, Fax machine and r\^^^^ ^h^l^^^^i^F 3l a Moderate I Canon photocopier.intended to fulfil - tough but not luxurious. There is sufficient office equipment available, in working order: computers, printers, photocopiers - also stationery and materials for training and HSAM. The running costs for these vehicles and equipment are met from dependable, sustainable sources. Findinqs Describe the situation: Type of vehicle No. Source Functional status and adequacy for CDTI tasks*** 1 2 4WD Toyota Double Cabin 1 APOC Functional Motor les 6 APOC Functional Type of equipment AIo. Source Functional status and adequacy for CDT! tasks*** 1. 2. 2 2 APOC 1 Functional and 1 clashed Portable nters 2 APOC 1 Funtionable and I stolen Des co uters 1 HKI Functional Training/ HSAM material No. Source* Functional status and adequacy for CDTI tasks* * 1 NIL * other ** ,s it and is there of itfor the ? Describe the adequacy/ functionality of the present vehicles, equipment and materials, considering the work sti!! to be done in the coming 540 years: o The logisfics and capital equipment are adequate and functional but the vehicle will need to be replaced in the fifth year. lf transport, equipment and materials are inadequate and/or funded frot sources which are not dependable: . Why is this? . A digital camera to record field activities is required and replacement of the stolen portable pinter. ls this indicator for sustainability being achieved?I Not at all Not applicableFully Highly SlightlylVloderately 7.2 Check if transport and other material resources in use at this level are adequately and maintained This indicator assesses whether the programme is functioning eflectively and efficien{y. 32 Characteristics of the indicator Findings There is a routine maintenance schedule for each vehicle, which is adhered to and recorded. This includes weekly driver maintenance, scheduled garage servicing, and replacement of worn tires. Equipment such as photocopiers and generators is regularly maintained according to a schedule, and this is recorded. Staff members have ways of coping when vehicles break down or are not available, so that CDTI activities are not disrupted. The costs for vehicle and equipment maintenance and repair are met by from dependable/ sustainable sources. Repairs to vehicles and equipment are rapidly and efficiently done. There's routine and regular seruicing of vehicles according to the kilometerstraveled. All capitalequipments are seruiced regularly. The cost of maintenance of these mateials is with the APOC funds There's presently a dependable source of support from APOC. HK lntemational has promised to support the maintaining of these ifems when APOC leaves. t !s this indicator for sustainability being achieved? Fully Highly Slightly Not at all Not applicable 7.3 Check if the transport available at this level is appropriately managed and used This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Findings Transport is used at this level, and to undertake support activities at the next level. lt should not be used for CDTI implementation activities at lower levels. The use of transport is properly controlled:x Trips made for CDTI purposes should be properly authorised in writing by the relevant official. * Each trip undertaken for CDTI purposes should be recorded in a log book. * Trip authorities and log book entries and regularly reconciled, and action taken if there are discrepancies. There are logbooks for vehicle movement. The poject coordin ator m u st approve each tip undeftaken and vehicles a re not used in implementation at lower levels. I ls this indicator for sustainability being achieved? Fully ftffi Moderately Slightly Not at all Not applicable 7.4 Check if there are appropriate and realistic plans for the replacement of transport and other material resources, when APOC support comes to an end. This indicator assesses whether the programme managers are taking ownership of the and are able to find resources for it. Characteristics of the indicator Findings Management should know that replacements will be needed before the end of the programme, and have specilic, realistic plans to meet the need at that time. There's no formal agreement for the replacement of vehicles by the suppofting NGO. But the tasraaaalalirra aeaaiaaA lhal har 33 [/loderately I representative promised that her agency would be ready to give fhe necessary supportwhen the time*comes. It should be planned that the government will:x Provide replacements for vehicles and equipment. * Maintain existing vehicles and equipment. * Provide stationery and materials for training and HSAM. !f it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in these donor organisations. !s this indicator for sustainability being achieved?I Not applicableFully Highly Moderately Not at allSlightly 8. lndicators of resources: human resources 8.1 Check whether staff at this leve! is skilled, knowledgeable and committed, regarding the implementation of CDT! in its area of operation. This indicator assesses whether the programme has been able to develop sufficient resiources for itself. Characteristics of the indicator Findings The number of staff members in the government health service at this level should be appropriate to the task in hand: not too many or too few. Team members should have enough knowledge and skillto undertake allthe key CDTI activities themselves, without help: * Planning * Data management * Report writing * Computer skills * Training and HSAM * Mectizan ordering/ distribution * Monitoring/supervision There should be evidence that the team is committed to the success of the programme (from the evidence of the partners, as well as workers at the next level below; from written reports and timetables). * The staff are knowledgeable and committed to undeftake CDTI activities. Findi Both the officers working directly with the oncho program at the regional offtce are committed to their work. They are full time staff and are interested in the control effort. Level of skill: is it adequate to perform the Adequate Adequate dequate 2 1 2 2 2 2 2 A A A Area of skill Plannin Report witing Training and HSAM Monitoing/ superuision Data Computer skills Mectizan distibution No. of skilled persons Adequate Describe the present situation: . Particulars of current staff ls this indicator for sustainability being achieved?I Fully Moderately Slightly Not at all Not applicableHrghly 34 8.2 Check whether staff at this level is stable, and whether provision is made for passing on GDTI skills when a trained person moves away. This indicator assesses whether the programme has been are able to maintain ifs resources. 9. lndicators of impact: coverage 9.1 Check whether all projects in the country have a satisfactory therapeutic coverage rate. This indicator assesses whether the programme is effective - if the rates are poor the project is clearly struggling, and /ess sustainable. Characteristics of the indicator Findings 147 villages in 2003119 villages in the fist year of CDTI 77.47% therapeutic coverage rate was attained in the 118 villages was attained. ln the second year 69.3 % lhaeaaar*ia ^Atra.aa6 vtao ahlainad in 11O rillqaao ln lha Characteristics of the indicator Findings . Staff at this level should remain in one post for at least five years. . There should be immediate orientation (in CDTI) of new, unskilled project staff members. There are seven members on the regional health management team(RhMT). Two are permanent staff of the onchocerciasis control and have been on the team since inception. There's no indication that either of these staff members will be transfened out the program in the near future. Members have been on the team since it started. Before APOC funding treatment for oncho was clinic based in some of the villages in Muheza distict. APOC's anival allowed the formation of a project team when REMO/ REA sfudies established the presence of onchocerclasis in Muheza, Korogwe, and Lushoto drsfncfs. I !s this indicator for sustainability being achieved? Futly IiMil Moderately Slightly Not at all Not applicable 35 therapeutic coverage was obtained in 119 villages. ln the third year 1 47 villages are targeted for treatment. And 1 18 villages in 2001 Mass treatment with mectizan started in these communities in the year 2001. Based on reactions that the population had on srde effecfs afterthe first doze, there was a decrease in the coverage rate for the second year. It is hoped that after the cunent health education sensrtisafrbn and mobilisation, treatment coverage rate for the their year will improve significantly. It is difficult to calculate the geographical coverage rate since the projects have not reached their ultimate treatment objective. t Your overall iudqement: is this indicator of sustainability beinq achieved? Fully (1007o of projects have a therapeutic coverage rate >65% - stable or increasing) Highly (90- 99o/o of projects) Moderately (80-89% of proiects) Slishtly (70- 79Yo ol projects) 36 Negligrbly (<70o/o of protects) 1lnstrument 2: DISTRIGT/ LGA LEVEL lndicators of activities and processes: planning 1.4 Check whether the year plan for GDTI appears as part of an overa!! written plan for the activities of the districU LGA. This indicator assesses whether the programme has become integrated into the health seruice, and whether managemenf is beginning to accept ownership of the programme - both good for sustainability. This indicator assesses whether ,s of the Characteristics of the indicator Findings I CDTI should be integrated into the overallwritten plan (showing that staff at this level consider CDTI to be part of their yearly routine, like any other programme). The plan should make provision for all key activities: Mectizan supply; targeted training; targeted HSAM, targeted monitoring/ supervision. Year plans should be drawn up in a participatory way. Year plans must take into account community requirements for the timing of distribution. The year plan was available and integrated in the overalldistict development plan, drawn in a participatory manner. Some NGO like lish AID do pafticipate. Butthe plans do not take into account community requirements for the timing of the distibution. The distict and the health facilities determine the time of the distibution. t ls this indicator for sustainability being achieved? Fully Highly Slightly Not at all Not applicable 2.1 Check whether the districu LGA health management team is taking full responsibility for the implementation of GDTI at this level. Characteristics of the indicator Findings It should be the management team at this level, and not higher levels/ NGDO leadership, which is initiating the key CDT! activities: planning, targeted monitoring/ supervision, targeted training and HSAM, Mectizan ordering/ distribution. There should be a focal person for CDTI activities. The Council Management Team / or Health Management Team composed of the Pharmacist, DMO, the Medical Officer in charge and the Hospital Secretary and the Focal Person for CDTI take full responsibility for management and implementation of CDTls. The team is highly knowledgeable about program implementation and management issu*es I Your overalljudgement: is this indicator for sustainability being achieved? Fully Hrghly Moderately Slightly Not at all Not applicable 37 'ffiilr{i'H .,{ is. - I nd icatop_ ele_clivitj es aOC-pf gqe999s : 4 o n i to ri n g/ s u pe rv i s i o n Characteristics of the indicator Findings The reporting process should take place within the government system, not using other resources. Data being transmitted includes: coverage reports; Mectizan statistics; trai ning reports; distri bution reports; fi nanci al reports. The repofting system is within the govemment system. Coverage reports and mectizan sfafrsfrbs were available. T ls this indicator for sustainability being achieved? Fully Highly lVloderately Slightly Not at all Not applicable 3.1 Check whether routine data concerning CDTI activities are being transmitted from this within the nt This indicator assesses whetherthe programme has become more integrated into the govemmenf sysfem, and is functioning effectively. 3.3 Check whether there is a routine process of management of problems and successes, which are indicated by the monitoring system. This indicatorassesses whetherthe programme rs running efficiently and effectively, and whether management is beginning to accept ownership of the programme. 3.2 Check whether the responsible persons at the districU LGA level are efficiently s GDT|activ at the FLHF level in an rated manner, Characteristics of the indicator Findings I Staffat this level should routinely only superuise the FLHF, and not the community level. 'Spot checks' may however be done from time to time. Although one routine supervision visit per FLHF per year is necessarT, supervision visits should focus more on FLIIFs where there are proven problems - each supervision visit must be justified. Supervision visits for CDTI should be integrated with supervision of other programmes (e.9. through a shared checklist). Transport for supervisory visits should be shared with other proqrammes. Superuision is integrated with other programs like MCH kit distibution and immunisation. I ls this indicator for sustainability being achieved? Fully Hrghly Moderately Slightly Not at all Not applicable 38 Characteristics of the indicator Findings I As soon as problems are identified as a result of supervision visits, or from coverage data (i.e. areas with low coverage) the appropriate manager should dealwith them. \A/lrere relevant such problems should be passed on to the appropriate FLHF staff to deal with, with the necessary support - thus empowering these persons. Successes should be noted and reported, and appropriate feedback given. There should be evidence of action taken based on recommendations in previous monitoring exercises. I I Little attempt is made atthis levelto manage problems and successes indicated by the monitoing sysfem. lt is evident that much of this is left to the lower levels. There was no evidence of supervisory visits, itineraies and reports. However, visitor"s book at FLHL showed that the DOT had visited on severaloccasions. I ls this indicator for sustainability being achieved? Fully Highly llilr rr lr11';1[,..1y Slightly Not at all Not applicable 1. !4dicators of activities and processes: Mectiz-a4@p1oc urement and distribution 4.1 Check whether sufficient Mectizan is being ordered and received yearly, and in good time. This indicator assesses whether the programme is functioning effectively. Characteristics of the indicator Findings The order forms for the districU LGA exist, and should be based on FLHF and community requests. The Mectizan should be available at this level in time for distribution at the time requested by the communities. There should be no reports of shortages and/ or late supply. lf there have been shortages, there should be specific plans to remedy them. Considei ng the di stibution peiod it appears they receive the mectizan on time, the orders are e.nouOh. I ls this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 4.2Checkwhether Mectizan is being collected, stored and effectively administered within the ment at this level. Characteristics of the indicator Findings I The Mectizan should be controlled within a government system. This does not have to be the system routinely used for the supply of other drugs. The system should be effective, uncomplicated and efftcient The districU LGA level should fetch its Mectizan from the project level itself, by means of transport supplied and paid for by government at this level. I The mectizan is controlled within the govemment system. lt is collected, stored and administered within the system but there was sheerabsence of logbooks and tip authoisation forms. 39 D l I Your overall judgement: is this indicator for sustainability being achieved? Fully Highly lVloderately Slightly Not at all Not applicable 5.1 Check whether staff members at this level are bei used as trainers. Th i s i ndicator assesses whether the prog ram me is fu nctioni ng effi ci entl y. Characteristics of the indicator Findings Staffshould routinely only train staffat the FLHF level, and not at the community level. Staffshould have empowered staffat the FLHF level to see to their own training needs as much as possible, and to conduct training activities at the com munity level independently. \Mrerever possible staff at this level should conduct their own training for CDT!, if they have need for such traininq. Somefimes DOIs go as far as training CDDs. This should not be the case. The situation has been so since the program stafted. The DOT seerns to be ovenealous and even though they have trained the FLHF staff, the DOT still goes furtherto train the CDDs and routinely superuised them. !s this indicator for sustainability being achieved?, Fully Highly [/oderately Slightly Not at all Not applicable 5.2 Check whether training is being planned and carried out in an efficient and integrated manner This indicator assesses whether the ,s 5.3 Check whether staff at this level is planning and carrying out HSAM activities in an efficient manner. This indicator assesses whetherthe prqlramme isfunctioning efficiently and effectively, and whether managers are taking ownership of the programme. FindingsCharacteristics of the indicator There should be an objective need for each episode of training. This means there should be evidence that staff to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Repeat training of already skilled staff should not happen. If circumstances permit training for CDTI should be integrated with other trainin& e.g. in in-service training programmes. Resources for training (human, transport etc.) should be efficiently used:x Using as few staff members as possible.x Using as little time as possible (without sacrificing quality) * Choosing the most cost-effective site etc. I Training is being planned but does not seem to be cost effective. The duration of the training is too long. Some trai ning mateials were available - manuals. The training/ retraining for CDDs /asfs for three days even when they were trained the previous year. The situation has been the same since the program sfarfed. !s this indicator for sustainability being achieved?I Slightly Not at all Not applicableFully Highly IVloderately 40 tr Characteristics of the indicator Findings Staff members identify situations where decision makers lack information abouU commitment to CDT!, and undertake activities to inform and persuade these persons. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. Such activities should only be carried out at this district/ LGA level, and at times at the FLHF level (but only when staffat that level asks for help). There is evidence that these HSAM activities have been effective and have led to action. The staffs have planned for the advocacy workshops and have already conducted some at vaious levels including training of councillors at the ward level. Similar activitiestook place in 2001 and 2002. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable lndicators of resources: financial 6.1 Check whether appropriate amounts are budgeted for planned GDTI activities at this level. This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Findings The costs for each CDTI activity in the year plan at this level should be clearly spelt out in a budget. There is evidence of a cost reduction/ containment strategy (e.9. targeted training, HMAS and monitoring/ supervision; training conducted at FLHF leveletc). Managers at this level should have a clear estimate of the funds that will be available to them for CDTI in the coming year, and should be able to justify this belief. The total amount budgeted for in the year plan should fall within this estimated income. I * There are pointers that the policy makers are aware that the distict budget should be increasing as APOC funding reduces. B udgeta ry p rovi sio n s for O ncho control activities at the distict level have been met.Ihis is indicated by the 0.5 USD per capita given to the distict for vulnerable lssues ! Your overalljudgement: is this indicator for sustainability being achieved? Fully Hrghly Moderately Slightly Not at all Not applicable 6.2 Check whether the government at this level is budgeting and disbursing increasing amounts for CDTI This indicator assesses whetherthe programme is becoming integrated, and whether management is beginning to accept ownership of the programme and can mobilise the resources it needs. 4t 6, Characteristics of the indicator Findings The relative budgetary contributions of the local government and other partners to CDTI should be clearly spelt out. The amount that the government has budgeted in one or more specific CDTI budget lines should be increasing yearly. By the end of Year 5 of APOC funding the bulk of CDTlexpenses at this level should be met from local government funds. The amounts actuallv disbursed from such budget lines should be increasing yearly, as a proportion of total expenses. (Note that actual disbursement is more important than budgeting, and is a real sign of political commitment). {. Budgetary provisions have been made forthe cunent year. There is documentary evidence thatfor one distict and vehal assurance from the other. I ls this indicator for sustainability being achieved? Fully Highly Moderately Shghtly Not at all Not applicable 6.3 Check whether in case of a deficit between estimated costs and the amount provided by the government, dependable provision is being made to meet it. This indicator assesses whether management is able to mobilise the resources ff needs, as well as its commitment to ownership. Characteristics of the indicator Findings !f there is a shortfall the management should have specific and realistic plans to bridge it. !f it is planned that non-government sources of funding are to be used after APOC funding ends, written commitment for this should have been obtained at the highest level in these donor orqanisations. * The deficit cannot be calculated because there are no budget records. ls this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicabte 6.4 Check whether funds disbursed for CDT! from the budget at this level are efficiently managed. This indicator assesses whether the programme is functioning efficiently. 42 n I Characteristics of the indicator Findings The budget holder should be using a control system with the following elements: x Approval of each item of expenditure. * Allocation of expenditure against specific budget headings. * Regular calculation of residual amounts under budget headings. Allthe funds released yearly should be spent as budgeted. * Budgetary provisions have just been made and disbursements have not been done. I !s this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 7. lndicators of resources: transport and other material resources 7.1 Check whether adequate and appropriate transport and materials are available for necessary CDTI activities at this !eve!. This indicator assesses whether the programme is functioning effectively, and whether it is able to mobilise the resources ff needs. Describe the situation: Describe the adequacy of the present vehicles and materials, considering the work still to be done in the coming 410 years: There are 6functional motorcycles two per district visited There arc thto DOTs per distia therefore these vehicles are adeqrmte. Characteristics of the indicator Findings I There are adequate numbers of appropriate, functional vehicles available for necessary CDTI activities. The running costs for these vehicles are met by the government. There are sufficient materials available for training and HSAM. Each distict hastwo motorcycles provided APOC. Running cosfs is with APOC funds. Disticts have promised to provide funds when APOC leaves. Findi for CDTI tasks**of vehicle No. Source* Functional status and Motorcycles @ district has 2 motorcvcles 6 APOC Functional Trainine/ HSAM material No. Source* X'unctional status and adequacy for CDTI tasks** N/ A N/Anone ** Is it worhing, and is there enough of ir lor the job?* other ls this indicator for sustainability being achieved?I Fully Hrghly Moderately Slightly Not at al! Not applicable T.2Checkwhether transport at this level is adequately and appropriately maintained. This indicator assesses whether the programme is functioning effectively and efficiently. 43 Characteristics of the indicator Findings There is a routine maintenance schedule for each vehicle, which is adhered to and recorded. This includes weekly driver malntenance, scheduled garage servicing, and replacement of worn tyres. The costs for vehicle and equipment maintenance and repair are met by the government. Repairs are rapidly and efficiently done. Staff have ways of coping when vehicles break down or are not available, so that CDTI activities are not disrupted. Operation and maintenance are budgeted for since 2003. No logbooks and maintenance schedules were seen. APOC meets a// cosfs. I ls this indicator for sustainability being achieved? Fully Hrghly Moderately Slightly Not at all Not applicable 7.3 Check whether the transport available at this level is appropriately managed and used, in an integrated way. This indicator assesses whetherthe programme is functioning efficiently. Characteristics of the indicator Findings Transport is used at this level, and to undertake support activities at the FLHF level. lt should not be used for CDTI implementation activities at the community level. Trips made for CDTI purposes should be properly authorised in writing by the relevant official. Each trip undertaken should be recorded in a log book. Transport provided for CDTI, and that provided for other programmes, should be combined as a poolto be used for legitimate activities of all programmes at this level. No logbooks nor superuisory plans seen. Managers repofted they only travel when authoised. Management not empowered to control logistics at this level. I ls this indicator for sustainability being achieved? Fully Highly Moderately Slrghtly Not at all Not applicable 7.4 Check whether there are appropriate and realistic plans for the replacement of transport and materials when APOC support comes to an end. This indicator assesses whether the programme managers are taking ownership of the programme, and are able to tind r*ources for it. 44 Characteristics of the indicator Findings I Management should know that replacements will be needed before the end of the programme, and have specific, realistic plans to meet the need at that time. It should be planned that the government will:x Provide replacements for vehicles. * Maintain existing vehicles. * Provide stationery and materials for training and HSAM. !f it is planned that replacement will be ftom non-government sources, written commitment for this should have been obtained at the hiqhest level in these donor organisations. I * No plans indicated re pl aceme nt of tran sport. They are yetto embark on a fully-fledged plan for transport. I ls this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable ig' lndicators of resources: human resources 8.1 Check whether staff at this level is skilled and knowledgeable, regarding the implementation of CDTI in its area of operation. This indicator assesses whether the programme has been able to develop sufficient resources for itself. Describe the present situation: . Particulars of current staff . lnformation about staff stability and in-service training: The sfaffs are permanent and pensionable in the civil seruice. Characteristics of the indicator Findings Staff should have enough knowledge and skill to undertake allthe key CDTI activities themselves: planning, training, HSAM, ensuring Mectizan supply, monitoring/ supervision. Staff at this level should remain in one post for at least five years. There should be immediate training (in CDT!) of new, unskilled project staff members who have CDT! responsibilities. I I Staff atthe distict and FLHF are very knowledgeable on CDTI and HSAM. rf Find !s this indicator for sustainability being achieved?t Not at all Not applicableFully Hrghly Moderately Slightly Area of skill No. of skilled oersons Level of skill: is it adequate to perform the job? Planning Training and HSAM 4 Adequate Monitoing/ superuision 4 Adequate Mectiza n orde i ng/ di sti b utio n 4 Adequate 45 Characteristics of the indicator Findings Staff members express satisfaction with their present responsibilities There is evidence of specific motivational practices and rewards within the programme: awards, financial incentives, compensation in cash or kind. Salaries/ wages and allowances are paid regularly. Staff members mention non-financial rewards inherent in CDT|work. There is evidence from partners and workers at the FLHF level that staff members are committed to their CDT work. * The staff are highly commifted, however, there are no incentive schemes. I ls this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 8.2 Check whether staff members at this level are committed to their CDTI work. This indicator assesses whetherthe programme has been able to develop sufficient resources for itself. is. lndicators of j411act: qgveJage 9.1 Check whether the districu LGA has a satisfactory geographical coverage rate. This indicator assesses whether the programme is effective - if the rate is poor the project is clearly struggling, and /ess sustainable. 9.2 Check whether the districu LGA has a satisfactory therapeutic coverage rate. This indicator assesses whether the programme is effective - if the rate is poor the project is clearly struggling, and /ess sustainable. Characteristics of the indicator Findings All sub-districts and communities identified by the latest REMO should be under treatment (i.e. the geographical coverage rate is 1007o). This rate should be stable or increasinq.I Distibution is on going. The geographical coverage rate cannot be calculated because the projects have not reached their ultimate treatment goal. rl. !s this indicator of sustainability being achieved?I Fully (100% of communities are doing CDTI) Highly (95-99% of communities - stable or increasinq) Moderately (90-94Yo of communities - stable or increasing) Slightly (85-89% of communities - stable or increasinq) Negligibly (<85% of communities) 46 Characteristics of the indicator Findings All communities in the districU LGA should have a therapeutic coverage rate of 65% or higher. These rates should be stable or increasing. I Distibution is sfr/ ongoing,f Your overalliudgement is this indicator of sustainability being achieved?I Fully (100% of communities have a therapeutic coverage rate >65% - stable or increasing) Highly (9G 99% of communitie s) Moderately (80- 89% of communities) Slightly (70- 79o/o of communities) Negligibly (<70o/o of communities) 47 1lnstrument 3: first line health facility (FLHF) level lndicatgrrs of activities and processes: planning 1.1 Check if there is a written year plan for CDTI in the FLHF area This indicafor assesses whether the programme is being planned in an effective and integrated manner, and whether management is beginning to accept ownership of the programme. 2. lndiqator. gf activiti.eg and prgcesses: leadership 2.1 Check whether the FLHF management team is taking full responsibility for CDTI at this level, in an integrated manner This indicafor assesses whether the CDTI project is integrated into the health system, and whether management is beginning to accept ownership of the programme. Characteristics of the indicdor Findings I There should be a written plan or timetable in existence, for the most recent round of CDTI (this is recommended). ldeally the plan should be integrated into the overall year plan for the FLHF area. ldeally CDTI should form part of the 'minimum' or 'recommended' package for this !eve! No witten plans seen at the health facility level. The FLHF fully depend on the distict work plans. Sfaffs at this level are not empowered to develop their own plans. I ls this indicator for sustainability being achieved? Fully Highly Moderately Slightly |[r@,rE@t} Not applicable Characteristics of the indicator Findings The FLHF management team and all health staff at this leve! consider the program as theirs and are initiating the key CDTI activities: planning, monitoring/ supervision, training, HSAM, Mectizan ordering/ distribution. The political head/ senior politician at this level should know about CDTI and have participated in some CDTI activities. T lntegration of the activities was low. DOTs prepare plans and decide when treatment is to be given. Some community leaders are involved in the training as trainees with the CDDs. The first line health facility management team consider the program as theirs but do not initiate activities eg planning and monitoing. The management of the program is taken over by the DOT. * ls this indicator for sustainability being achieved?! 48 Not applicable Fully Highly Moderately Not at allSlightly 3. lndicators of activities and processes: mon su 3.1 Check whether routine and necessary data concerning CDTI activities at this level are being transmitted entirely within the government system. This indicafor assesses whether the programme is becoming more integrated into the gove rn m e nt h e alth sysfem. 3.2Check whether health staff at this level is routinely and efficiently supervising CDTI activity at the communities on site in an integrated manner This indicaforassesses whether the CDTI programme is being implemented efficiently. Characteristics of the indicator Findings Although one routine supervision visit per community per year is necessary, supervision The FLHF sfaffs supervise CDTI ^^t:,.:l:^^ ^l tL^ ^^--.,^:1,. t^..^t^ Characteristics of the indicator Findings The reporting process should be within the government system, not using other resources. Data being transmitted includes: coverage reports; distribution reports; Mectizan statistics; training reports. The reporting process and data are transmitted within the government sysfem. The FLHF officers superuise their subordinafes ie CDDs and submit reports to DOTs. Treatment summary sheefs were available. ! Your overa!!judgement: is this indicator for sustainability being achieved? Fu!!y Moderately Slightly Not at all Not applicable 49 Highly visits should focus more on communities where there are proven problems- each supervision visit must be justified. During visits to communities FLHF staff should turn their attention to as many health related programmes and problems as possible. activities at the communi$ levels especially dui ng census, distribution and collection of data. Superuision is not based on identified problems and is therefore not targeted but a routine activity. There are no superuisory checklists. There is no transport available fo sfar7s at the FLHF therefore no logbooks. Each community had a register where the superuisor signed when they visited. ! Your overalljudgement: is this indicator for sustainability being achieved? Fully Moderately Slightly Not at all Not applicable 3.3 Check whether there is a routine process of management of problems and successes, which are indicated by the monitoring system (coverage data, visits and reports) This indicafor assesses whether the programme is being implemented efficiently and effectively, and whether management is beginning to accept ownership of the programme. Characteristics of the indicator Findings As soon as problems are identified through supervisory visits, coverage data etc. (e.9. communities with low coverage) health staff at this leve! should dealwith them. Where relevant such problems should be passed on to the relevant community to dealwith, with the necessary support- thus empowering communities to make decisions on CDTI and cope with problems. Successes should be noted and reported, and appropriate feedback given to communities. There should be evidence of action taken based on recommendations in previous monitoring exercises. The common problem repofted was mild reactions to treatment. There is no routine management of side effects but when problems are identified the patients are given health advise but if such cases occur where the FLHF is available, treatment with anti-histami nes is given if the lafter is available. The rssue of not managing mild reactions, especially when anti-histamines were not available, led to a drop in the number of people taking mectizan in the sec*ond year from 74% to 69%. t Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Not at allModerately Not applicable 50 Highly I Slightly 4. lndicitors of activities and processes: Mectizan@ procurement and distributlon 4.1 Check whether sufficient Mectizan is being ordered annually, and in good time. This indicaforassesses whether the programme is functioning effectively. Characteristics of the indicator Findings I The order forms for the FLHF area exist, and orders should be based on the requests from the community. The Mectizan should be availableat this level in time for distribution at the time requested by the communities. There should be no reports of shortages and/or late supply. !f there have been shortages, there should be specific plans to remedy them. Reguesf forms were available to DOTs. The orders are based on populations and the mectizan anives early except for the current year when drugs anived late due to problems with the Tanzania Tax authoity. No shorfages of drugs were reported at the FLHF. Orderforms are available and no shorfages repofted for all years. tlr I Your overalljudgement: is this indicator for sustainability being achieved? Fully Moderately Slightly Not at al! Not applicable 5l Highly 4.2 Check whether Mectizan is being collected, stored and effectively administered within the government system at this Ievel. This indicafor assesses whether the programme is functioning efficiently, its processes are simple, and it is becoming more integrated into the government system. 5. lndiqators qf actiyities and plocesses: training and HSAM Characteristics of the indicator Findings The Mectizan should be controlled within a government system. This does not have to be the system routinely used for the supply of other drugs. The system should be effective, uncomplicated and efficient The FLHF level should fetch its Mectizan from the districU LGA !eve! itself, by means of transport supplied and paid for by government at this level. Communities should fetch their Mectizan from the FLHF themselves. However in situations where villages are very far from health centres, or where it is easy/practicable for FLHF staff to deliver the Mectizan, they may help with the delivery. ! I Mectizan is controlled within the government sysfem at the FLHF and distibuted to allthe communities. ls this indicator for sustainability being achieved?I Fu!!y Moderately Slightly Not at all Not applicable 5.1Check whethertraining is being planned and carried out in an efficient manner This indicafor assesses whether the programme is functioning efficiently. Characteristics of the indicator Findings There should be an objective need for each episode of training. This means there should be evidence that CDDs to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Resources for training (human, transport etc.) should be efficiently used: * Using as few staff members as possible. * Using as little time as possible (without sacrificing quality) * Choosing the most cost-effective site etc. Training was not efficient because it was not targeted, it was done routinely. Routine and non-targeted training was done at the FLHF, No needs assessment was done before training. It was also noted that the period of training and retraining is long taking 3 !"' 52 Highly - I ls this indicator for sustainability being achieved? Fully Highly Moderately Not at a!! Not applicable 5.2Check whether staff at this level is planning and carrying out HSAM activities in an efficient manner. This indicaforassesses whether the programme is functioning efficiently and effectively, and whether managers are taking ownership of the programme. 6. Indlcators of resources: flnancial 6.1 Check whetherthe costs involved in planned CDTI activities at this level a: defined and bu eted for This indicaforassesses whether the programme is functioning efficiently. FindingsCharacteristics of the indicator HSAM is done and planned for the cunent year. Health education is done efficiently and effectively. The situation was the same for the year 2002 The district has been keen at plan*ning for HSAM in the year 2003. Staff members identiff situations where decision makers lack information aboufl commitment to CDTI, and undertake activities to inform and persuade these persons. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. There is evidence that these HSAM activities have been effective and have led to action. !s this indicator for sustainability being achieved?I Not at all Not applicable Fu!!y Highly Slightly[/oderately Characteristics of the indicator Findings There is no evidence of a budget at the FLHF The district provides funds when activities are to be implemented. Budgets are not prepared at the FLHF level. Therefore the cost at this level cannot be calculated. The costs for each CDTI related activity in the year plan should be clearly spelt out in a budget. These activities include monitoring/supervision, training, HSAM, and arranging Mectizan supply. The staff should be able to justiff the amount they plan to use. There should be evidence of cost containment (e.g. by targeting training, HMAS and supervision). ls the indicator for sustainability bdng achieved?I Fully Highly Moderately Slightly Not licable Not at all 53 Slightly 6.2 Check whether sufficient funds to cover these costs are fully or increasingly bein disbursed from FLHF and/or districU LGA resources Characteristics of the indicator Findings Funding disbursed is enough to enable targeted, essential CDTI activities at this !eve! to be carried out. The relative contributions of all sources of funding should be clearly spelt out. The proportion provided by the government (FLHF and/ or district/ LGA levels) should be the maior one by now. * No records available. Government not providing any funds is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not applicable 7. lndicators of resouqces: transport and other material resources 7.1Check whether adequate and appropriate transport and materials are available for necessary GDTI activities at this level. This indicafor assesses whether the programme is functioning effectively, and whether it is able to mobilise the resources it needs. 7.2Check whether transport at this level is adequately and appropriately maintained. Characteristics of the indicator Findings There are adequate numbers of appropriate, functional vehicles (of any type) available for necessary CDTI activities. The running costs for this transport are met by a dependable source (e.9. the government). There are sufficient materials available for training and HSAM. There is no transport available and no training / HSAM materials seen. !s this indicator for sustainability being achleved?! Fully Highly Moderately Slightly Not licable Not at all 54 Not at all n This indicafor assesses whether the programme is functioning effectively and efficiently. 7.3 Check whetherthe transport available at this level is appropriately managed and used, in an integrated way. This indicafor assesses whether the 7.4 Check whetherthere are appropriate and realistic plans for the replacement of transport and materials when APOC support comes to an end This indicaforassesses whether the programme managers are taking ownership of the programme, and are able to find resources for it. ,b FindingsCharacteristics of the indicator There is a routine maintenance schedule for vehicles (where relevant), which is adhered to and recorded. The costs for vehicle and equipment maintenance and repair are met by dependable sources (e.9. the government). Repairs are rapidly and efficiently done. Staff have ways of coping when transport breaks down or is not available, so that CDTI activities are not disrupted. There is no transport nor provisions to acquire it. ls this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all a Not licable FindingsCharacteristics of the indicator Trips made for CDTI purposes should be properly authorised in writing by the relevant officia!. Each trip undertaken should be recorded in a log book. Transport provided for CDTI, and that provided for other programmes, should be combined as a pool to be used for legitimate activities of a!! proqrammes at this level. Your overalljudgement: is this indicator for sustainability being achieved?I Fu!!y Highly Moderately Slightly Not at all a Not licable 55 Characteristics of the indicator Findings Management should know that replacements will be needed before the end of the programme, and have specific, realistic plans to meet the need at that time. It should be planned that the government will: * Provide replacements for vehicles. * Maintain existing vehicles. * Provide stationery and materials for training and HSAM. lf it is planned that replacement will be from nongovernment sour@s, written commitment for this should have been obtained at the highest level in these donor organisations. * !s this indicator for sustainability being achieved?I Fu!!y Highly Moderately Slightly Not at all Not applicable 8. lndicators of resources: human resources 8.1 Check whether the team at this level is skilled and knowledgeable, regarding the of CDTI in its area of operation This indicaforassesses whether the programme has been able to develop sufficient resources for itself. Descibe the present situation: . Particulars of current staff Characteristics of the indicator Findings Staff should have enough knowledge and skill to undertake a!! the key CDTI activities themselves: planning, training, HSAM, ensuring Mectizan supply, monitoring/ supervision. Staff at this level should remain in one post for at Ieast five years. There should be immediate training (in CDTI) of new, unskilled project staff members who have CDTI responsibilities. I The staff are we!! knowledgeable to undertake a!! CDTI activities. They are full time employees of government and will remain in thei posts for long periods. New CDDs are trained before distribution. * rla {. Find Area of skill No. of skilled persons Level of skill: is it adequate to perform the job? Planning 2 Clinical officer, assisfanf clinical officer Training and HSAM 2 Clinical officer, assisfanf clinical officer Monitoing/ superuision 2 Clinical officer, assisfanf clinical officer Mectizan ordeing/ distibution 2 Clinical officer, assisfanf clinical officer 56 , !nformation about stability and in-service training: Sfaffhas adequate knowledge and skills to undeftake CDTI activities. Staff at this level are stable and have been at their posfs for at least 3 years ls this indicator for sustainability being achieved?I Not at all Not applicable Hrghly Moderately SlightlyFully 9.1 Check whether the hical in the FLHF area is satisfacto 9. lndicators of i ct: covera This indicator assesses whether the project iseffecfive - if the rate is poor the project is clearly struggling, and /ess sustainable. lnstrument 4: community level lndicators of activities and : planning and manaqement 1.1 Check whetherCDDs are ni and their CDT!work This indicafor assesses whether the programme is efficient and simple. The more streamlined and time-efficient the iob, the higherifs susfatnability. Characteristics of the indicator Findings I All villages identified by the latest REMO should be under treatment (i.e. geographica! coverage should be maintained at {00%). The rate should be stable or increasing. It is difficult to calculate the geographic coverage for the project because the project is in its * year and more communities are being included. Presently all identified communities in the areas are l OOo/o covered. The year before: I I Your overall iudgement: is this indicator of sustainability be!!g achieved? Fully (100% of communities are doing cDTl) Highly (95-99% of communities - stable or increasing) Moderately (90- 94o/o of communities - stable or increasing) Slightly (8549% of communities - stable or increasing) Negligibly (<85% of communitie s) 57 i Characteristics of the indicator Findings CDDs should plan their work efficiently, e.g. by: . Carrying out census and distribution during the same visit (using this census data for the following year's oder). . Arranging with the community leadership for help with specific problems, such as families who are not willing to participate in the programme. . Qhoosing visiting times and routes which will make the work less burdensome. The CDDS are managing thei r work efficiently, community treatment registers are available, and census is caried out before treatment. Community chairmen assr'sf wifh specific problems especially in areas where families are unwilling to take tablets. We noticed though that the communities' do not choose the time and mode of distribution. I Your overalljudgement: is this indicator for sustainability being achieved? ModeratelyFully Slightly Not at all Not applicable 2. lndicato_rq of astivitles e[{plocesses: leadershipnd ownership 2.1Check whethercommunity leadership is managing problems with the distribution. This indicafor assesses whether the programme is effective, and whether the ls of it. Characteristics of the indicator Findings The community leadership should be taking responsibility for the distribution of Mectizan within the community. lf coverage (geographical and therapeutic) is not adequate or not being maintained, the leadership should understand the reasons for this. Togetherwith the community at large, the leadership should identify and solve problems related to the distribution. ! There were no minutes of communitylcouncil meetings but commun,ty meetings are held before d i stri b uti o n. Co m m u n ity chairmen are fully involved rn assisting CDDs and in some instances the deputy chairs are also CDDs. {. ls this indicator for sustainability being achieved?I ModeratelyFully Slightly Not at all Not applicable 58 tr 2.2Checkwhether the community at large has been involved in taking decisions the distribution cess This indicafor assesses whether the community is taking ownership of the programme. 2.3Check whetherthe community members value and accept longterm annual treatment. This indicafor assesses whether the ,'s own of the Characteristics of the indicator Findings The community should have taken responsibility for decisions such as: . The selection/ changing of CDDs. . The timing and mode of distribution. The community at large are not involved in choosing the time and mode of distribution. ln most of the communities CDD selection was done by then communities. The district and FLHFL personnel determine the time and mode of distribution. {. I ls this indicator for sustainability being achieved? Fully Highly Slightly Not at all Not applicable FindingsCharacteristics of the indicator ln all communities visited, people appreciate the program and the drug and are ready to take the drug as long as necessary. There were cases of refusals after the first treatment round. This was due to mild reactions but after heath education and future sensrTrsafio n and mobilisation such refusals have now resumed taking the tablets. Community members should be able to mention one or more advantages of taking Mectizan. Community members should express the need for annual treatment with Mectizan. People should show understanding of the need for, and express interest inlong ferm treatment with Mectizan. I !s this indicator for sustainability being achieved?! Not at al! Not applicable Slightlyffi; ModeratelyFully 59 3. lndic_ntgrs of q_clivltieq_a4dplgqesses: monitoring 3.1Check whetherCDDs are reporting appropriately to the FLHF Ievel This indicafor assesses whether the programme is effective. lf such reporting is not taking place Mectizan supply will be compromised, which is bad for sustainability. 4. lndicators of agtiviti_es_and processes: obtaining and managing Mec!i74n 4.1Check whetherthe right amount of Mectizan is received. This indicafor assesses whether the programme is effective. lf the right amounts are received it willfoster community ownership, which is good for sustainability. Characteristics of the indicator Findings Reports to the FLHF !eve! should get there on time. Reports may be summary reports, or the original community distribution record, depending on the level of skill of the CDD. Adequate transport should have been arranged for distribution records/ reports to be handed to the appropriate person. The FLHF worker does monitoing duing the census and the distribution peiods. Comm unity registers were examined and found to be fairly completed. Update of registers was found poor in sorne cases while no summary sheets were available at the community level. I ls this indicator for sustainability being achieved? Fully Highly Slightly Not at a!! Not applicable Characteristics of the indicator Findings All community members who were eligible for treatment got it, and some Mectizan was left over for absentees and those who were temporarily non eligible. There should be a rational explanation about how the amount ordered for the community is calculated (on the basis of population). Treatment is on going in all the communities' visited. Majority of the community members eligible for treatment received the drugs. However, the period given to CDDs to keep the drug at community level was found to be too short for absentee communt$ members to receive it on their retum to communities. The quantity of drugs released to each community was adequate. rl. ls this indicator for sustainabil ity being achieved?I Fu!!y Moderately Slightly Not at all Not applicable 60 4.2Check whetherthe CDDs or community members themselves fetch the yearly su of Mectizan Characteristics of the indicator Findings I The CDDs or community members fetch the Mectizan they need every year, from a designated and mutually acceptable place. Adequate transport should have been arranged for Mectizan to be collected from such a place. ln the case of remote communities, the districU LGA, in collaboration with FLHFs, should ensure that supplies reach such groups. The CDDs are responsible for fetching mectizan tablets from the FLHF. ln all cases though communities are informed immediately the tablet arrives at the FLH facilities. I ls this indicator for sustainability being achieved? Fu!!y Moderately Slightly Not at all Not applicable lE; lndicators of activities and processes: x-snn 5.1Check whetherCDDs and community authorities continue to be ogaged in HSAM of other commun members. This indicafor assesses whether the proiect is effective, and whether the commun@ has taken o of it. FindingsCharacteristics of the indicator Both the CDDs and community leaders engage in health education sensifisafio n and mobilisation. Sfeps are taken to provide necessary information and to promote acceptance. ln some communities the local leaders have gone ahead to introduce by-laws to promote acceptance. CDDs/ community authorities identify situations where community members require information. CDDs/ community authorities take necessary steps to provide required information; encourage community members to provide resources; promote acceptance and ownership (meetings, sanctions, community by-laws). is this indicator for sustainability being achieved?t Not at all Not applicable SlightlyModeratelyFully 6l 16. ln{icalors of regoulces : financing 6.1Check whetherthe community has made arrangements to fund local costs of distribution. This indicafor assesses whether the project can mobilise the resources it needs, and fosfers co m m u n ity own ers h i p. 7. lndicatorC oi iesouices: human resources 7.1 Check whetherthere is a satisfactory ratio of GDDs to households This indicaforassesses whether the project can mobilise the resources it needs, and whether the communtty has taken ownership of the programme. Characteristics of the indicator Findings The community should support individuals who are providing CDT! services for them. The community should make provision for the supply of record books, pencils, transport and other expenses incurred during CDTI. Some of the communities were motivating their CDDs through initiatives like exemption from community seruice, cooking for them lunch during training while sorne villages had at one time contibuted 500 shillings per households. However, it was also noted that these initiatives had not been replicated in allthe communities. Fully Highly Slightly Not at all Not applicable Characteristics of the indicator Findings A ratio of at least one CDD to 20 households (or 2 CDDs per 250 population) is recommended. The households for which CDDs are responsible should be close to their own homes. There were vaiations in the ratio of CDDs in relation to the households. ln some villages the number of households to be serued by a single CDD was more than 20 households while in some insfances it was /ess. /f was however noted that even where the number of households to be serued by each CDD was more than 20, it was not burdensome srnce community members could loan bicycles fo CDDs fo leach out the distant families. I ls this indicator for sustainability being achieved? Fully Highly Slightly Not at all Not applicable 7.2Gheck whether all CDDs have received a traini This indicafor assesses whether the project is effective, 62 Characteristics of the indicator Findings I CDDs should be skilled at their work: doing the census; giving the right dose; knowing who is not eligible; knowing what to do with side-effects There should be a plan in place for training CDDs to replace those who drop out, or when new ones are elected for other reasons. The CDDs are skilful at their work. The right doses are given, eligibili$ citeria is followed and the right advice is given for minor symptoms. Targeted training for new CDDs to replace fhose who drop out is undeftaken at the FLHF. However some need refresher training in data management. t Your overalljudgement: is this indicator for sustainability being achieved? Fully Moderately Slightly Not at a!! Not applicable 7.3Check whetherCDDs are willin to continue their work in CDTI This indicafor assesses whether the programme has mobilised the resources it needs, and whether it fosters 8. lndicators of impact: coverage FindingsCharacteristics of the indicator Allthe CDDs interuiewed were willing to continue with their work. There were also a large number of female CDDs. This was auguring wellwith the cultural norms of the communities where by a male CDD cannot ask a female community member if she is pregnant. CDDs should express willingness to continue with distribution in the long term, given the conditions which prevai! in the community. Few CDDs in this community have dropped out from the distribution work. Not applicable ModeratelyFully Slightly Not at a!! 8.1 Check whether the hical cove in the commun is satisfacto This indicator assesses whether the project iseffecfive - if the rate is poor the project is clearly struggling, and /ess sustainable. Characteristics of the indicator Findings A!! households and areas in the community are being treated. ,f Treatment is ongoing 63 l This includes the hamlets for which the community has agreed to be responsible. !f this geographical coverage is not 100%, it should be lmproving. Geographical coverage was 80.02% and 80.09% in 2001 and 2002 respectively. ln 2003 fhis is expected to reach its ultimate treatment goal. * ! Your overalljudgement is this indicator of sustainability being achieved? Highly (only nomads in the surrounding area were missed Moderately (the outlying hamlets were also missed) Slightly (some wards of the community were also missed) Negligibly (no- one got treated, or only a few families) 8.2 Check whetherthe commun has a satisfactory therapeutic cove e rate This indicafor assesses whether the programme is effective - if the rate is poor the project is clearly struggling, and /ess sustainable. PROGRA}4ME FOR THE SUSTAINABILITY EVALUATION OF THE TANGA REGION CDTI PROJECT MARCH 7TH T0 MARCH 21sT2003 Characteristics of the indicator Findings t The community overall has a therapeutic coverage rate of 65% or higher. This rate should be stable or increasing. * Distribution is ongoing * However, there was a decline in therapeutic coverage from 74.4%o in 2001 to 69% in 2002 treatment cycles t Your overalljudgement: is this indicator of sustainability being achieved? Fully (the community has a therapeutic coverage rate 2650/o - stable or increasing) Negligibly (the therapeutic coverage rate is <65%) DATE ACflVIIY RESPONSIBLE Saturday 08.03.2003 1. Arrival of Evaluator (Team Leader) in Dar es Salaam. 2. Meeting with Members of the Team Team Leader Sunday 09.03.2003 No activity. Public 0ffices are not open Monday 10.03.2003 1. Briefing of the WR - Tanzania 2. Resolve Administrative lssues at World Health Organization ffice, Tanzania. 3. Meeting with the Deputy National Coordinator 4. Departfor Tanga by Road AllTeam members Team Leader AllTeam Members Tuesday 1 1.03.2003 1. Meeting with the Tangaflegional Medical Officer. 2. Visit to the Tanga Regional Commissioner/RegionalAdministrative Secretary. 64 3. Meeting with the Project Coordinator, NGDO Representative and Depuff National Coordinator to select the Evaluation sites. 4, Formation of sub teams - 1&2. 5. Departure to evaluation sites by sub team 1&2 (lnterview of DMo/DoT) 6. lnterview the Project Coordinator and NGDO Representative. Team Leader AllTeam Members Wednesday 12.03.2003 Visit to Evaluation sites- First Line Health Facilities Teaml&2 Thursday 13.03.2003 I Visit to Evaluation sites -First Line Health Facilities and some villages Teaml&2 Friday 14.03.2003 Visit to remaining evaluation sites, villages Teaml&2 Saturday 15.03.2003 . Data Analysis , Draft Report Writing started AllTeam Members Sunday 16,03.2003 Report Writing continue AllTeam Members Monday 17.03.2003 ' Debriefing of RHMT . Planning Workshop for Project Level Personnel . Review of Planning Workshop r Preparation for District Level planning meeting Team Leader AllTeam Members AllTeam Members All Team Members AllTeam Members Tuesday 18.03.2003 Planning Workshop for District Level Personnel Review Workshop AllTeam Members AllTeam Members Wednesday 19.03.2003 . Preparation of planning workshop meeting report ' Tentative work plans for years 2003 & 2004 . Final Data Analysis and Report Review AllTeam Members AllTeam Members AllTeam Members Thursday 20.03.2003 . Depart for Dar es Salaam . Debriefing meeting with WR Tanzania AllTeam Members Friday 21.03.2003 . Courtesy call on Ministry of Health Tanzania . Departure AllTeam Members List of documents sighted Project Level o Project CDTI Plan of Action for year 2001,2002,2003 o REMO list of endemic communities o CDTI budget 2001,2002,2003 o Financial report to APOC o Cash book APOC funds o First and second year technical reports o Supervisory visit reports o Mectizan stock register o Mectizan application forms o Training reports for 2001, 2002,2003 o Trainng and health education materials 65 . CDD manual o RHW training manual o CDTI treatment with imerctin guide for training of CDDs o Regional training guide for onchocerciasis District level o Treatment summary records, District First Line Health facility and community integrated workplan for CDTI activities 2001,2002 o Annual budget estimates Health Facility Leve! o Treatment summary sheets 2001,2002 o Mectizan inventory 2001, 2002,2003 o List of CDDs trained at community level Community Level o Treatment Registers 66 LIST OF PERSONS INTERVIWED Regional Level l. Dr. Maggid - .2. Ms Harriet Lyimo District Level Muheza District l. Dr. Shayo 2. Mr. John Kilimhana Korogwe District l. Mr. Francis Assenga 2. Mr. Herman J Magembe Lushoto District l. Mr. Richard A Mbughuni 2. Mr.JPOWatae 3. DrSG Mgema 4. Mr Abdul A Daffa FLHF Amani Misozrve Zirai Kwatango Community Kwelumbizi Mbomole Kizerui Mashewa Kwatango Misozwe Mkwajuni Lushoto FLHF Mbelei Regional Project coordinator HKI representative DMO DOT Ag. DMO DOT Council Chairman Planning offrcer DMO DOT Symphrosa Ritte Partricia Msofe Lema Philemon Mmbanga Clinical Officer Clinical Offrcer Clinical Assiatant Ass. Clinical offrcer George Francis Monica Shafi Hadija Musa Anna Dereza Nuhu Akida Andrias Makowe Mariamu Mwwhehe CDD CDD CDD CDD CDD CDD CDD Mayo Community Hassan Sangoda Omari Salim Beatrice Msumai Rajab Moshuza Mbwana Jumaa Rehema Kimwesi Godfrey Hiza CDD/ Village Chairman CDD CDD CDD 67 Frederick Chigazi Hassan Kirima Musa Kwijanera Fadhila Saidi Korogwe FLHF Community Abdi Rashide Mariam Saidi Ali Pali Zainabu Shabani Nuru Salihu Musa Hamisi CDD CDD CDD CDD Ibrahim Lupatu Village Chairman CDD CDD CDD CDD CDD 68 PI.ANNING/ FEEDBACK MEETING FOR PROJECT AND DISTRICT LEVEL PERSONS MARCH 17th/ 18th 2003 AGENDA s/N Activity Time Facilitator 1 Opening remarks 1000 _ 1010 Deputy National Coordinator 3. General introduction to the FeedbacUWorkshop programme and Evaluation objectives, guidelines for sustainability 1020 - 10s0 Team Leader James Mugeshi 4. Methodology and Limitations 10s0 _ 1 100 Mrs. Maduka 5. Summary of findings . Community Level . Health Facility Level . District Level 1 1 1 00_ 1110 10 - 1120 20 _ 1130 Mr. Pallangyo G Ms. Maria N. Mr. Malley K. D, 6. Discussions 1130 _ 1200 Team Leader 7 Summary of findings . Proiect Level 1200 _ 1215 Mrs. Maduka 8. Discussions 1215_ 1245 Team Members o BREAK 124s- 1300 AllTeam members 10 Brainstorm on solutions to problems identifiedi Groups 1. Planning, Monitoring, Supervision and Mectizan Distribution 2. Finance, Training, Mobilisation and Transport. 'Each group appoints a rappoteu" 1300 _ 1345 Dr J, Y. Jiya 11, Report of Group work & Discussion 1345 1415 Mr. Pallangyo G. 12. LUNCH 1445 - 1530 AllTeam members 13. Planning for selFsustainability at Project Level. Groups to find answers to:- . Action between now and the end of year . Programme for CDTI in year 2004 , Resources for CDTI in year 2004 1530 _ 1630 Team Leader 14. Plenary Session and Discussion 1630 _ 1700 Mr. Malley K. D. 15. Matters Arising 1700 _ 1715 16. Concluding Remarks by Project Coordinator and NGDO Representative 1715 _ 1730 Project Coordinator NGDO Rep. 17 Conclusion/ Closing 1730 All Participants 69 Minutes of the Sustainability Plan and Feedback Workshop Tanga, Bandari House March 17-18th,2003 Day 1, March llh Gathering of the delegates, l0:45 a.m. Introduction of Participants Tea Resume I l:30 a.m. Mr O.Kaitaba Gave a brief introductory remarks concerning sustainability of the Tanga Project o Introduction of APOC five year period, phasing out, transition to local authority Attendance Register for Tanga Feedback Workshop 17th March 2003 Name Title Address Msekeni R. Luchele Planning Offrcer Tanga Octavian D. Mhagama Planning Offrcer Korogwe John Kilimuhana DOT Muheza Francis Assenga Actins DMO Korogwe Herman J. Magembe DOT Korogwe Dr. W.O. Shayo DMO Muheza Daniel T. Lusingu Planning Officer Muheza George E. Mhada Council Chairman Muheza fuchard A. Mbuehuni Council Chairman Lushoto J.P.O. Watae Planning Officer Lushoto S.G. Mgema DMO Lushoto Abdul A. Daffa DOT Lushoto Charles Frarzdn IMA Rep.A.{GDO Coalition Chairman Dar es Salaam James Mugisha Evaluation Team Member Uganda MariaNakabiito Evaluation Team Member Uganda Oscar Kaitaba Deputy NOCP Coordinator Dar es Salaam Dr. Jonathan Jiya Evaluation Team Leader Nieeria Chinyere U. Maduka Evaluation Team Member Nigeria Karoli D. Malley Evaluation Team Member Tukuyu George Pallangyo Evaluation Team Member Songea Dr. R.B. Maggid Project Coordinator Tanga Harrieth Hamisi HKI Proiect Officer Tanga D.A. Mwalyambwile Acting RMO Tanga Dr. Simon Katenga NOCP Coordinator Dar es Salaam 70 Muheza District Council Chairman was elected as the chairman of this workshop and he is installed by acclamation Remarks by Dr Jiya - discussion of the change in plan . Need to go to Lushoto either on Tuesday or Wednesday for a day or two . Brief introduction of ownership of the oncho projects at the village/community level . What does sustainability mean? Keep programm going and making sure it continues to work o Self-sustainability means that resources can still be generated to keep the program going even though they are not all from local sources o If Helen Keller is there continuing to help, this is quite all right o Simple plans and simple designs required o lntegration is necessary so as not to cause great waste o Affitude of the persons is extremely important- what is their attitude and what is there desire? o Programmes are more sustainable if they have resources - committed people and materials and funds o Community ownership extremely important, almost more impotant than anything else - the people who are committed will push the health officers hard to make sure their programme moves and medicine is made available Mr James Mugisha - Gave a brief on the Indicators of sustainability o planning o Coverage o Mectizan supply -procured within the government structure o Training and Mobilization - targeted training at all levels, is it cost effective? o How is the project financed - entirely funded by APOC or others and not the government? Sustainable projects should be able to run on their own o Transport and Materials - if it is there, is it well-maintained, are there checks and controls, are there logbooks, are people wellfacilitated, who provides the transport, is it the government, APOC? o Supervision - is it over or under-supervised? Is it integrated and efficient? Is it targeted or is it routine? o Human Resources - are there enough people, are they skilled enough? What about motivation, are they motivated enough? Are they stable- here today and gone tomorrow? o Leadership - are we expanding the leadership beyond the project? Are the government structures involved in the project? Are the politicians involved? How? What do they do and will they continue to do it? Mrs C. Maduka - briefed the participants on how the instruments were applied and what precisely they were, she also described the Methodology and limitations. o John the Baptist sent by APOC to prepare the ground for the Tanga evaluation- this was the Deputy National Coordinator o Sample villages/communities were selected - high and low coverage in a number of areas 7l o Main criteria was coverage - looked at two types of coverage (geographic and therapeutic), plus endemicity, areas that were near and far from urban o Korogwe had the highest coverage, followed by Lushoto and Muheza (74%o,73%o and 72%) o Tomorrow will go to continue in Lushoto District with the evaluation o In Muheza visited four first line health facilities; Korogwe visited five villages and Muheza eight villages o FOUR INSTRUMENTS: regional, district, health facility and community o Interview important people in each of these four areas o Much data was collected, well-documented, analysis continues, and Lushoto is still to be visited tomorrow . Fully (4), highly (3), halfivay tulfilled (2), slightly marginal (l), not tulfilled at all (0) o At the end of the exercise to grade the entire project of Tanga to give an idea about the overall sustainability future of the Tanga Project o What needs to be done to improve towards sustainability? Guidelines for the next two years to improve o Findings to be shown today r Expectations are that at the end of the workshop a plan will be worked out to move towards a higher degree of sustainability in all three districts and for the project as a whole o Report will be written and circulated to all participants o Limitations: not all documents were provided; it was not possible to go directly to the field due to problems in Dar es Salaam; could not see everyone needed (RMO and the RC); but John the Baptist has done a wonderful job in providing what was requested by the team Acting Regional Medical Officer arrives and is introduced at 12 noon Questions: o Indicators are not clear to the DPLO of Lushoto- revisit the nine indicators o The limitations stated are not real limitations and a correction should be made but it is true that some documents are not available- there is no attempt at castigation Mr G. Pallangyo briefly narrated the evaluators findings at the Community level. Community is involved Leadership are involved with community selection CDDs were supervised by health facility people but no transport provided Mectizan supply - tablets are collected from health facilities and eligible people are treated house-to-house - transport was not provided, CDDs loaned bicycles, some tablets are reserved for absent people, but time of distribution is too short Assistance was requested from health facility people in education and training Financing - different methods for motivating CDDs, loan bicycles, given sticks, some money was given here-andthere (500/:), some CDDs were exempted from community services, some food was given Two CDDs per subvillage (l:200) a a a a a a a 72 o CDDs knew a lot about the programme but there were some deficiencies in record keeping o One female in every two CDDs o Therapeutic coverage was 74%o and 69%o o GeoglaPhic coverage was 80%o and 8l%o Dr. Simon Katenga arrives and is introduced at l2:15 p.m. Ms Marie Nakabiito briefed the meeting on the findings at the first line health facility level o No written plans were found o [n terms of leadership, the chairman was involved directly o [n terms of supervision, the community reports to the health facility who supervises the CDDs, the staff at the health facility do supervision in an integrated manner- there is an integrated approach o Mectizan procurement - FLHF orders from the District onchocerciasis team, order forms were available and adequate supplies were delivered - however, communities are not involved in planning when they will take the drug- this is decided for them o CDDs pick up drugs from the healthfacility o Training is done for CDDs at the health facilities- but training is not prioritized and the same routine is done every year- not dependent on what was found- the training was too long and not cost-effective o Refusals were not dealt with properlyas they should have o No resources and transport and no evidence of health materials o In terms of human resources, adequate staff were available but proper planning was not done with no written plans o Staffdid a good job but needed extra motivation o Some of ttre records were not clear on therapeutic coverage Mr K.D.Malley -gave a brief on findings at the two districts visited in Korogwe and Muheza (Lushoto will be seen in a few days) o Planning - oncho plans are integrated into the district health plans but do not take into account some issues such as time of distribution o Leadership - fully responsible for CDTI activities o District Oncho Teams do well but go beyond their scope by supervising CDDs o APOC funds are the main source of funds o Mectizan supply was always on time except for the current year o Training - district oncho teams trained health facility personnel, it is an effrcient and cost effective method - even go so far as to train the CDDs which is not very cost-effective- better to let the health facility people do this- central training of CDDs is well done but no integration with other programs. o Advocacy done before census and distribution o Funds appeared to be allocated in Muhezz and Korogwe from the basket funds but not to a very high degree (short of TSh2 million) 73 o Each district has two motorcycles provided by APOC but IEC materials were not seerF there are no logbooks or transport authorization at the district level, used for other health activities o Knowledgeable staffwho can do training and supervision at all levels o Committed staff with salaries paid by the government regularly o Coverage dropped in the 2nd year because of refusals Discussion o Why is the geographical coverage only 80%? Should have been 100% from the beginning - Frarzdn asks, after much debate, that the team should clariff 100% village coverage as different from 80-81% sub-village coverage- team leader agrees that this should be done o What does it mean that DOTs are going too far down in the system, are they bypassing other channels? o This issue cleared up by one DOT about his experiences o How can IEC materials not be available at the community level- where are they? Also, one DMO brings up the point about usefulness of IEC materials - have they been tested in all areas? Mrs C Maduka-gave a summary of findings at the Regional level Drop in therapeutic coverage (74-69%) due to people refusing due to sidoeffects; increased mobilization and education required No programme plan at the Regional level, absolutely necessary to have the government and the politicians involved Mectizan arrives on time but a bit of delay at the port this time around- national level should do 'whatever it takes' to ensure that this does not happen again Training and mobilization should be done by each at his level, not higher level people going to the lowest levels- people at the higher level should empower those at the lowest level so that activities can be carried out cost effectively No budget at the Regional level for any of the three districts- but at the district level there is basket funding which should be encouraged much more APOC has provided transport, one car at the regional level and two motorcycles to each district Very good staff involvement and skills at the regional level No regional onchocerciasis task force RHMT very active with Project Coordinator a member There is a very good chance of integration at the regional and district level a a o a a a a a a a Questions and Comments: . Everything is done in Tawania at the district level, including all planning, not atthe regional level which is reserved for training and supervision - basket funding is government money - Dr. Katenga cannot budget money to be given by the region for onchocerciasis programme 74 GROUPWORK Brainstorming Sessions - 1:3G2:15 p.m. o Planning, Monitoring and Supervision- Muheza o Coverage, Mectizan Distribution and Finance- Lushoto o Training, Mobilisation and Transport- Korogwe PRESENTATIONS Presentation of District Solutions to Various Problems & Shortcomings Identified Korogwe District o Transportation as the CDDs have no bicycles: solution is that the community should be encouraged to buy bicycles for the CDDs o Training not targeted to the community problems: solution is that hainings should be done according to weaknesses which arise in the communities o Mobilization is not always targeted to the needs of the communities: solution is that mobilization should be based to the community needs Lushoto District o Delay of Mectizan Distribution: solution is to have a three month backstock should always be available at region o Lack of financial motivation to the CDDs: solution is that CDDs should be organized for income-generating groups o Low coverage compared to that of the l't year: solution is that more community sensitization required on side effects and target training on reactions Muheza District o Not involved in decision-making in regard to distribution period: solution is to involve the village health committee, the village govemment and the village assembly o No plans at the higher level: solution is to educate them on how to make their own plans o Community requirements are not taken into consideration: solution is to liase with community to know their plans and to incorporate them into district plans o No provision of transport to CDDs at community level and front line health staff: solution is for local govemment to prepare budget for buying bikes, donkeys and motorcycles to health facilities o Monitoring and supervision goes beyond the scope of the health facilities: solution is that supervision and monitoring to be done to the level of the health facility, follow up specific problems at community level and integrated supervision Additional Comments Concerning the Presentations: o Increase the number of CDDs to lower the ratio from I :200 to I : l50or I :100 Lunch 3-4 p.m. 75