Evaluation of the sustainability of the Jigawa CDTI project, Nigeria : November 2004

World Health Organisation African Programme for Onchocerciasis Control Evaluation of the Sustainability of the Jigawa CDTI project, Nigeria November 2004 Evaluation Team Members: Oladele O. Kale (Team Leader) Oyabure A. Abu Cyrille T. Evini Nicholas N. Ogweng Mrs. Esther T. Oyeyi Yisa A. Saka TABLE OF CONTENTS Acronyms ............................................................................................................................ 3 Acknowledgements ............................................................................................................ 4 Executive Summary ............................................................................................................ 5 1. INTRODUCTION .............................................................................................................. 8 2. METHODOLOGY ............................................................................................................. 8 2.1 SAMPLING ..................................................................................................................... 8 2.2 SOURCES OF INFORMATION ...................................................................................... 9 2.3 ANALYSIS ...................................................................................................................... 9 2.4 Team composition ................................................................................................. 10 1. Prof. Oladele O. Kale (Team Leader) ...................................................................................... 10 2. Mr. A. O. Abu ................................................................................................................................. 10 3. Mr. Cyrille Evini ............................................................................................................................ 10 4. Dr. Nicholas N. Ogweng ............................................................................................................. 10 5. Dr. Esther Tinuade Oyeyi .......................................................................................................... 10 6. Dr. Yisa A. Saka............................................................................................................................ 11 2.5 Advocacy Visits and ‘Feedback/Planning’ Workshop ........................................ 11 2.6 Limitations ............................................................................................................. 11 3. MAJOR FINDINGS, DISCUSSIONS AND RECOMMENDATIONS........................... 12 3.1 State Level ............................................................................................................. 12 3.2 LGA Level .............................................................................................................. 17 3.3 Front Line Health Facility Level ............................................................................ 22 3.4 Community Level .................................................................................................. 27 3. CONCLUSIONS ........................................................................................................ 31 4.1 Grading the Overall Sustainability of the Jigawa CDTI Project ............................... 31 4.2 Grading of project as a whole .................................................................................... 34 ANNEXES .......................................................................................................................... 35 Annex 1: INTERVIEWS ...................................................................................................... 35 Annex 2: Schedule for the Evaluation, advocacy ........................................................... 39 Annex 3 : STATE & LGA LEVELS WORKSHOP PROGRAMME ..................................... 41 Annex 4 : REPORT ON FEEDBACK/PLANNING WORKSHOP FOR JIGAWA STATE CDTI PROJECT ....................................................................................................................... 44 Annex 7: List of Participants ............................................................................................ 46 3Acronyms APOC African Programme for Onchocerciasis Control ATO Annual Treatment Objectives CDD Community-Directed Distributor (of Ivermectin) CDTI Community-Directed Treatment with Ivermectin CBM Christofel Blinden Mission CSM Community Self-Monitoring FLHF Front Line Health Facility GCR Geographic Coverage Rate H/C Health Centre HMIS Health Management Information System H/P Health Post H/Q Headquarters HSAM Health Education, Sensitisation, Advocacy and Mobilization IEC Information, Education and Communication LOCT Local Government Onchocerciasis Control Team member MoH Ministry of Health NGDO Non-Governmental Development Organization NGO Non-Governmental Organization NID National Immunization Day NOTF National Onchocerciasis Task Force NPI National Programme on Immunization PECP Primary Eye Care Programme PHC Primary Health Care PHCDA Primary Health Care Development Agency REMO Rapid Epidemiological Mapping of Onchocerciasis SOC State Onchocerciasis Coordinator SOCT State Onchocerciasis Control Team TCR Therapeutic Coverage Rate WHO World Health Organization 4Acknowledgements We would like to express our appreciation and gratitude to the following institutions and persons for their contributions in various ways to the successful completion of the assignment.  Dr A. Sékétéli, Director, Dr U. Amazigo, Chief of the Sustainable Drug Distribution Unit and other staff at the Headquarters of the African Programme for Onchocerciasis Control (APOC) in Ouagadougou, Burkina Faso.  The WHO Country Representative and his staff, Nigeria  The National Coordinator for Onchocerciasis Control Programme, Nigeria  The Country Representative of the Christofel Blindness Mission (CBM), Nigeria  The officials and staff of government, at State, LGA and Health institutions, with whom the team interacted in Jigawa state.  Political and traditional leaders, health workers and community leaders and members of all areas visited in Birniwa, Gwaram and Kaugama Local Government Areas 5Executive Summary The Jigawa State CDTI project was launched in 1999 and is therefore, in 2004, in its last year of scheduled funding from APOC. At the request of APOC, a team of evaluators, external to the project, from Cameroon, Nigeria and Uganda carried out an evaluation of the sustainability of the project between 22 November and 04 December 2004. The evaluators were assigned three specific tasks:  To evaluate the sustainability of the project.  To present and discuss the results with the State, NGDO and LGA authorities.  To support the State and LGA levels staffs in developing post-APOC sustainability plans, using the guidelines for sustainability planning developed by APOC. Information was gathered from the review of relevant documents and reports. Interviews were held with the State team, LGA political, administrative and health officials, FLHF staff, community leaders and members as well as CDDs. The evaluation team also held community meetings in all the communities sampled. Field observations at the four levels provided additional source of information. Findings Table 1 provides a summarised overview of the evaluation team’s score for the different standard APOC-approved groups of indicators at the four levels of project operations – State, LGA, FLHF and Community. The findings of the evaluation team were presented at a feedback and planning meeting held with officials directly involved with the implementation of the CDTI project in Jigawa. This provided an opportunity for both the evaluation team and participants to get a better understanding of project achievements and weaknesses. Table 1, Scores on groups of sustainability indicators at all project levels. Groups of Indicators S TA TE LG A FL H F C om m un ity A ve ra ge Planning 3,20 3,00 2,00 4,00 3,05 Integration - 3,00 3,00 - 3,00 Leadership and ownership 4,00 3,50 2,00 3,67 3,29 Monitoring and supervision 3,50 3,00 3,30 2,00 2,95 Obtaining & managing Mectizan 1,00 2,50 3,00 1,50 2,00 Training 2,50 2,50 2,00 - 2,33 HSAM 2,50 3,00 3,00 3,00 2,88 Financing 2,22 3,00 1,00 3,00 2,31 Transport resources 3,00 2,50 0,66 - 2,05 Human resources 3,50 3,50 2,00 3,33 3,08 Coverage 4,00 4,00 4,00 - 4,00 Average 2,94 3,05 2,36 2,93 2,80 6Coverage (geographical 100%, and therapeutic well over 65% in all LGAs), an output indicator was the most satisfactory area of the project, indicating a situation, which fully supports sustainability. While in general, the evaluation team was impressed by the good performances of the project with regards to this particular indicator, it noted that the project still uses “eligible population” rather than total population as the denominator when calculating the TCR. The team strongly admonished the officials on this and urged them to conform to the standard APOC formula. It is noted, with satisfaction, that the TCR is still consistently above 65% when the correct denominator is used. The team also recommended that project officials at all levels ensure that all households be visited during distribution campaign to ascertain that all eligible populations are being effectively treated. Mectizan® collection, storage, delivery to lower levels and control is the weakest part of the project. Although Mectizan® storage and delivery are carried out effectively and in an integrated manner within the government system, the team observed that Mectizan® procurement for the Project is still overwhelmingly dependent, both technically and financially, on the NGDO partner. The team has recommended that for enhancement of the prospects for sustainability, a greater share of the responsibility for this critically important and sensitive indicator be borne by the State apparatus. Although Mectizan® is generally available on time and in adequate amounts there was however a delay in its delivery during the last round of treatment. There were also reports of shortages in some villages. A major weakness found by the team was that it was difficult to track down the amounts of Mectizan® received at and delivered to lower project levels, especially starting at LGA level. Planning Planning of CDTI activities is being carried out effectively at State, LGA and Community levels. Although activities are reasonably effectively being implemented at the FLHF, little or no effective planning is being done at this level. Communities are effectively involved in the planning process and all of those visited knew when and how activities were to be carried out. Planning is integrated in the overall health plans, it however appeared that specific planning for sustainability is not being effectively undertaken, although it was said to have been thought of. Leadership and ownership State, LGA, FLHF and communities provide strong leadership for CDTI. There are sustainable focal points for CDTI at all levels including in some communities visited. Community leaders play leadership roles in CDTI; they select CDDs and change them as they deem necessary, and mobilize communities for CDTI. All individuals interviewed expressed their wish to take Mectizan® for as long as it takes to get rid of the disease. Many advantages for taking Mectizan® were mentioned including its perception as a ‘wonder’ drug that improves virility and sexual performance. Integration The present level of integration of CDTI activities is very satisfactory. CDTI activities are being carried out in an integrated manner at all the levels visited. But for the lack of written evidence with regards to this indicator, the project would have been scored higher. Human resources Staff at all levels have been remarkably stable and are highly motivated. They all expressed their willingness and dedication to continue with their CDTI activities. Some of them even remarked that they considered it as a moral obligation to serve the people who selected them. However, it was observed by the team, that the number of staff at FLHF was 7generally not adequate. With regards to CDDs, the ratio of two CDDs for 250 persons, as recommended by APOC, is generally not being adhered to. The norm in Jigawa is two CDDs per village irrespective of the population size or type of settlement. Drop out from the ranks of CDDs is rare and when it occurred, in the past, CDDs were promptly replaced. There is need for Project officials to set up a reward system for deserving staff at all levels. Monitoring and supervision is carried out mostly in the month of Mectizan® distribution. These supervisions mainly target the immediate lower levels. There were no reports of excessive routine supervision visits. Supervision checklists exist at State and LGA levels, but these are not being systematically used. Supervision at each of the levels has for most of the time been integrated with other programmes. Reports on CDTI activities are communicated within the HMIS. Project sustainability performances for other indicators were rated as moderate. Training and HSAM is carried out well. Staff who were interviewed (at all levels) have adequate skills, value their work and expressed strong willingness to continue serving in their present position. However there were instances of inadequate HSAM materials, and training for staff at all levels has not always been targeted. Budgets, in general, were not seen by members of the project team. It was claimed that at the State and LGA levels, budgets contained all CDTI key activities with estimates clearly spelt out. Budgetary contributions of partners were also said to be clearly spelt out. The evaluation team however did not see any written evidence to substantiate these or in support of claims on cost-containment and money release. Officials at the State level expressed satisfaction with and were of the opinion that funds received from the government were adequate for CDTI activities. It was announced, during the debriefing session that the CDTI budget for 2005 had been approved and that there is a need for the State Onchocerciasis Coordinator (SOC) and NOTF to follow-up on that issue so as to ensure timely release of the said funds for effective implementation of CDTI With regard to transport, vehicles at all levels are operated in a pool. Funds for maintenance are also provided from pooled resources. Specific CDTI transport means are old and will require replacement. Though management was aware of the necessity to replace existing transport means, there was no realistic plan, at the time of the team’s visit, to replace the vehicles. Control of vehicles was considered adequate although no written evidence was provided. There is no means of transport at the FLHF level and there is an urgent need for State officials to solve the problem. In the past staff’s coping mechanisms when motorcycles or vehicle broke down were most of the time effective, and included using other vehicles or motorcycles from the pool where available. In other cases, it involved trekking, borrowing or the use of public transport. In all cases, CDTI activities have apparently not been unduly disrupted on account of lack or breakdown of vehicle. On the overall grading of the project the evaluation team found that: Only one of the seven aspects of sustainability, ‘resources’ was not helping. With regard to ‘critical elements’ it was found out that only one, ‘money’, was not satisfied. In accordance with APOC’s standard Evaluation Guidelines, the team has concluded that the Jigawa State CDTI Project is making satisfactory progress towards sustainability. 81. INTRODUCTION Jigawa State is one of the 36 States in Nigeria. Jigawa State, which is situated in the northern part of Nigeria, was carved out of Kano State in 1991. The State is in the Sudan Savannah and Sahel zones. It is drained by many rivers, notably Hadejia and Chiyako, and their tributaries. The total population of the State, including those living in hypo-endemic onchocerciasis communities, is 2.5 millions. There are 27 Local Government Areas (LGAs). CDTI was introduced into Jigawa in 1999 and targets the eight hyper- and meso-endemic LGAs in the State. There are, according to REMO findings, nine hypoendemic LGAs, whose populations are also receiving ivermectin treatment, though not through CDTI. The State Government and the NGDO partner, CBM, provide funding for onchocerciasis control activities. The Jigawa CDTI Project was subjected to an independent monitoring exercise in 2001. 2. METHODOLOGY  Evaluation question: How sustainable is the Jigawa CDTI project?  Design: Cross-sectional, descriptive.  Population: The Jigawa project, including: The State Onchocerciasis Control Team (SOCT); the NGDO partner; the LGAs with their Local Government Onchocerciasis Control Teams (LOCTs); the Front Line Health Facilities (FLHFs); CDTI project communities and villages, as well as their CDDs, community leaders and community members.  Sampling: Details of the sampled LGAs, FLHFs and communities/villages are shown in Table 2. 2.1 SAMPLING A multi-stage sampling approach was adopted in selecting the sample for the evaluation. First as treatment had been taking place in the communities for the past five years, including 2004, and only the results of the past four rounds of distribution (excluding 2004) were available for use by the evaluation team, the average treatment coverage for the past four years was computed for each of the eight LGAs. Next, the average coverage rates of the eight LGAs were arranged in an ascending order, from lowest to highest. These were then placed into three strata. From each stratum, one LGA was selected. The selection took into consideration treatment coverage and geographical representativeness as some LGAs had been sampled during the Independent Monitoring exercise and therefore did not need to be included in the present evaluation. Through sampling, Gwaram, Kaugama and Birniwa LGAs were selected for the evaluation. From each of the three LGAs, two health facilities were selected at random and, based on their recorded 2003 treatment coverage, one community/village with low coverage and another one with high coverage were selected. 9Table 2: Distribution of Samples in LGAs, FLHFs and Communities: 2.2 SOURCES OF INFORMATION Information was collected from interviews, verbal reports and documents. Various categories of people were interviewed, at the State level, (State Onchocerciasis Control Team members, Permanent Secretary, Director PHC MOH, Project Accountant, CBM Country Representative), at the LGA, (PHC Coordinator, LOCTs, Chairman, Treasurer), at the frontline heath facility/health area level, (the ‘In-charge’) and at the community level, the evaluation team interviewed community chiefs and elders, community members and CDDs, and held community meetings. Information was recorded on the standard evaluation instruments and discussed extensively among members of the team before grading of the performance of the relevant levels on the indicator. 2.3 ANALYSIS Based on the information collected, project performance in respect of each indicator is graded on a scale of 0-4, in terms of its contribution to sustainability. The average 'sustainability score' for each group of indicators is calculated, for each level. A graph is plotted for the level being assessed. The evaluators discussed, among themselves, qualitative description of problems, and made likely suggestions for solving the problems on which there was deliberation. Recommendations were thereafter made, based on the findings of each level evaluated. S/N LGAs Treatment (Coverage Rates) FLHFs Communities/Villages 1 Birniwa Relatively Low (78%) Kubuna 1. Kubuna 2. Kwarkwasa Matarauku 1. Matarauku 2. Digilfani 2. Kaugama Medium (81%) Ubba 1. Ubba 2. Dingare Dakayawa 1. Dakayawa 2. Hadin 3. Gwaram High (84%) Sakuwa 1. Sakuwa 2. Kajuma Yarfi 1. Yarfi 2. Bagadaga 10 2.4 Team composition The core team members were the following: 1. Prof. Oladele O. Kale (Team Leader) ∗ Department of EMSEH, ∗ Faculty of Public Health, ∗ College of Medicine, ∗ University College Hospital, Ibadan, Nigeria ∗ Tel/Fax (Home) +234 02 8100397 T(Mob) +234 080 22912224 ∗ Email: ookale@yahoo.com; ookale@skannet.com 2. Mr. A. O. Abu ∗ State Onchocerciasis Control Coordinator ∗ Edo State CDTI Project ∗ Ministry of Health , Benin City, PMB 1113 ∗ T (Mob) (234) 8023411316 ∗ E-mail: abuduabu2000@yahoo.co.uk 3. Mr. Cyrille Evini ∗ Helen Keller International, Cameroon ∗ P.O. Box 14227, Yaounde, Cameroon ∗ T (mobile): 237 771.07.21; T (office): 237 220.97.71 Email: cevini@hki.org; cyrilleevini@yahoo.fr; 4. Dr. Nicholas N. Ogweng ∗ District Co-ordinator Moyo Phase IV, CDTI project ∗ P.O. Box 1 Moyo ∗ T (Home): +256 77543588 Email: nicholasoge@yahoo.com; nicholasoge@hotmail.com 5. Dr. Esther Tinuade Oyeyi ∗ Department of Biological Sciences ∗ Bayero University, Kano, Nigeria ∗ PMB 3011 ∗ T(Mob) +(234) 8037875716 ∗ Email: estheroyeyi@yahoo.com 11 6. Dr. Yisa A. Saka ∗ Federal Ministry of Health, Federal Secretariat ∗ 9th Floor, Room 956 Ikoyi Lagos ∗ T (Home): +234 018184362; T (office): +234 018910575; T(mob) +234 80033029387 ∗ Email: yisaasaka@yahoo.com The evaluation team members were grouped into three sub-teams for the purposes of fieldwork. Each sub-team had one or two persons that served as guides, facilitators, and translators. 2.5 Advocacy Visits and ‘Feedback/Planning’ Workshop Advocacy visits were paid to relevant people at each level. Debriefing was done at the feedback and planning workshop. During the workshop, the evaluation team presented its findings and guided the Sate and LGA teams in the development of a three-year post-APOC sustainable plan, which took the evaluation findings into consideration. 2.6 Limitations Reports of annual financial expenditures for both POC and MOH, by items, were not available for review by the team, thus the financial efficiency of the project could not be meaningfully assessed. Two FLHF staff were not seen. 12 3. MAJOR FINDINGS, DISCUSSIONS AND RECOMMENDATIONS 3.1 State Level Scores at State Level 3,2 4 3,5 1 2,5 2,5 2,22 3,00 3,5 4 0 0,5 1 1,5 2 2,5 3 3,5 4 Pl an ni ng Le ad er sh ip a nd o wn er sh ip M on ito rin g an d su pe rv isi on M ec tiz an m an ag em en t Tr ai nn in g HS AM Fi na nc in g Tr an sp or t re so ur ce s Hu m an re so ur ce s Co ve ra ge Groups of indicators Sc or es PLANNING (Highly, 3,2): Onchocerciasis control is planned for on a yearly basis as an integral part of the State’s overall diseases control programme. The onchocerciasis plan is anchored on the State’s CBM- sponsored Primary Eye Care Programme (PECP). There is a written plan, which shows variations from year to year. Although the planning process is initiated by the State Onchocerciasis Control Coordinator, he does not seem to be conversant with the final outcome and the ultimate support provided for CDTI depends on the ‘goodwill’ of the Executive Secretary of the State PHCDA who administers the State’s integrated disease control programme. The planning process clearly involves all partners in a meaningful, participatory and proactive way. The CBM’s PECP is an important feature of the CDTI Oncho-related activities. Although it was reported that there were plans for sustainability of CDTI after APOC fund is withdrawn, these plans were of a general nature and non-specific. There were no documents made available to the team to support the declaration. Credibility is however assumed when reference is made to the initiative of the State government and its partners in instituting a sustained distribution of ivermectin, without APOC funding or structure, in 162 communities in the nine onchocerciasis hypoendemic LGAs in the State. This reflects ownership and some evidence of ability to mobilize resources and use them for onchocerciasis control. 13 LEADERSHIP (Fully, 4): The management team at this level has the capacity to initiate key CDTI activities as evidenced by the presence of detailed plans for the last treatment rounds. There is a focal person in charge of CDTI at this level. This focal person is also involved in other health programmes. As the State Onchocerciasis Coordinator (SOC) and the PHCDA meet regularly and problems relating to CDTI implementation are dealt with promptly. Reports to donor agencies and other State operatives are submitted on time. The State Onchocerciasis team works in a collegial manner with specific tasks assigned to individuals. Responsibilities are delegated as needed. MONITORING/SUPERVISION (Highly, 3.5): There was little doubt that at the State level, monitoring and supervision was planned and carried out in an efficient and integrated manner. The SOCT participated in monitoring and supervision of other disease control programmes. Resources are used efficiently, and the availability of a supervision checklist was confirmed. There is a credible process to manage problems and successes. Verbal commendations are said to be the norm, but it is proposed that, starting from 2005, formal letters of commendation would be issued to deserving staff. Six pioneer project CDDs have gone on to be elected as LGA councilors, obviously benefiting from the prominence they gained in their communities for their work as CDDs. Virtually all of them indicated their intention to continue as CDDs. With regards to problems, the State officials arrange for meetings periodically, as and when necessary, at which ‘programme problems’ that are not necessarily oncho-programme specific, are tackled. The PHCDA Executive Secretary and the SOC meet at least once a week, and sometimes as often as three times a week, to discuss issues related to the CDTI programme. The NGDO once successfully intervened with State Governor when the counterpart funding from the State government had not been released when due. MECTIZAN® PROCUREMENT AND DISTRIBUTION (Slightly,1): There is no doubt whatsoever that sufficient Mectizan® is being ordered, properly stored and efficiently distributed in an integrated manner within the government system. However a disproportionately large amount of the resources and effort at procurement of Mectizan® is derived from the NGDO partner. The evaluation team feels that this over-dependence on the NGDO is not sustainable, and strongly urges that the government assumes a much greater share of the responsibility for the management of Mectizan® for the State’s CDTI programme. TRAINING (Moderately, 2.5): All aspects of this indicator are apparently being satisfactorily carried out. There is reasonably convincing evidence from testimony at lower levels that these lower levels are well ‘empowered’ by the State level. However, the evaluation team has some reservations on the information obtained from staff at these levels because of a lack of documentary evidence in support. The staff interviewed admitted that hitherto training had been “routine and regular”, however it is proposed by the State project officials that, starting from the next calendar year, training (and re- training) would be more integrated and targeted. It was asserted that CDTI staff at all levels are very knowledgeable and conversant with all relevant procedures. It was not possible to ascertain if the resources for training are being used in an efficient and sustainable manner. The NGDO partner, CBM, has limited its training inputs to only the transfer of new skills and procedures e.g. on Community-Self-Monitoring (CSM) and computer-related skills. 14 HSAM ( Moderately, 2.5) All principal officers, including the Executive Secretary of the PHCDA, the Director of Disease Control, Director Ministry for Local Government, the NGDO State Coordinator and the SOC are engaged in advocacy at least once a year. Mobilization of the LGA authorities takes place on a yearly basis. There is little evidence that these HSAM activities have led to specific actions. The bulk of HSAM materials have been procured with APOC funds. CBM has provided some of the resources for procurement of the materials. FINANCIAL RESOURCES (Moderately, 2,22): The absence of documentary evidence posed a problem with ascertaining if the cost of each onchocerciasis control activity was clearly spelt out as claimed by the SOC. However there was agreement by staff at the state level that the SOC received all he requests for to run the programme. Apparently only the polio eradication programme receives more funds than CDTI. It was also difficult to establish if there is in existence a constant reduction/containment strategy or if the total amount budgeted for fell within the estimated income. From all indications the ‘project managers’ do not have a clear estimate of the funds that would be available to them specifically for oncho control in the ensuing year, since funds for all disease control programmes are operated more or less as a ‘one line vote’. The amount of money actually released for CDTI in Jigawa State, from the Disease Control budget was said to have been increasing yearly. (In 2001 -- N263,472; In 2002 – N324,640; In 2003 – N392,590; and In 2004 – N 462,400). These sums also covered the expenses on the 9 extra non-APOC LGAs that were supported directly by CBM. CBM’s budget for 2003 was N 2.5million The information needed to determine the proportionate relationship of the increase in absolute figures, of the funding to the overall funding and expenditure for CDTI, from all sources, was not available for the team to review. Deficits in respect of financial resources are rare. However, whenever they have been encountered resort was made to application by the PHCDA to the appropriate State Authority (the Ministry of Budget and Economic Planning) for virement of funds in accordance with government regulations. The team however did not think virement, which should be a last ditch measure is a solid or satisfactory basis for sustaining the CDTI. The NGDO partner, CBM does not operate a deficit budget, and apparently looks at its Primary Eye Care Programme as a veritable guarantee for the Onchocerciasis Control Programme. There was however no written commitment on this. Budget documents could not be accessed, but officials interviewed at the State level asserted that all the elements of a properly functioning and efficient control system are in place and that all funds released are spent as budgeted. The PHCDA Executive Secretary and the NGDO Representative are signatories to the APOC account, while the Executive Secretary and the PHCDA Accountant are the signatories to the State account. Although the State Oncho Coordinator is not aware of the total budget available from all sources for CDTI it was established by the team that there is no reason why he should not be aware if he wants to. He was encouraged to obtain the information and make use of it for programme efficient planning and execution purposes. 15 TRANSPORT AND OTHER MATERIAL RESOURCES (Highly, 3): Type of vehicle No. Source* Functional status and adequacy for CDTI tasks** Hillux 4 W-D vehicle 1 APOC OK Motorcycles 5 APOC All are 5 yrs old & need replacement; 3 more needed Misc. items ++ CBM Used on a pooling basis, from HQ in Joss Type of equipment No. Source* Functional status and adequacy for CDTI tasks** Generator 1 APOC Not functioning for 6 months Photocopier 1 APOC Not functioning for 3 months Overhead projector 1 APOC Functioning Fax machine 1 APOC Functioning Laptop computer 1 APOC Not Functioning, a desktop computer is also needed Training/ HSAM material No. Source* Functional status and adequacy for CDTI tasks** Posters, Handbills etc. APOC Adequate when needed Tee shirts APOC Adequate when needed Magnetic writing Board 1 APOC Broken in transit, needs replacement IEC materials: ++ CBM Supplied adequately and used on a pooling basis There has been little motivation or pressure to repair the non-functioning generator and photocopier because the programme has access to more reliable and functioning ones belonging to the ‘parent’ body, the PHCDA It was stated that the State intended to fill any gaps that may arise in maintaining equipment and other material resources as from the time of the withdrawal of APOC funds. Vehicle control documents were not available for inspection by the team, but inspection of the vehicle itself showed that it was well used and maintained. The Team was assured that all trips made for CDTI purposes were properly authorized by the relevant official and recorded in a logbook. The Team had no reason to think this was not so. Whereas the SOCT has ready and unfettered access to other vehicles in the Departmental pool, when project vehicles were “off the road”, there are no special plans for replacement of project vehicles. The CBM has made provision for three additional motorcycles to be given to the CDTI programme for the three CDTI project LGAs that at present do not have motorcycles. The motorcycle needs of the 9 non-APOC LGAs will be taken care of under the PECP. HUMAN RESOURCES (Highly, 3.5): The SOCT is composed of 5 members. The table below shows some relevant attributes of current SOCT staff: Area of skill No. of persons qualified in this area N° of persons competent enough to perform the job Planning 2 3 Report writing 4 4 Training and HSAM 5 5 Monitoring/ supervision 5 5 Data management 5 5 Computer skills 1 1 Mectizan ordering/ distribution 3 3 16 The staff are very stable and all have been at their CDTI – related posts for upwards of five years; they are all said to be very knowledgeable, highly motivated and dependable. COVERAGE (Fully, 4): Geographic coverage was said to have been 100% of the 8 LGAs that were meso endemic for Onchocerciasis and hence eligible to mass treatment with Mectizan®. The TCR has gone from 83% in 2002 to 88% in 2003 and it has been stable since then. Data for the 2004 treatment round was not available as mop up was still taking place. Recommendations for the State Level Recommendations Implementation Mectizan® procurement and distribution: State Government should assume a much greater share of the responsibility for the drug management process, particularly the ordering aspect, of Mectizan® for the State’s CDTI programme. Priority: MEDIUM Indicators of Success: Overdependence on NGDO for Mectizan® procurement is reduced. Who takes action: State’s Health Office Deadline for completion: December 2005 Planning: 1. Present initiatives taken to develop plans for CDTI should be continued to ensure sustainability of CDTI when external funding is withdrawn. 2. Copies of all relevant documents related to CDTI should be made available to all stakeholders to enable appropriate monitoring of the programme to be conducted. Priority: HIGH Indicators of Success: 1. There is a strategic and functional plan for CDTI programme that appears as part of an overall health plan 2. CDTI programme is properly budgeted and documented Who takes action: State Onchocerciasis Control Programme Coordinator Deadline for completion: End of December 2004 Finance: 1. The budgetary provision from the government for CDTI activities should be clearly spelt out. 2. SOC should acquaint himself with budgetary matters related to CDTI for efficient planning and implementation of activities. Priority: HIGH Indicators of Success: 1.Clear budgetary provision for CDTI programme 2. SOC is aware of total budget available from all sources for CDTI implementation. Who takes action: State Onchocerciasis Control Programme Coordinator Deadline for completion: End of December 2004 Training: 1. Staff training should be documented, carried out in an integrated and targeted manner so as to ensure efficiency and sustainability. 2. Training manuals should be developed and made available for all levels. Priority: HIGH Indicators of Success: 1. Staff training is now properly documented 2. Staff training is no longer routine but targeted and integrated 3. Training manuals are now available for all levels Who to take action: PHC, SOC Deadline for completion: January 2005 17 3.2 LGA Level SCORES AT LGA LEVEL 3 3 3,50 3 2,5 2,5 3 3 2,5 3,50 4 0 0,5 1 1,5 2 2,5 3 3,5 4 Pla nn ing Int eg rat ion Le ad ers hip an d o wn ers hip Mo nit ori ng an d s up erv isio n Ob tai nin g a nd m an ag ing M ec tiz an Tra inn ing HS AM Fin an cin g Tra ns po rt res ou rce s Hu ma n r es ou rce s Co ve rag e GROUPS OF INDICATORS SC O R ES /4 PLANNING (Highly, 3): Yearly plans of CDTI were seen and some of these plans were clearly integrated into the overall LGA plans for health. Plans take into consideration key CDTI activities such as training, Mectizan® supply, mobilization, etc. Some of the plans had budgetary provisions but it did not seem as if plans were drawn up in a participatory way. No minutes of the planning meetings were available for inspection. INTEGRATION (Highly, 3) There is evidence that CDTI activities are integrated at LGA level. Staff carry out more than one CDTI activity at a time when possible and are involved in other health programmes such as Polio eradication, Leprosy control, etc. LEADERSHIP (Highly, 3.5): Responsibility for management of CDTI at the LGA level has been fully taken over by LGA staff. In all the three LGAs visited, CDTI was being implemented as part of the LGAs routine activities. Supervision and monitoring are initiated at this level with support from political and administrative leaders. For instance, HSAM materials have been translated into local languages, such as Hausa. However, training, Mectizan® ordering and distribution are initiated at a higher level. 18 SUPERVISION and MONITORING (Highly, 3) Supervision and monitoring visits are being carried at the FLHF level in an efficient and integrated manner. There were reports of visits of staff from this level at community level, but this did not seem to be the routine. Supervision visits were reported to be problem-related and oriented in most cases. Problems identified during supervision are apparently addressed in conjunction with personnel from FLHFs and other relevant authorities. CDTI reports are submitted to the State level within the government system and as part of Health Management Information System. Even though the reports incorporate all key CDTI activities, checklists are not used as a routine. Well performing personnel are said to be verbally encouraged. MECTIZAN® SUPPLY (Moderately, 2.5): Supply of Mectizan® at the LGA level has always invariably been sufficient and ordered in good time. Shortages or late supply do not occur at the LGA level though on one isolated occasion shortage and late supply were experienced at a lower level. Documentation relating to Mectizan® ordering, collection and delivery to lower levels does not exist. Mectizan® tracking was very difficult for the team. In one case, a slip made out for the supply of Mectizan® was seen but it was very vague. The number of tablets requested was not indicated. Mectizan® is routinely collected, stored and distributed within the government system. Although Mectizan® is mostly collected at the LGA by FLHF staff, there were reports of systematic delivery to some FLHF by LGA staff. TRAINING (Moderately, 2.5): Staff at this level mostly trained FLHF staff. There were reports of LGA staff training CDDs. Although this did not seem to be the routine, is difficult to justify. FLHF training seemed complete though not necessarily targeted. Training agenda and training reports were not seen to ascertain the quality of training. Training materials were reported to have been sufficient and a few were seen in the LGAs visited. New personnel are taken into consideration during training. There was no evidence that training of FLHF was being done in an integrated way. HSAM (Highly, 3) HSAM activities are said to be planned and carried out in an effective manner. These activities are carried out, both at LGA and community levels, in support of FLHFs who generally lack adequate transport. These activities have led to acceptance of the programme at FLHF and community levels and have contributed to sustained and improving treatment coverage. 19 However, documentation of HSAM activities was not seen; minutes of meetings were not available, nor were there any yearly reports of HSAM-related programme activities. FINANCING (Highly, 3): CDTI budgets for this level were seen. Amounts budgeted ranged from N150,000 to N 300,000. These amounts fell within the estimated income of the LGAs. LGA officials are all aware of the need for them to provide counter part funds for CDTI. In most LGAs visited, it was said that money had been released according to expressed needs. It was however difficult for the evaluation team to ascertain, in the absence of documentation, the degree and consistency of provision of financial support of CDTI activities. The only document seen showed that money was spent for disease control in general and was not specific to onchocerciasis. Money was disbursed in block for health programmes, including CDTI. There was no cost breakdown per activity for the oncho programme and there was no evidence of cost containment. . TRANSPORT AND OTHER MATERIALS (Moderately, 2.5) Although no APOC motorcycle was seen in the three LGAs visited, LGA staff have unlimited access to pooled resources, which are used in an integrated manner. There is evidence of routine maintenance for pooled motorcycles. CDTI activities have not been disrupted because of lack or motorcycle breakdown. Logbooks are not being used. Trip authorisation forms are said to exist, but the evaluation team saw none. There are no specific plans for the replacement of existing transport materials, which, in the opinion of the team seem inadequate for all the health programmes being implemented at LGA level. HUMAN RESOURCES (Highly, 3.5) Staff at this level are very stable and most had been in their positions for the past six years. They are very committed and dedicated. They all expressed their willingness to continue to do their job. Salaries are being paid at the appropriate time. However, in-service training plans were not reflected in the action plan. COVERAGE (Fully, 4) Geographic coverage was said to have always been 100% and therapeutic coverage has been consistently well above 65%. 20 Recommendations for the LGA Level Recommendation Implementation Planning: Minutes of meeting should be documented and there should be evidence that plans were developed in a participatory manner and are integrated in the over all health plan. Priority: HIGH Indicators of success: 1. There are yearly plans of CDTI which are developed in an integrated manner 2. The plan takes cognisance of key activities such as training, mectizan supply, and mobilisation. 3. The plan has an attached budget with timeline. Who to take action: LGA Coordinator. Deadline for completion: By the end of December 2004. Leadership and Ownership 1. LGAs should be empowered to initiate their own programme of action. Priority: MEDIUM Indicators of success: 1. LGA management takes full responsibility for CDTI activities in term of initiation of key programme activities. Who to take action LGA and PHC Coordinators. Deadline for completion: January 2005. Monitoring and Supervision Supervisory checklist should be developed and used in an integrated manner. Priority: HIGH Indicators of success: 1. LGA staff are carrying out monitoring and supervision In an efficient manner using checklists. Who to take action: LOCT and PHC Coordinator. Deadline for completion: Immediately. Obtaining and Managing Mectizan® 1. Documents related to Mectizan® requisition and delivery to lower levels should urgently be developed and used. Priority: HIGH Indicators of success: 1. Mectizan® available on time and in adequate quantity. Who to take action: PHC and LOCT. Coordinator. Deadline for completion: January 2005 21 Recommendation Implementation Training Training should be targeted and integrated so as to be efficient. 1. There should be provision of enough training materials and evidence shown on training done. Priority: MEDIUM Indicators of success: 1.Staff at this level train the FLHF in an efficient and integrated manner. 2.There are enough training materials and evidence of training done Who to take action. PHC and LGA Coordinator. Deadline for completion Before the beginning of next year’s distribution round. FINANCE 1. Document relating to budget at the LGA level should be clearly written and put in appropriate file for easy reference. 2. Funds release should be done on time and properly documented. 3. There should be evidence of cost containment within estimated budgets Priority: HIGH Indication of success: 1. Yearly budget are document. 2. There is evidence of funds released. 3. There is evidence of yearly reduction in estimated budgets Who to take action: LOCT, HOD. Deadline for completion: As from now TRANSPORT 1. LGA authorities should make provision to increase the number of motorcycles in their pool to accommodate the growing needs of health programmes. 2. There should be a plan at the LGA for the replacement and repairs of existing motorcycles. 3. Logbooks or authorization form should be designed for usage at the LGA level. Priority: MEDIUM Indicators of success: 1. There is provision for additional motorcycles to accommodate health programmes 2. There is a plan being implemented for the replacement and repairs of existing motorcycles 3. Logbooks are being used Who to take action PHC coordinator Deadline for completion: As from now HSAM 1. HSAM activities should be targeted and specific. 2. Yearly report of HSAM activities should be documented. Priority: MEDIUM Indicators of success: 1.HSAM now effectively planned and implemented. 2.there are yearly reports of HSAM activities Who to take action HOD and LOCT. Deadline for completion: Jan – Dec 2005 22 3.3 Front Line Health Facility Level SCORES AT FLHF LEVEL 2 3 2,00 3,3 3 2 3 1 0,66 2,00 4 0 0,5 1 1,5 2 2,5 3 3,5 4 Pla nn ing Int eg rat ion Le ad ers hip an d o wn ers hip Mo nit ori ng an d s up erv isio n Ma na gin g M ec tiz an su pp ly Tra inn ing HS AM Fin an cin g Tra ns po rt res ou rce s Hu ma n r es ou rce s Co ve rag e GROUPS OF INDICATORS SC O RE S/ 4 PLANNING (Moderately, 2): At the FLHF level, timetables of CDTI activities and other health programmes are pasted on the wall in some FLHFs. In some FLHFs, staff are not conversant with the programme timetable. CDTI activities are not systematically planned and staff lacked knowledge on why and how to plan. In spite of this staff apparently carry out CDTI activities effectively when necessary. INTEGRATION (Highly, 3) Staff in some LGAs carry out CDTI activities in an integrated manner; by including other activities of other programmes such as NPI, malaria and TB and leprosy CDTI activities that are carried out in an integrated way include, HSAM, training, monitoring, and supervision, Mectizan® delivery and collection of data. LEADERSHIP (Moderately, 2): Responsibility for CDTI and other health activities are fully taken by the staff. While some had itinerary of CDTI activities and initiate activities, other do not. They wait for the LOCT to bring the drug and are told when to distribute them. However the political heads are aware of CDTI activities participated in them and assisted in ensuring their success as a result of advocacy carried out by staff at this level. 23 MONITORING AND SUPERVISION (Highly, 3.3): All relevant data related to CDTI implementation were available at the FLHF level. In some cases however, the reports were scanty and difficult to comprehend. This could however be adjudged to reflect the educational level of the staff at FLHF. Integrated routine supervision was done twice a year for oncho and other health programmes. Timetables of visits were seen in some facilities. Problems identified during visit such as lack of incentives to CDDs and cases of drug refusals are treated appropriately, and when necessary referred to LOCT. While successes are sometimes acknowledged with reward of T-shirts and written commendation, in some health facilities, most are limited to verbal commendation. MECTIZAN® PROCUREMENT AND DISTRIBUTION (Highly, 3): Ordering of Mectizan® was said to be based on census data, and in some health facilities order forms were seen which indicated timely delivery. Distribution is being controlled within the government system in a simple and efficient manner. In most cases FLHF collect the drug from the LGA level and CDD collect the drug from FLHF. However, there were cases of late delivery due to logistic problems. Mectizan® shortages were also experienced in some units but additional supplies were readily provided to solve the problem. TRAINING (Moderately, 2): Training at the FLHF is usually carried out once in a year as refresher course to CDDs. Training is said to be targeted. Training of CDD appears to be efficient and this was evidenced by CDDs demonstrable skills and knowledge when carrying out activities. A few training materials were seen by the team but there was no documentation related to the training of CDDs. The training of FLHF takes place in the LGA while the CDD training is done at FLHF level. HSAM (Highly, 3) Health facility staff inform leaders about important issues like drug shortages, which lead to appropriate measures to rectify the situation. However no yearly plan or report were seen on HSAM activities. There was some evidence of FLHF staff carrying out education and mobilization activities, which contributed to improved community acceptance of the Programme and increased therapeutic coverage. FINANCIAL RESOURCES (Slightly, 1): Budgeting is not usually done at the FLHF. The staff has no idea of how to get funds from the LGA. They do not know about budget breakdown and sources of funds. It is not uncommon for staff to use personal resources for routine CDTI activities and this seemed to have been the pattern for the past three years. The team thought it was remarkable that despite the poor funding at the FLHF CDTI activities are still being implemented in a reasonably effective manner. Nevertheless the team is of the opinion that it would be extremely difficult to sustain CDTI operations at FLHF of the Jigawa project if staff continue to be peripheral to the generation and management of the financial resources and if the non-existent funding pattern persists. 24 TRANSPORT AND OTHER MATERIALS (Not at all, 0.66): There is absolutely no means of transport provided to staff at this level .The staff walk, borrow and use personal means to cope with Programme activities. Although there is some awareness on the part of the government of the need to solve this problem, the team did not find any evidence of a clear-cut plan on how this was going to be done. HUMAN RESOURCES (Moderately, 2): Staff at the FLHF in general have adequate skills to carry out relevant CDTI activities such as, training of CDD, monitoring and supervision, as well as Mectizan® ordering and distribution. The staff are quite stable. Personnel at FLHF receive routine in-service training at the State School of Health Technology. Generally the team is convinced that staffing at this level is not adequate although when transfers occur replacements are immediately trained. COVERAGE (Fully, 4): Geographical coverage rate (GCR) was reported to have been 100% for the past 3years. Reported Therapeutic coverage rate (TCR) of the FLHFs visited had an average of: - 69.4% in 2002 - 75.0% in 2003 - 80.4% in 2004 TCRs have been high and are increasing. 25 Recommendations at the Front Line Health Facility Level Recommendation Implementation Planning: 1. FLHF must be encouraged to develop CDTI plans of activities in a participatory way. Priority: HIGH Indicators of success: 1. Integrated plan at the FLHF is now available. Who to take action: FLHFs Deadline for completion: NOW. Leadership and Ownership 1.The FLHF should be empowered to initiate key CDTI activities such as planning, monitoring, supervision, training HSAM, Mectizan® ordering and distribution. Priority: HIGH Indicators of success: FLHF now carry out CDTI activities on their own. Who to take action FLHF and LOCT. Deadline for completion: Now Monitoring and Supervision 1. All supervisory visits should have reports. 2. Reports collected should be used in future plans and actions. 3. Supervisory checklist should be available at this level. Priority: HIGH Indicators of success: 1. Reports of visit are available. 2. Future activities are based on reports. 3. Presence of supervisory checklist. Who to take action: FLHF and LOCT. Deadline for completion: As from the launching of next distribution Obtaining and Managing Mectizan Adequate Mectizan® should be acquired and delivered to communities on time. Priority: HIGH Indicators of success: There is no report of shortages or late delivery to the community. Who to take action: FLHF staff and LOCT. Deadline for completion: As from the launching of next distribution. Training 1. All activities carried out on training should have documented for future use and reference. Priority: HIGH Indicators of success: Availability of report on training by FLHF. Who to take action: FLHL staff. Deadline for completion As from next distribution Priority: HIGH 26 Recommendation Implementation HSAM HSAM material should be procured or printed and made available at this level for use of FLHF and community. All political heads in the community where FLHF are situated should be mobilized to participate in CDTI activities. Indicators of success HSAM materials are available for use. Political heads are now fully participating in CDTI activities Who to take action: LOCT, HOD Deadline for completion: As from distribution round Finance 1. Staff at the FLHF should be acquainted with matters relating to budgeting: techniques and potential sources of funds. Priority: HIGH Indicators of success: FLHF staffs are now skilled in writing budgets Who to take act LOCT. Deadline for completion: January 2005. Transport and Material Resources Transport and funds for their maintenance should be made available at this level by LGA with logbook. Priority: HIGH Indicators of success: 1. Presence of transportation at this level. 2. Funds are made available for the maintenance of transport. Who to take action: LGA authorities Deadline for completion: Before the next round of treatment Human resources Staffing at FLHF should be adequate and well skilled to be able to implement CDTI activities efficiently. Priority: HIGH Indicators of success: Adequate and skilled staff at FLHF level are now available to efficiently implement CDTI activities. Who to take action HOD and LOCT. Deadline for completion: NOW. 27 3.4 Community Level SCORES AT COMMUNITY LEVEL 4 3,67 2 1,5 3 3 3,33 0 0,5 1 1,5 2 2,5 3 3,5 4 Pl an nin g a nd m an ag em en t Le ad ers hip an d o wn er sh ip Mo nit or ing Ma na gin g M ec tiz an su pp ly HS AM Fin an cin g Hu ma n res ou rce s Co ve ra ge GROUPS OF INDICATORS S CO RE S/ 4 PLANNING AND MANAGEMENT (Fully, 4): Communities decide on time and routes of distribution. In some communities visited, census update and Mectizan® distribution are carried out at different periods. In others, distribution and census are carried out at the same time. Community leaders and the entire community fully participate in planning along with CDDs. Population figures for the current year are used for estimating Mectizan® request for the coming year. In many of the communities, leaders participate in carrying out sensitization and mobilization. Decisions relating to the Distribution process are made during community meetings called by community leadership. LEADERSHIP AND OWNERSHIP (Highly, 3.67): In most cases, community leaders take on the responsibility of sensitizing and mobilizing their people, choosing CDDs and deciding on the mode of Mectizan® distribution. This has been compatible with good coverage. In some cases, communities provide bicycles to CDDs to help with the distribution process. Community leaders actively participate in managing problems relating to Mectizan® distribution and ensuring that all eligible populations took the drug. Some leaders even visit households to convince those refusing to take the drug. In one case, the community appointed a focal person for CDTI activities. However in some communities, CDDs complained of not receiving financial assistance from the community. Oftentimes the communities meet to select CDDs. In some cases, women are consulted during women gatherings for CDD selection and mode of distribution. However, in one instance a CDD was designated by the FLHF as a result of poor community education and mobilization. In another, the chief decided on both the CDD and the mode of distribution. In two cases, CDDs and the community reported to be dependent entirely on the LGA to decide on the timing and mode of distribution, as they did not know when the distribution was to take place. In some others, CDDs decided on both the timing and the mode of distribution and the community accepted the CDDs choice. Most communities have Community Monitoring Committees and 28 are responsible for deciding the form of support to provide for CDDs on CDTI duties. These include working in CDDs’ farms and the giving of financial incentives. Though community members were not aware of the precise duration necessary for mass treatment with ivermectin, they passionately expressed their wish to take Mectizan® for as long as it is necessary and available. In a few isolated cases some community members expressed reservation about ivermectin based on the controversy generated by the oral polio vaccination programme. All the community members interviewed knew more than one advantage of taking Mectizan®, they even said it improved their sexual performances. Most communities visited asked to know if they could not be given the drug more than once a year. MONITORING and SUPERVISION (Moderately, 2): CDDs appropriately report to the FLHF. Reports consist of tally sheets and contain all necessary details. Reports are sent promptly after the ivermectin distribution. Unfortunately, in many instances, lack of adequate support compelled CDDs to spend their own money on some CDTI activities. This is clearly not sustainable especially seeing that it is in only in a few communities that receive assistance with OBTAINING AND MANAGING MECTIZAN® (Slightly, 1.5): In the majority of communities there have always been adequate and timely supplies of ivermectin but with hardly anything left for absentees. Some communities experienced acute drug shortages for those eligible and available for treatment. Requests for Mectizan® are invariably based on the treatment figures for the preceding year. CDD records are good enough to enable accurate calculation of the amounts of Mectizan® needed. However, CDD do not know how to estimate the quantity of ivermectin needed for their communities, therefore Mectizan® was given to CDDs in bulk not infrequently irrespective of the actual need. Most CDDs and community monitoring committees collected Mectizan® from the nearest FLHF. In others Mectizan® was systematically brought by either FLHF or even LGA. In a few cases, villagers arrange transport for CDDs. but in many, CDDs were not given a means of transport. . HSAM (Highly, 3): CDDs and village leaders have actively been involved in HSAM activities. HSAM is mainly done once a year, right before the distribution campaign. HSAM addressed issues relating to dosage, importance of taking the drug, etc. They sensitized and educated community members, religious authorities and town criers. Religious leaders also participated in disseminating CDTI messages. However, it was reported from very few villages that CDDs are not engaged in HSAM as they lived outside of the village. Women were educated and mobilized during ceremonial festivals. In some communities, health committees had been formed and these also carried out HSAM on CDTI. FINANCING (Highly, 3): The situation varies from place to place; in some villages CDDs are supported by their communities, in others (about 50%) they are not. In areas where CDDs are supported, prayers are said for them, money (10-20 naira) has also been given to some. Some CDDs have free labor or were exempted from community work. In some cases, CDDs refused any form of support from the community, as they believed it was their moral responsibility towards their community. Village heads also support CDDs financially or in kind. 29 Stationery materials for CDTI mainly come from the LGA health office. In few cases, they come from village heads. In all cases, communities are willing to provide stationery if need be. HUMAN RESOURCES (Highly, 3.3): In most villages, the recommended ratio of CDDs to population or households is not attained. It ranges from one CDD per 20 households to one CDD per 125 households, and one CDD per 300 persons to one CDD per 2500 persons. Generally, CDD cover distances of 2-3 km per treatment round. In some areas however, CDDs cover longer distances, sometimes up to 10 km (Bagadaga) from their homes. In most cases, lack of awareness about the recommended ratio was the reason given for the low ratio. The norm is two CDDs per village irrespective of the size of the population and type of settlement. Also, communities expressed concern about having to cope with an increase in the cost of remunerating CDDs if the numbers of CDDs were to increase. CDDs are trained on a yearly basis. They seem to be well trained and skilled in carrying out CDTI activities. They however lack knowledge on Mectizan® supply calculation. They all were able to say what criteria excluded people from getting Mectizan® treatment. Where CDDs fail to discharge their CDTI duties correctly, they are replaced with new ones who received training subsequently. CDD dropouts are re also immediately replaced in like manner. All the CDDs interviewed expressed their willingness and dedication to continue with their CDTI activities. Some of them even remarked that they have a moral obligation to serve the people who elected them. Very few CDDs have dropped out during the lifespan of the project. COVERAGE ( )  All villages visited had TCR of more than 65% in 2004. These TCR have apparently been increasing through the years. The team observed that in a number of communities only eligible population were being recorded in CDD registers while in others only people treated were being recorded; local calculation of coverage was based on the eligible population rather than total [population as denominator. Besides the team, from interview at this level and a careful inspection of the available records, has cause to believe that the high TCRs being reported did not reflect the true picture. 30 Recommendations at the Community Level Recommendation Implementation Planning: CDD should be encouraged to carry out census updates and Mectizan distribution at the same time for efficiency purpose. Priority: LOW Indicators of success: Census update and treatment are now carried out simultaneously Who to take action: CDDs, FLHFs & Community leaders Deadline for completion: During next distribution Leadership and ownership: 1. Community should be encouraged and left to take responsibility for decisions such as selection and number of CDD, time and mode of distribution and especially type of support to give to CDDs 2. Affected communities’ religious leaders should be targeted for HSAM on CDTI so as to address misconceptions about Mectizan® (as it relates to polio) 3. Community leaders should arrange for CDD transport to make their work easier Priority: HIGH Indicators of success: 1. Communities now take full ownership of CDTI 2. Mectizan® is no longer being linked to polio 3. CDD now have transport means Who to take action: FLHF, Community leaders Deadline for completion: During next distribution Managing Mectizan® Mectizan® supplies given to CDDs should be based on updated census data and CDDs should collect their Mectizan® stock at once to avoid shortages. Priority: HIGH Indicators of success: CDDs now collect enough Mectizan® for distribution at once. Who to take action: FLHFs Deadline for completion: Before next round of distribution Human Resources: 1. The ratio of 2 CDDs to 250 persons per community should be considered to reduce fatigue on CDDs. 2. CDDs should be trained on how to estimate their Mectizan® needs Priority: HIGH Indicators of success: 1. Adequate number of CCDs are selected for CDTI activities 2. CDDs can now make their own estimates of drug to be given Who to take action: Community members, FLHF Deadline for completion: Before next round of distribution Coverage: Therapeutic coverage should be based on total population, not eligible population or Annual Treatment Objectives (ATO) Priority: HIGH Indicators of success: TCR are calculated using total population Who to take action: FLHF Deadline for completion: During next round of distribution 31 3. CONCLUSIONS 4.1 Grading the Overall Sustainability of the Jigawa CDTI Project A judgment of the sustainability of the Jigawa CDTI was made according to the following “aspects of sustainability” and “critical elements”. Aspects of Sustainability: Aspect Judgment: to what extent is this aspect helping or blocking sustainability of this project? Integration Very much helping Resources Not helping Efficiency Helping Simplicity Very much helping Attitude of Staff Very much helping Community Ownership Helping Effectiveness Helping Integration: Very much helping sustainability CDTI activities have been very well and highly integrated with other health programmes in Jigawa. Integration has been one of the strongest points of the Jigawa CDTI. Resources: Not helping Even though little money and resources have been pumped into CDTI by the various operational, CDTI activities have not collapsed. There has been no direct designated and specific allocation of funds by the State government to CDTI and there are no plans to do so after APOC funding is withdrawn. Unless the government takes prompt action to provide more resources for CDTI the prospects of the Programme being sustainable are not bright. Efficiency: Not blocking sustainability Regarding efficiency, strengths have been observed in such areas as Mectizan® distribution, using pooled vehicles, but there is poor data management, which does not augur well for sustainability. Lack of budgetary allocation and a cost breakdown for CDTI activities demands prompt rectification by State officials. That the project has been performing well is due to a large measure to the efficient support provided by the NGDO partner. How sustainable this position is is open to question. 32 Simplicity: Very much helping sustainability CDTI activities in Jigawa have been fairly simplified. Reporting and data collection systems are simple and effective. All procedures are clear and simple and sustainable, and there is no Programme-inhibiting bureaucracy. Attitude of Staff: Very much helping sustainability Personnel at the different levels of the project have shown great enthusiasm and interest in carrying out CDTI activities. This is one of the keys to ensuring the sustainability of CDTI. All the staff interviewed at all levels expressed their strong willingness to continue to carry out CDTI activities. Many have indeed been contributing substantially to the oncho control programme. Community Ownership: Helping sustainability The evaluation team found out that community members were playing active roles in CDTI programme. They participate in selecting CDDs, changing CDDs when necessary, and some times provide compensation to CDDs. Moreover, the CDDs are very committed to the Programme. The communities included in Jigawa CDTI Project have demonstrated a sound degree of ownership of the programme. Effectiveness: Helping sustainability The project has been effective. It has been achieving its overall goal to cover 100% of meso endemic communities with a therapeutic coverage reported at more than 65%. There are however a number of issues relating to effective planning and budgeting, fund raising and effective fund release and proper management. 33 Critical Elements of Sustainability: Critical Element Yes/No Money: Is there sufficient money available to undertake strictly necessary tasks, which have been carefully thought through and planned? (absolute minimum residual activities). No Transport: Has provision been made for the replacement and repair of vehicles? Is there a reasonable assurance that vehicles will continue to be available for minimum essential activities? (note that ‘vehicle’ does not necessarily imply ‘4x4’ or even ‘car’). Yes Supervision: Has provision been made for continued targeted supportive supervision? (the project will not be sustained without it). Yes Mectizan® supply: Is the supply system dependable? (the bottom line is that enough drugs must arrive in villages at the time selected by the villagers). Yes Political commitment: Effectively demonstrated by awareness of the CDTI process among policy makers (resulting in tangible support); and a sense of community ownership of the programme. Yes Money: No There have been critical problems with regard to release of budgets and flow of funds that has a blocking impact unless some measures are taken soon. On the other hand, the process of planning for sustainability was started with the team of evaluators and no provision has been made as to how activities will be carried out. There has been little effort put in mobilizing additional funding from other sources. Transport: Yes Although transportation facilities have not been sufficiently availed for CDTI activities at some levels, the traditions of using pooled transport and providing transportation assistance across 34 different health programmes will help the sustainability of the project. It should also be noted that it would become essential for programme authorities to develop plans for the replacement of existing transport means. Supervision: Yes Past supervisions were said to have been targeted and supportive though not documented. The evaluation was informed, during the debriefing/planning meeting that the State Government had made provision for key CDTI activities. It was revealed to the team that some of this money would be used for targeted supervision that will be used to carry out on-the-job-training as formal training is to be reduced. Mectizan® Supply: Yes The supply system for Mectizan® distribution to the communities has been simple and reliable. In most cases, Mectizan® was available in villages in good time for planned distribution. The present delivery system relies on CBM’s high involvement but there is assurance that should CBM fail to fulfil this duty, the NOTF is ready to take over as it is the case in other parts of the country where Mectizan® is ordered by the NOTF, collected by the zonal Onchocerciasis Coordinator and delivered in the State. Political Commitment: Yes The government has accepted CDTI as one of the health programmes. Administrative and political leaders at all levels have shown their supports for the project even though adequate financial support is still being sought. 4.2 Grading of project as a whole On the overall grading of the project, the team found that six of the seven “aspects of sustainability” were either ‘helping’ or ‘very much helping’ the project moving towards sustainability; only lack of adequate resources at all levels was seen as not helping sustainability of the project. In relation to “critical elements”, it was found that “Money” was not sufficient at all levels. The Evaluation Guidelines indicate that where “one or two aspects are not fulfilled and one or two critical elements are not satisfied, the project is making satisfactory progress towards sustainability”. Therefore, the Independent Evaluation Team concludes that the Jigawa CDTI Project is making satisfactory progress towards sustainability. 35 ANNEXES Annex 1: INTERVIEWS State Level 1. Mustapha Abdu Executive Secretary Primary Health Care Agency, Jahun 2. Hadi Hassan Ringim Jigawa State Onchocerciasis Control Coordinator, Jahun 3. Sani Galadima Project Accountant, Jigawa State 4. Chris Ogoshi CBM Country Representative, Joss LIST OF PERSONS INTERVIEWED GWARAM LGA LGA OFFICE Chairman –Ahmed Rufai. Treasurer - Musa Adamu Oncho coordinator - Ado Uguta FLHL Nobody seen. Communities Yarfi Community.  15 people were talked to.  11 male, 4 female. All aged between 16 years to 60. CDD  Magaji Ibrahim VILLAGE LEADER.  Mohd Wanba. Bagadaza Community Community Members.  25 people interviewed, 17 were male while 8 were female CDD interviewed  Yesufu Gambo.  Village Head  Musa Sule. Sakuwa Village  14 people interviewed : 13 were male, 1 female. CDD Interviewed  Audu Ibrahim 36 Village head interviewed  Alhaji Mohd. Kajuma Village Community Members interviewed 19 people interviewed : 17 were male 2 female. CDD interviewed • Ishaku Mohammed. Village Head seen • Mustapha Salim BIRNIWA LGA People Interviewed Vice Chairman : Mohammed Buguna. H O D : Malam Adamu Abdullahi. Oncho Coordinator : Mohammed Aliu Idris. Kubuna Village FLHF Kubuna Person Interviewed • Garuba Bakuli. Village Members 15 people interviewed : all were male. CDD Interviewed • Alhassan Abdullahi. Village Head • Ibrahim Buluma. Kwarkwasa Village. Community Members Interviewed • 17 people interviewed: All were male. CDD interviewed • Jamiu Yahaya Village Head • Ibrahim Maigore. Dagilfanni Villabe. Community Members Interviewed: 9 people interviewed, all were male. 37 CDD interviewed • Dunduma Badimu. Village Head • Maina Bukar. Matara Uku Village FLHF Person Interviewed • Ahmed Amukule. Community Member interviewed 14 people seen: 13 were male , 1female. CDD interviewed • Mohhamed Abubakar. Village Head • Nuhu Yaro. KAUGAMA LGA. Supervisory Councilor ForHealth: Ibrahim Alifa. Treasure : Gambo Ahmed Oncho Coordinator: Sheidu Abubakar Uba Village. FLHF Person Interviewed. • Mohammed Abdullahi. Community Members Interviewed. 18 people interviewed: 15Male , 3 female CDD Interviewed • Mohammed Bagama. Village Head • Isa Usman. Dakayyawa Village FLHF Person Interviewed • Dati Yesufu Community Member Interviewed. 9 people seen: all were male. 38 CDD • Sallau Na Gambo. Village Head. Sabiu Yakubu. DINGARE VILLAGE • Community Member . 14 people seen: all were male. CDD Seen • Yahuza Dan-ladi. Village Head seen • Baffi Abdullahi. Annex 2: Schedule for the Evaluation, advocacy DATE TIME TEAM 1 TEAM 2 TEAM 3 Sunday 21st November Arrival of External and Internal evaluators to Dutse capital of Jigawa State. Monday 22nd November 08:00 – 10:00 Orientation of members on evaluation instruments and review of provisional timetable 10:00 – 12:00 Evaluators review strategies for the evaluation in preparation for the assignment 12:00 -17:00 Courtesy call on Ministry of Health officials and collection of data from officials and SOCT. 15:30 – 16:30 Evaluators review day’s activities Tuesday 23rd November 08:00 – 17:00 Visit to Gwaram LGA to collect data Visit to Kaugama LGA to collect data Visit to Birniwa LGA to collect data 19:30 – 20:30 Evaluators review day’s activities Wednesday 24th November 08:00 – 17:00 Visit to Sakuwa H/F, and 2 villages (Sakuwa and Kajima) in Gwaram LGA to collect data Visit to Ubba H/F and 2 villages (Ubba and Dingare) in Kaugama LGA to collect data. Visit to Kabuna H/F and 2 villages (Kubuna and Kwarkwasa) in Birnwa LGA to collect Data 19:30 – 20:30 Evaluators reviews day’s activities Thursday 25 November 08:00 – 17:00 Visit to Yarfi H/F and 2 village (Yarfi and Bagadaga M. Gwaram LGA to collect data Visit to Dakayeusa H/F and 2 villages (Dakayawa and Hadun) in Kaugama LGA to collect data Visit to Matara Uku H/F and 2 villages (Madara Uku and Digilfani) in Birnwa LGA to collect data 19:30 - 20:30 Evaluators review day’s activities 08:00 – 17:00 Data analysis; Debriefing/Advocacy visit to the decision makers at the LGA levels 40 Friday 26th November 19:30 – 20:30 Evaluators review day’s activities. Saturday 27th November 08:00 – 17:00 Analyze final data; Work out recommendation and work on draft report 19:30 – 20:30 Evaluators review day’s activities Sunday 28th November 12:00 – 17:00 Preparation for feedback/planning workshop 19:30 – 20:30 Evaluators review day’s activities Monday 29th November 09:00 – 17:00 Feed back on evaluators findings to MOH officials and SOCT member /Planning Workshop for State project Tuesday 30th November 09:00 – 17:00 LGA level feed back/ planning workshop all LGAs Wednesday 1st December 09:00 – 17:00 LGA level feed back /planning workshop continues Thursday 2nd December 09:00 – 17:00 Final evaluation report writing for the Jigawa State CDTI Friday 3rd December 09:00 – 18:00 Departure from Dutse, Report writing Saturday 4th December 10:00 – 16:00 Report writing Sunday 5th December 10:00 – 16:00 Report writing Annex 3 : STATE & LGA LEVELS WORKSHOP PROGRAMME SUSTAINABILITY OF JIGAWA CDTI (LGA) Feedback/planning Meeting AGENDA DAY 1: 29 November 2004 ITEM ACTIVITY TIME FACILITATOR 1. Registration of Participants 8.30-10.00 Secretariat 2. Introduction of Participants 10:00 – 10:15 Abdul Hadi 3. Welcome and Opening Remarks 10:15- 10:20 Director PHC 4. Introduction to the workshop What are the objectives? What is sustainability? Methodology for Evaluation 10:20 –10:30 Prof. O. O. Kale 5. Tea Break 10:30 –11:00 Abdul Hadi 6. “Feedback” on achievements, issues and lessons from the evaluation on the sustainability of Jigawa CDTI project at Community Level, FLHF level, LGA Level and State Project Level 11:00 – 12:00 Cyrille Evini Esther Oyeyi Nicholas Ogweng A. O Abu O. Oladele O. O. Kale 7. SWOT Analysis 12:00 – 12:30 Saka 8. Discussions on problems identified and the solutions to these problems using SWOT analysis in groups: Planning/Integration/Leadership/Monitoring & Supervision Mectizan/Finances/Training & HSAM Transport/Human Resources/Coverage 12:30 – 13:00 Saka 9. LUNCH 13:00-14.00 Abdul Hadi 10. Group work by LGA: Identifying the SWOT for all the groups of indicators 14:00 –17:00 Saka 11. Tea Break 17:00 – 17:30 Abdul Hadi 12. Steps in planning for sustainability in CDTI 17:30 – 18:00 Cyrille Evini 13. Closing remarks 18:00 – 18:05 Director PHC 14. Way forward, APOC NGDO partner MOH 18:05 –18:30 Evaluation Team NGDO Rep. Director PHC 42 STATE & LGA LEVELS WORKSHOP PROGRAMME SUSTAINABILITY OF JIGAWA CDTI Feedback/planning Meeting AGENDA Day 2: 30 November 2004 S/No ACTIVITY TIME FACILITATOR 1 Introduction of the chair 09:00 – 09:05 Director PHC 2 Summary of previous day’s workshop proceedings 09:05 – 09:15 Saka 3 Discussions: • What is a sustainable CDTI Plan? Features? Format? Characteristics? 09:15 – 10:00 Evini 4 Coffee break 10:00 – 10:30 Abdul Hadi 5 • Group work: Developing sustainable plans for the LGAs and State of Jigawa 10:30 – 13:00 Team of evaluators 6 Lunch break 13:00 – 14:00 Abdul Hadi 7 Group work continued 14:00 – 17:00 Team of evaluators 8 House keeping matters 17:00 – 17:30 Chair 43 STATE & LGA LEVELS WORKSHOP PROGRAMME SUSTAINABILITY OF JIGAWA CDTI Feedback/planning Meeting AGENDA Day 3: 01 December 2004 S/No ACTIVITY TIME FACILITATOR 1 Introduction of the chair 09:00 – 09:05 SOC 2 Summary of previous day’s workshop proceedings 09:05 – 09:15 Saka 3 • Introduction to the day’s proceedings 09:15 – 10:00 Evini 4 Coffee break 10:00 – 10:30 Abdul Hadi 5 • Group work: Developing sustainable plans for the LGAs and State of Jigawa 10:30 – 13:00 Team of evaluators 6 Lunch break 13:00 – 14:00 Abdul Hadi 7 Group work continued 14:00 – 17:00 Team of evaluators 8 Review/Endorsement of Plans 17:00 – 17:30 Chair 9 House keeping matters 17:30 – 18:00 Chair 44 Annex 4 : REPORT ON FEEDBACK/PLANNING WORKSHOP FOR JIGAWA STATE CDTI PROJECT A three-day feedback and planning workshop was held for both State and LGA personnel involved in the CDTI programme. The principal aim of the workshop was to give direct feedback, to the officials of the project at the State and LGA levels, on the findings and recommendations of the APOC Evaluation Team. The workshop was also used as a forum for the development of a three-year CDTI sustainability plan for both the State and LGAs. . The workshop was held in Dutse, the State Capital of Jigawa, from 29 November to 01 December 2004. Apart from the evaluation team, the following participants came from all the eight APOC-supported CDTI-implementing LGAs:  Chairmen of the three LGAs visited  Representatives of Chairmen of the other 5 CDTI LGAs  Four members of SOCTs  Eight members of LOCTs  Eight LGA PHC Coordinators Day One: November 29, 2004 The workshop started with opening remarks made by the PHC Executive Secretary who chaired the meeting and workshop. In his opening remarks, the chairman explained the reasons for the present evaluation exercise and impressed on the participants the need to be present throughout the period of the workshop. The APOC’s evaluation team leader, Professor O. O. Kale, then gave a succinct review of the rationale, background and objectives of the evaluation process and the workshop. Dr. M. Evini, a member of the evaluation team next took the meeting through the methodology of the evaluation. Thereafter the findings of the evaluation were presented to the participants. Presentations were made as follows:  Community level findings: Dr Esther Oyeyi  FLHF level findings: Dr Nicholas Ogweng  LGA level findings: Mr. A. O Abu  State level findings: Professor O. O. Kale  Reacting to the report, participants sought to know the fate of CDTI after APOC pulls out given the perceived prevalent weaknesses of the project. Dr. Saka and Prof. Kale in response reassured participants of the continuous support from the NOCP and the NGDO partner after APOC. After these the concept of SWOT was presented. After explaining the meaning of SWOT, Dr Saka sought to know if participants had had any experience with such concepts. He thereafter got participants to look at activities being implemented at their various levels of intervention and indicate what they themselves considered were weaknesses, strengths, opportunities and threats to sustaining CDTI implementation post-APOC. Thereafter the meeting adjourned. 45 Day Two: November 30, 2004 The following day’s session started with Mr. Evini reminding participants of the objectives of the meeting. He then explained that the subsequent steps should be to make proper use of the evaluation team’s findings and the SWOT analytic exercise. This facilitates the process of making effective changes that will help overcome the weaknesses and threats identified. Dr. Evini also explained that participants need to define the resources that they will need to implement the changes; the sources of the resources so that planning takes into account resources that will be available. Participants then retired into their LGA groups to work on their sustainability plans. The assignment was to complete the SWOT analysis that they had started the day before and then to: 1. propose solutions to the weaknesses and threats, 2. define funding sources and amounts that they expect they will have in implementing CDTI for the coming years; 3. define new objectives based on the findings and SWOT analysis 4. develop the year one sustainable plan for CDTI in their various levels The whole of day two afternoon was used in developing sustainable plans. The evaluation team facilitated this process. By the end of Day two, the group from Birniwa LGA presented its plan. Appropriate recommendations were made by both the participants and the evaluation to help not only the presenting group but also all the groups in fine-tuning their plans. Day Three: December 01, 2004 On the third and final day, the SOC, chaired the workshop. The agenda of the day started with participants concluding their previous day’s group work, on the development of sustainable plans. This was followed by presentation and discussion of plans developed for each LGA. The workshop continued with presentations on practical steps to implement these plans as well as the way forward for the prepared plans. 46 Annex 7: List of Participants No Name Position/Affiliation 1 Mohammed Aliyu Idris Oncho Coordinator, Burinwa LGA 2 Sani Barde Kudai PHC Coordinator, Taura LGA 3 Sabo Sai’du BLK Focal Person , B/ Kudu LGA 4 Salisu Manu Lofungu Focal Person , Taura LGA 5 Hassan Abdulkadir Focal Person , K/ Hausa LGA 6 Lawal Bala SOCT, Primary Health Care Agency 7 Garba Ubali SOCT, Primary Health Care Agency 8 Magaji Abdulhamid SOCT, Primary Health Care Agency 9 Aminu U. Abdulrahman SOCT, Primary Health Care Agency 10 Alh. Sani Usman Chairman, Taura LGA 11 Talatu Yangayamu Councilor, Health, Taura 12 Garba Bako PHCC/HOD Health, Gwaram 13 Hajiya Iyan Ismail Councilor, Health, K/Hausa LGA 14 Marwa Bako PHCC, B/Kudu LGA 15 Bashir Abdu Ringim Treasurer, Ringim LGA 16 Dansalla Zakar Treasurer, K/Hausa LGA 17 Ado Iguda Coordinator, Gwaram LGA 18 Adamu Abdulaih PHC Coordinator, Birninkwali LGA 19 Abdulmasid Adamu Treasurer, Taura LGA 20 Alh. Ahmad Rufai Chairman, Gwaram LGA 21 Moh’d Mustapha E.S/DPHC 22 Sani Galadima SOCT, Jigawa State 23 Garba Moh’d Bulanga DFH PHCA, Jalingo 24 Muhtar Usman Babara PHCC Dutse 25 Bello Wangar Focal Person, Dutse 26 Shehu Mohammed Treasurer, B/Kudu LGA 27 Dr. Y. A. Saka Deputy National Coordinator; Evaluator 28 A. O. Abu State Coordinator, Edo State; Evaluator 29 Evini Cyrille Evaluator; WHO Cameroon 30 Nicholas Ogweng Evaluator; WHO-APOC, Uganda 31 Esther Oyeyi Evaluator; WHO-APOC, Nigeria 32 Oladele Kale Team Leader, WHO-APOC, Evaluation Team 33 Abdulhadi Hassan State Coordinator 34 Musa Suleman PHCC, KGIM LG 35 Ibrahim Alfa Councilor, Daugama 36 Abdulwahab Sueliman Chairman, K/Hausa 37 Moh’d Moh’d PHC Coordinator, K/Hausa 38 Adamu Bello Treasurer, Dutse 39 Ibrahim Y Chairman, Dutse 40 Moh’d Usman B Councilor, B/Kudu 41 Shehu Moh’d Treasurer, B/Kudu 42 Saádu Ayuba Chairman, B/Kudu 43 Musa Adamu Treasurer, Gwaram 44 Alh. Ali Councilor, Gwaram, 45 Ahmed Rufai Chairman, Gwaram 46 A. Ali Agala Councilor, Gwaram 47 A. Baffa Abubakar Treasurer, Birniwa 48 Sani Sale Chairman, Kaugama 49 Ahmed Sale K Chairman, Birniwa 50 Sabo Haruna LGA Coordinator 51 Gambo Ahmed Councilor Kaugama 52 Ibrahim Alfa Treasurer, Kaugama 53 Bala Adamu Councilor, Dutse