Assessment of the sustainability of the Plateau State CDTI project, March 2003

World Health Organisation African Programme for Onchocerciasis Gontrol Assessment of the Sustainability of the Plateau State CDTI project March 2003 Eleuther Torimo (Teom l*ader) Obiomo Nwoorgu Koyode Ogungbemi Horrieth Homisi Sundoy Isiyoku Abel Eigege 24-:.oiVIP-o3 ii ) ,f, u rrt i I I li i ,!(',rI r, gr*tg ;UlJ."l l t:

Index Page Abbreviations/ acronyms and acknowledgements Executive summary Introduction and methodology Findings and recommendations 1. State level 2. LGA level 3. District/ health centre level 4. Village level 5. Overall sustainability grading for the project and way forward 6. Appendix: I Agenda, Plateau State planning Meeting I1 Agenda, LGA Planning meeting (iii) State/LGA Planning meeting program. (iv) State Level, Problems and solutions (v) LGA level, problems and solutions (vi) FLH F/Community, problems/solutions (vii) Ust of people interuiewed. (viii) Ust of evaluators. Detailed flndings 1. State level 2. Health district/ LGA level 3. Sub-district/ first line health facility level 4. Community level 3 4 5 9 11 16 22 26 28 30 31 3? 33 35 36 37 39 ) Abbreviations/ acnonyms APOC CDD CDTI CHEW DHS FLHF GRBP HOD HSAM IEC LF LGA LOCT MOH MOF NGDO NOCP NOTF NPI PHC RBM REMO soc SOCT SPO UNFPA WHO zoc ZOTF WR African Programme for Onchocerciasis Control Community Directed Distributor Community Directed Treatment with Ivermectin Community Health Extension Worker District Health Supervisor First Line Health Facility Global 2000 River Blindness Programme Head of Department Health Education, Sensitisation, Advocacy and Mobilisation Information, Education and Communication Lymphatic Filariasis Local Government Area L Onchocerciasis Control Team Ministry of Health Ministry of Finance Non-Governmental Development Organisation National Onchocerciasis Control Programme National Onchocerciasis Task Force National Programme on Immunisation Primary Health Care Roll Back Malaria Rapid Epidemiological Mapping for Onchocerciasis State Onchocerciasis Co-ordi nator State Onchocerciasis Control Team State Programme Officer United Nation Funds for Population Activities World Health Organisation Zonal Onchocerciasis Co-ordi nator Zonal Onchocerciasis Task Force World Hea lth Orga n isation Cou ntry Representative Acknowledgements We would like to thank the following persons for their assistance and support: . The staff at APOC Headquarters in Ouagadougou: The Director Dr S6k6t6li, Dr Amazigo, Mr Aholou, Mr Agbonton and others for overseeing and facilitating the mission. . The WR and his staff in Lagos and Jos for administrative and other support. . The NOCP Zonal Co-ordinator: Mrs. Patricia Ogbu-Pearce for her extensive support to the team . Staff of the Plateau State Ministry of Health: The Hon. Commissioner, Barrister John Magaji; Director PHC, David S. Belin; State Oncho Co-ordinator Henry Filda and his team, who despite the strike actlon in the State civil service ensured that the team was able to carry out its assignment. . GRBP Country Representative, Dr. E. Miri and his staff for providing office space and other suppott. . LGA Chairmen, Health workers and community members in Bokkos, Jos East and Pankshin LGAs who provided information and suggestions related to CDTI. 3 Executive Summary The Plateau CDTI project has been supported by APOC since April 1998 and it is in its last year of agreed funding from APOC. An evaluation of the sustainability of the pCIect was carried out between 17b February - 2nd March 2003 by a team of evaluators from Nigeria and Tanzania. The evaluators were charged with the following tasks: - 1. Evaluation of the sustainability of the Plateau State CDTI project, including field assessments in selected LGAs, Districts and communities 2. PreparaUon and conduct of feedbac(planning meetings with the State and Local Government AuthoriUes (LGA) 3. Analysis of data and preparing a report. Using a multistage sampling approach, information was collected using the 4 standardized sustainability evaluation instruments related to the State, LGA, FLHF and community levels. Findings At the Community level, geographic coverage is 1000/o while therapeutic coverage is over 650lo with an increasing trend. Communities like taking mectizan and are aware that it has to be taken annually for a long time and know its benefits. "The drug makes us feel strong, improves vision, prevents blindness and expel worms from the body". Community leaders and CDDs jointly take responsibility for distribution of Mectizan within their communities as evidenced by their active role in community mobilization, decisions on timing and treatment venue, regular consultations with CDDs and suppoft for CDDs. Census and registration are carried out by CDDs before each registration. CDDs submit repofts and drug balances to first line health facilities. Treatment of Lymphatic Filariasis (LF) has been integrated with that of Onchocerciasis and on the whole is doing well. On the negative side CDDs were not taught how to determine amount of drugs to be requested for their communities. Lack of a raUonal criterion for determining quanUty of drugs needed often resulted in surplus drugs in some communities and shortage in a few. CommuniUes are aware of their roles to suppoft CDDs and agreed to do so but have often failed to do so. Attrition rate of CDDs is high (about 500/o). The ratio of households covered by CDDs is higher than that recommended by APOC. IEC materials for HSAM are not adequate. Finally there is some confusion among community members and CDDs, in communities where both Lymphatic Filariasis (LF) and Onchocerciasis are being co-treated, on what causes what disease and what treatment is for which. Reasons for this confusion need to identified and addressed. IntegraUon of the training tools for the two diseases might ameliorate the situation. At first Level Health Facility, there is a good Mectizan inventory of drugs received and balances returned. FLHFs seem to be a weak component part of the CDTI system. Most staff at this level do not have adequate knowledge and skills for CDTI. CDTI training for health workers is restricted to one per health facility, usually the District Health Superuisor, who is the most 4 senior officer. CHEWs are more stable, not often transferred out of their stations but are not trained in CDTI. There is confusion (to evaluators as well as CDTI staff at all levels) over CDTI leadership at this level. There are five LOCTs, one in charge of each of the flve health districts and there are also five DHSs, one in each of the health districts. LOCTs are often junior in position to DHS yet they are expected to supervise CDTI in FLHFS. All health staff at this level should be trained in CDTI. At the LGA Level, LOCT/ Oncho focal persons are skilled and knowledgeable about CDTI. But during field visits SOCTs appeared to be more informed about issues at FLHFs and community levels than LOCTs, which raises doubts about the adequary of their (LOCTS) empowerment by the State level to direct CDTI activities at the LGA level. Ownership and leadership provided by LGA management is weak. For example PHC Coordinators and LGA Management are not fully involved in CDTI process. There has not been a budget for CDTI activities in LGAs since the inception of the program. LOCTs did not know how to use the census figures collected from CDDs to calculate quantity of mectizan to be requested for annually. There is need for rethinking and recasting the packaging of IEC materials to avoid confusion in communities where both LF and Oncho ae being treated together. CDTI data is not part of the Health Management Information System. The State level has spearheaded the implementation of CDTI all levels. Extensive suppoft and leadership has been provided by GRBP particularly in the early years of the program. The State has assumed increasing leadership of the project, most marked in the last year. To enhance sustainability, leadership needs to be strengthened through involvement of other partners in the State Ministries e.g. MOA, Ministry of Chieftainry and Local Government Affairs, Ministry of Planning and Finance. To enhance flnancial management a CDTI accountant should be appointed and trained to support the program. Some N2m of APOC funds has been released this year (June 2002 and May 2003). Information provided during interuiews show that the funds are adequate for essential CDTI activities. The 1997 "Proposal" spelled out contributions to be made by different partners. For example it is indicated that the contribution from State would increase annually as that of APOC decreases and ends at the end of the fifth year. To date the State has made no financial contribution. The State Deputy Governor, Commissioner for Health and Local Government Chairmen gave oral assurances that adequate funds will be provided to continue CDTI when APOC funding ends. The evaluation team learnt of a high level effort to obtain CDTI funds released before the evaluation team left the State but the effort was not successful. The evaluation team was informed of several initiatives to enhance integration of CDTI into PHC, including a Federal level initiative to integrate services in selected Wards. The evaluation team recommends strongly that a three-year sustainability plan be developed as a matter of urgency, building on the one-year plan prepared during the feedbaclVplanning meeting at the end of evaluation. The evaluation team was surprised to see the extent to which many leaders pafticularly LGA level did not know much about CDTI. The State should issue a directive that will clearly deflne the roles of different levels in CDTI implementation. 5 Overall assessment of project and way-forward Using the standard four-tier schedule for ranking projects evaluated: fully, High, Moderate and low, the evaluaUon team judges the Plateau State CDTI to be at the level of Moderate. The project is potentially sustainable as evidenced by the excellent coverage, high level of community leadership and approval and availability of mectizan everywhere. But this project requires rethinking and mobilizaUon of high-level decision-makers to provide financial resources to sustain CDTI. It is intriguing to the evaluation team that at the end of the fifth year the State has not made a single annual contribution on the lines agreed with APOC. Other important issues that need rethinking and action include: High attrition rate of CDDs and empowering of LGAs, through training and decentralization of authority from the State level to LGAs and FLHFs. Using the Abuja grading of: Fully sustainable; On the way to sustainability and not on the way to sustainability, the evaluation team concludes that the Plateau program is on the way to sustainability. Two feedback/planning meetings (of one and two days duration, for LGA and State management staff respectively) were successfully carried out. Way forurard Recommendations of the evaluation team for each of the four levels are contained in pages 13 (State), 19 (LGA), 24 (FLHF) and 27 (Community level). State and LGA leadership indicated that immediate and rapid action will be taken in a number of interrelated key areas. Foremost is the implementation of the one-year plan of action (developed during the briefing and planning meeting) and development of a three year CDTI sustainability plan. The process of developing the three-year plan has potentials for mobilizaUon of all partners. The evaluation team recommends that APOC follow closely the development of the three-year sustainability plan and implementation of the evaluation recommendations and provide support in key areas. It is hoped that the target of completing the three-year plan by Mid April and making it available to APOC will be met. 6 Introduction and Methodology 1. Introduction Plateau state is located in the Middle -Belt of Nigeria and it is bounded to the South West by Nasarawa state, while to the North West and North East are Kaduna and Bauchi states respectively with Taraba state to the South. The state has an estimated population of 3.2 million people. The people of the state are predominantly farmers living in scattered rural settlements. Most of the roads are bad, rough and sometimes not passable due to flood and lack of bridges, Also, the rocky nature of some LGAs is also a serious impediment to effective transpoftation. The state enjoys two types of seasons, the raining season, (May - October) and the dry season (Nov. - April). Most treatment/distribution activities in the state are preferably carried out during the dry season when farmers have less to do in their farms. The State is divided into 17 political Local Government Areas (LGAs). CDTI program is being executed in flve LGAs: Bassa; Bokkos; Jos East; Kanke and Pankshin. The remaining 12 LGAs with 313 villages are hypo-endemic and treatment is mostly carried out in health facilities at the village level with the assistance of Global 2000 and the State Ministry of Health. APOC financial assistance to the state CDTI Project stafted in April 1998. The project is in its fifth and final year of APOC flnancial assistance. (June 2002 - May 2003). According to the 1997 proposal to APOC, the project in their fifth year should have put in place adequate infrastructure and resources to sustain CDTI. The evaluators were charged with the following tasks: - 1. Evaluation of the sustainability of the Plateau State CDTI project including fieldwork in selected LGAs, Districts and communities 2. Preparation and conduct of feedbac(planning meetings with the State and Local Government Authorities (LGA). 3. Analysis of data and report writing. 2. Methodology . Evaluation question'How sustainable is the Plateau State CDTI project . Design; Cross sectional, participatory and descriptive . Population:The Plateau State project, including: its SOCT; its NGDO paftner; its LGAs with their LOCTs; the project communities/villages and their CDDs . Sampling: Details of the sampled districts and villages are contained in table 1 below. 7 Sampling The sampling was done purposively, based primarily on the coverage (geographical & therapeutic). Secondly the sampling was based on the: - . Endemicity level (the sample contained both hyper and meso endemic areas). . Geographical spread: sampled villages were from different clusters in the project area. In Plateau State there are 5 CDTI LGAs divided in three clusters; Cluster one consists of Bassa & Jos East. Cluster two consists of Bokkos and cluster three consists of Pankshin & Kanke. One LGA was picked from each cluster based on the above sampling criteria. The LGAs with the best average therapeutic coverage are Pankshin (96.8%) and Kanke (85.4o/o); Pankshin was picked because of its best coverage. LGAs with medium coverage are Bassa (82.4o/o) and Jos East (86.60/o); Jos East was picked because of its medium coverage. Bokkos LGA was picked because of its lowest coverage (79o/o). Details of the sampled Districts, Villages and FLHF are illustrated in the table on page 10 Sources of information Information was collected from interuiews, verbal reports and documents. Various categories of people were interviewed at the Zone (Co-ordinator), State (SOCT leader, team members, poliry-makers, management staff,), LGA (LOCT leaders, management staff, heath technical staff), the fronfline heath facility level and the community. Information was recorded on the evaluation instrument and discussed extensively before grading the performance of the relevant level on the indicator. Analysis Based on the information collected, each indicator is graded on a scale of 0-4, in terms of its contribution to sustainability. The average sustainability score' for each group of indicators is calculated, for each level. A graph was plotted for the level being assessed. The evaluators discussed emerging problems and possible solutions and made recommendaUons on remedial action related to each of the four areas, 8 TABLE 1: DISTRIBUTION OF SAMPLED DISTRICTS &VILLAGES LGA LGAs (Rx COVERAGE RATE) DISTRICT FLHF VILLAGE SELECTED (TREATMENT COVERAGE RATE) 1 BOKKOS LOWEST COVERAGE (79o/o) DAFFO (93.3%) HIGHEST COVERAGE) TOFF (98.6olo) KARFA (80.7o/o) MUSHERE (80.4olo) LOWEST COVERAGE KAWEL (1000/o) SAT (47.2o/o) 2 JOS EAST MEDIUM COVERAGE (79o/o) FEDERE (99.8o/o) HIGHEST COVERAGE) FEBAS (1000/o) GANDI (98.30/o) FOBUR (85.90/o) LOWEST COVERAGE NABATONG (100o/o) SABON GARI (51.8olo) 3 PANKSHIN HIGHEST COVERAGE (99.5olo) BAI-ANG SHIPANG (99. 5olo) HIGHEST COVERAGE) KAD\IS (100o/o) NYELLENG (98.60lo) JIBUK (73.60/o) LOWEST COVERAGE JING (99.2olo) TILLENG (73.60/o) 3 PANKSHIN HIGHEST COVERAGE (ee.s%) BALANG SHIPANG (99.5%) HIGHEST COVERAGE) KADYTS (100%) NYELLENG (98.6%) JIBLIK (73.60/o) LOWESTCOVERAGE JrNG (99.2%) TILLENG (73.60/o) 9 4.5 4 3.5 2.5 3 FINDINGS AT THE STATE LEVEL Fig. 1: Plateau CDTI Project: Sustainability at State Level 0 Group of lndicators *."" *.o.*N .."."" C .o'"r.t .%-.".' -.r"t" ".".-- tt -c, .9) o 3 o o)(U o 2 .5 1 1 0.5 trSeriesl 4 3 33 2.4 0.98 ; Plannino (2.4 Moderatelv): Check whether there is a year plan for onchocerciasis control appearing as paft of overall plan for the health service. Whether this plan contains details of activities needed for effective oncho control. Whether all partners including government, NGDO, UN Agencies are involved in the overall process. Whether specific planning for sustainability has taken place for the period after APOC funding is withdrawn. Control of river blindness appears as part of the annual Development Plan of Ministry of Health. In relation to APOC, MOH prepares yearly action plans based on the S-year plan developed with and agreed by all partners (1997). The plan, June 2002 - May 2003 contain key elements for an effective CDTI program including training, advocacy, and superuision/monitoring and mectizan distribution. The plan spells out contributions to be made by different partners: MOH; APOC and GRBP. A plan for sustainability is yet to be developed. A meeting of LGA chairmen convened in January this year deliberated on this and related issues. It was agreed that each LGA should take ownership of CDTI and provide funding. Follow up of the meeting has not taken place. Few LGA staff in the LGAs knew about the meeUng. l0 Monitorino/Supervision (2.3 Moderately) Check whether state level staff is being used appropriately for monitoring and supervision. Whether M/S is being planned and carried out in an efficient and integrated manner, Whether there is routine process of management of problems and successes detected during monitoring. Each of the flve SOCTs supervises assigned LGAs quarterly. SOCTs and Global 2000 staff also carry out spot check on District Health Supervisors (DHSs) and First Level Health Facility staff. Management problems such as those related to combined treatment and dosage are dealt with promptly. While superuision of CDTI is integrated with LF and control of Schistosomiasis at LGA and lower levels, it is not the case at the State level. SOCT supeMsion/monitoring is not synchronized with that of GRBP. Mectizan - Procurement and Distribution (2.6 Moderately): Check whether sufficient mectizan is being ordered, stored and distributed within the government system at the State level. Ordering mectizan is done by GRBP. The system operates well. Storage and distribution is mostly within government system. GRBP delivers mectizan to SPO who in turn delivers the drug to LGA, LGA to FLHF or FLHFs collect from LGA. Census is carried out yearly but there is no clear indications or evidence that calculations of mectizan requirement are based on APOC formula. Training & Health Education Sensitization, Advocacy and Mobilization (HSAM) (3 Hiohly): Check whether staff at the State level is being used appropriately as trainers. Whether training is planned and carried out in an efficient and integrated manner. Whether staff at the state level are planning and carryinq out HSAM activities efficiently Staff at State level routinely only train LOCTS. LOCT staff in turn train and superuise FLHF staff and FLHF staff train CDDs. CDTI training is provided for newly transferred staff and is carried out together with that for LF and Schisto control. Health Education, sensitizatlon, advocacy and mobilization activities are well planned. Staff reported shoftage of IEC materials. Inteoration of Suopoft Activities (2.6 Moderatelv) Check whether various program support activities are planned and carried out in an integrated manner, CDTI actlvities such as supervision, monitoring and training have been mostly decentralized to lower levels - LGAs and frontline health facilities. This step helps to facilitate integration. Training for CDTI, LF and Schistosomiasis take place at the same time and place. Mectizan and Albendazole are delivered together. But integration of CDTI within the overall PHC at State level remains weak. A new Federal level initiative on Model Wards is expected to find better and effective ways for enhancing integration in PHC. ll Financial Resources (0.98 Sliohtly): Check whether appropriate amounts are budgeted for planned oncho control activiUes, whether government is budgeUng and disbursing sufficient amount yearly for this and if there are shortfalls whether other sources of funding are used for this gap and finally whether oncho control funds from the are A total of N2m has been released by APOC for this year (June 2002 and May 2003). Information provided during interviews shows that available funding is adequate for essential CDTI activities. GRBP has supported training and logistics activities and provided backstopping whenever release of APOC funds is delayed. Control of APOC budget is good. The process for release of funds involves: SPO; Director PHC; Commissioner for Health; Governors office and MOF. The State has not provided financial contribution to the program since the inception of APOC. Other health programs like NPI, UNFPA, RBM obtain counterpart funding from MOH. Transpoft and other Material Resources (3 Hiohlv): Whether adequate transport and material resources are available for CDTI activities, whether they are adequately and appropriately maintained, whether appropriately managed and used and whether there are plans to replace them when APOC suppot comes to an end. Two vehicles (the second one is provided by GRBP as needed) are available. Vehicles are considered adequate for the next 2-3 years provided are adequately maintained. The use of vehicles is authorized by SPO and a logbook is used. A photocopier, computer and a printer are available. There are no plans for replacement of vehicles. Human Resources (3 Hiohlv): Whether adequate transport and material resources are available for CDTI activities, whether they are adequately and appropriately maintained, whether appropriately managed and used and whether there are plans to replace them when APOC support comes to an end. Staff at this level are skilled and considered adequate (six SOCTs including SPO). For sustainability an account on a part-time or full-Ume basis needs to be available in the program. The gap is partly filled by GRBP. The high attrition rate of CDDs is a statewide issue and it is important to document experiences and lessons from different areas as basis for future solution. Another issue the team took note of relates to the provision of top ups to salaries of selected staff in the APOC budget. The team wandered how this would be sustained and effect on staff motivation when APOC suppoft ends. Coverage (4.0 Fullv): Check whether all projects in the country have a satisfactory therapeutic coverage rate. Coverage is good, above 80o/o in all the communities. Concerted effort from all fronts will be necessary to maintain and preferably increase it. t2 Recommendatlons at this level Recommendations Implementatlon Planning The state should issue a directive to all partners indicatlng clearly the role to be played by different partners in sustaining CDTI. The State and other partners should develop a three year plan for sustainability on the lines agreed at the planning meeting as a matter of urgency. The plan should be realistic, based on government resources and other sustainable sources. The State plan should build on and respond to the needs of the LGA plan. Prion$ HIGH Indicators of success A directive available to all partners, Who to take action Director PHC Deadline for completion Middle March 2003 Priority HIGH Indicators of success Three-year sustainability plan is prepared. Who to take action Director PHC Deadline for completion Middle April 2003 Monltoring and Supervision The State level should strengthen and empower LGA staff to enable them to play a leading role in CDTI. Prion$c HIGH Indicators of success, Training of LGA leaders and staff particularly at FLHFs (Reports on training carried out) Who to take actiont SOCT and LOCTS. Mectizan Training of SOCTs and LOCTs outlined above should include a strong component of skills to calculate Mectizan requirement. The State should ensure the early supply of Mectizan using the necessary requisition forms for early distribution to the communities. Priority HIGH Indicatorc of success Calculation of mectizan requirement included in Training programs. Who to take action; SOCT and LOCTS. Training & HSAM Integration of training and HSAM materials for CDDs and communities. A redesigning of training material to prevent confusion and mix up about LF and Oncho incl udi ng simplification of trai ni ng i nformation. CDTI to be included in curriculum of school of health technologies Prionty. High Indiators of success. Integrated training materials for CDTI, LF and Related diseases available, Who to take action SOC-| Deadline for completion Integration of Support Activities Priority. Medium l3 Incorporate CDTI as component of the Model Ward initiative. Indicators of success CDTI is one of the elements of the Model Ward initiative. Who to take action. Dea d I i ne for com pletion: Finance MOH should carry out intensive advocacy to ensure that adequate funds to sustain CDTI including State and LGA budgets are released. Priorifin High Indicators of success. Adequate budgets for CDTI are included in State and LGA budgets. Who to take action Commissioner, PS, PHC Director, NOCP, and SOC[. Deadline for completion April 2003 Transport and Other Resources Plan for CDTI to share use of transport from different sources. Strengthen maintenance of vehicles. Priority, MEDIUM Indicators of success. Availability of adequate transport, Who to take action Director, PHC, PS Deadline for completion end of next distribution Human Resources: An accountant should be appointed and trained with the necessry skills in the MOH. This may not necessarily be a full time appointment. Identification and documentation of experiences and lessons on how to address the issue of attrition of CDDs Priority, MEDIUM Indicators of success, Report on experiences and lessons on retention of CDDs available, Who to take action SOC[. Deadline for completion June 2003. Coverage Maintain the high level of coverage. t4 FINDINGS AT LGA LEI'EL 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Fig.2z Plateau CDTI Proiect. Susutainability at LGA level t +a3 .9 o! o ED .EL o -""""-..". n."".t"- x.""*t ^-"" a.-os -.."' Groups of tndicators "*" a"*" "".""- Plannino (1.3 SliohUv): To check whether the year plan for CDTI appears as paft of an overall written plan for activities at LGA level. CDTI ls lntegrated lnto the overall PHC written plan. Year plan prepared by LOCT and submltted to PHC Coordlnator for vetting. This has provision for all key CDTI acHvities: Mectizan supply, tralnlng, HSAM and monitorlng/superulslon. Community requlrements taken into consideraton ln plan prepared e.g. time of distributions and mobilization. LGA Chairmen and LOCTs attended a CDTI sustalnabllity meeting ln January. Issues dlscussed Included: Advocacy, assistance from LGAs and communlties, counterpart fundlng and the comlng of the evaluaton team. No funds have been release for CDTI acUvities following request from CDTI focal person in the LGA even though CDTI is integrated into the PHC plan. 4 3.45 2.5 1 5 l5 Leadership (2.3 Moderately): To check whether the LGA health management team is taking full responsibility of the implementation of CDTI. In some LGAs the management team (LGA chairmen and other Leaders) are actively interested and involved in initiating key CDTI activities. Oncho Coordinator/LOCT focal person at the LGA level is the focal person for all CDTI activiUes at this level. PHC Coordinators, LOCTs and District Superuisors are the only individuals involve in CDTI activiUes in some LGAs. Oncho focal persons at the LGA level depend on the State (SOCIs) for all CDTI activities including finances. State has and conUnues to direct CDTI activities. State level staff have not empowered the LGA level staff. Monitorino and Superuision (2.3 Moderately): Check whether routine data concerning CDTI activities are being transmitted from LGA within government system. Whether persons responsible at LGA are efficienUy supervising activities at FLHL level in an integrated manner and whether there is routine management of problems and success indicated the Problems idenUfied during monitoring are dealt with at each level e.g. CDDs report drug shortage to village heads, village heads then report to FLHF staff, who then repoft to LOCT at the LGA and then LOCT repoft to SOCT at the State level. Successes are noted and communicated verbally to individuals concerned. All treatment summaries, Mectizan bulk transfer forms are submitted directly to SOCT at the State level. CDTI superuision is integrated with other disease programs e.g. Schisto, LF, and Malaria. LOCTs draw up monthly itineraries for this purpose. Transpoftation is shared for superuisory visits for all diseases. LOCT visits communities when there are problems e.g. CDDs not compensated by communities. Monitoring reports not available at all levels. In some Lags some of the superuisory visits by LOCTs at the FLHF level appear to be high. Transmission of data is not part of government Health management information system i.e. does not pass through the normal MIS. Some LOCTs collect routine CDTI data directly from the FLHF level and communities (CDDs). Mectizan (2.5 Moderately): Check whether sufficient Mectizan is being ordered and received yearly and in good time. Whether Mectizan is being collected, stored and effectively administered through the government system. Census is carried out annually by CDDs before distribution process. Mectizan received from State to LGA without previous requisition order forms from FLHF or communities. SOCT supply Mectizan annually to LOCT, staff at FLHF collect Mectizan for communities from LOCT at LGA level. CDDs collect Mectizan from FLHF for their communities. Mectizan at LGA level is usually distributed within 2 days and not stored at the LGA level. Bulk receiving forms were available at all levels but ordering forms were absent. Census figures do not seem to be used in calculation of annual Mectizan requirement. Finances for transpoftation of Mectizan from LGA to FLHF provided by APOC supported. Mectizan received from State to LGAs without previous requisiUon order form or request from FLHF or communities. It was obserued that staff at various levels t6 (LOC[, FLHF and CDDs) were unable to calculate quantlty of Mectizan required annually based on community census. Shortages reported in 2000 due to presence of refugees from other states but in one incident, team found that about 2500 tablets of Mectizan were given out in 2002 instead of about 3800 administered in 2001. In some cases, excess Mectizan tablets were returned. Training and HSAM (2.8 Moderately): Check whether staff members are being used appropriately as trainers. Whether training is planed and canied out efficiently and in an integrated manner. Whether staff is planning and carrying out HSAM activiHes efficiently. LOCT trains District level personnel and occasionally assists in training at community level. Staff at FLHF train CDDs. Training is conducted annually for 2 days for new staff and retraining for old staff and CDDs (before distributions). Health education on CDTI is integrated with that for LF and Schistosomiasis control. This takes place before, during and after distribution. LOCf undertakes advocacy visits to mobilize LGA management staff for CDTI. Training materials on CDTI were scanty at all levels. Some staff at FLHF level were not very knowledgeable about CDTI. State provides all training, supervision and monitoring and workshop materials for all workshops and training organized at all levels. Financing (0.O8 Slightly): Check whether appropriate amounts are budgeted for planned CDTI activities at the LGA level. Whether LGA government is budgeting and disbursing increasing amounts for CDTI yearly. Whether in case of a deficit between estimated costs and amount provided by government, dependable provision is being made to meet it and whether funds disbursed for CDTI from budget at LGA level are efficiently managed. Although LGAs have no budgetary provisions, funds from APOC and other sources have been adequate for CDTI activities. Salaries of LOCTs and FLHF staff are paid by the LGA. No financial budget for CDTI program by LGA management including release of counterpart funding since inception of CDTI activities 5 years ago in LGAs. No CDTI related expenditure in the past 5 years was approved at the LGA level, all CDTI activities are funded through APOC budget. Management at LGA level is not aware of funds available to them for CDTI activities annually. Such informaUon is available at the State level. APOC budget is used for all CDTI activities this includes payment of per diem to staff during training, monitoring/supervision, delivery and collection of mectizan at all levels and provision of Health education materials. Transpoft and other Materials Resources (1.5 Slightly): Check whether adequate and appropriate transport and materials are avallable for CDTI activities at LGA level. Whether transport at LGA level is adequately maintained. Whether transport available is appropriately managed and used in an integrated way. Whether there are appropriate and realistic plans for the replacement of transpoft and materials when APOC support comes to end. Transport for CDTI is considered adequate for the next 2-3 years provided necessary maintenance is carried out. Transport includes 11 motorcycles (3 APOC, 8 LF) and 35 bicycles t7 I(25 APOC, 10 LF). Transport at the LGA and FLHF used in an integrated manner for all programs e.g. CDTI activities, Schistosomiasis control, LF, malaria, NPI, etc. Trips made for CDTI purposes at LGA level are authorized by the PHC Coordinator and by the District Health Supervisors at the FLHF level. Itinerary for annual planned activities by LOCTs available. Transport provided by APOC for CDTI activities and other programs e.g. LF are used in an integrated manner for all programs activities at LGA and FLHF levels. Health educaUon materials (posters, flipcharts, leaflets) not adequate. The last time this was received was in 2001. Replacement of motorcycle spare pafts carried out with APOC fund and minor repairs by individuals. No logbook for use of transpoft by LOCTs and FLHF staff available. No plans by LGA management to replace available vehicles after APOC. Human Resources (3.45 Highly): Check whether staff at LGA level are skilled and knowledgeable regarding the implementation of CDTI. Whether are committed to their work. There are 5 LOCTs in each LGA. Most CDTI focal persons have been in place for at least 3 years. They acquired enough skills for CDTI activities. There is a better stability of staff at the LGA level compared with staff at FLHF level where some of the district health officers are only 6 months old. New staff at LGA level are trained annually. Staff members at the LGA level expressed satisfaction with their responsibilities. Some CDTI focal persons receive verbal praise for contribution towards CDTI programme. Cases of poor skill and inadequate skills found amongst some FLHF staff. Staff at LGA level involved in CDTI are owned 6 months salary arrears by LGA management. Instability of staff at FLHF level, some district health officers are only 6 months old at the present post. CDTI staff at LGA level receive allowances including top- ups after training and superuision of CDTI activities. Coverage (4.O Fully): Check whether LGA has a satisfactory geographical coverage rate. Whether LGA has a satisfactory thera rate. 100o/o of the communiUes are geographically covered with CDDs selected from each hamlet/ward in their communities. Therapeutic coverage over 80o/o in all the communiUes. CDDs move from house to house during distribution and drug reserued for at least 5 weeks for absentees. As reported by some LOCTs coverage has been increasing by 10o/o annually due to the introduction of LF program in some LGAS. l8 Recommendations Implementation Planning Three-year sustainability plan with a one year (2003) CDTI work plan to be prepared and integrated into the LGA/PHC work plan. The work plans to be prepared by the LGA management team, comprising PHC Coordinator, Superuisory Councilor for health, DFS, DPM and LOCT. The plans will focus on problems identified during monitoring and supervision exercises Priority High fndicators ofsuccess: . Avallability of year 2003 CDTI sustainability work plan. . Report of workshop where plan was developed . Report of CDTI sensitization meeting Who to take actlon . LOCT & PHC Coordinator Deadline for completion . March 2003 Leadership LGA to be empowered to play leadership role at this level, PHC Coordinator to be trained on CDTI. Priority High fndiator of Success: . Document showing reconstitution of LOCT and membership Who to take action . PHC - Coordinator and LOCT. Deadline . April 2002 Monitoring and Superuision . All CDTI supervisors should use supervisory checklists to ensure appropriate supervision is carried out . Include Oncho data as part of health management information system at the LGA level r A provision should be made for the PHC Coordinator to sign CDTI repofts before hansmission to the State. . Supervisory visits to be targeted and integrated with other health activities. Priority Medium fndbators ofsuccess; . Supervisory visit reports . CDTI data included in MIS Who to take adion . PHC-Coordinator,LOCT Dea dli n e for com pletlo n . By end of next distribution Mectizan o Trainlng of all individuals involve in CDTI at all levels on how to determine quantity of mectizan to be ordered using the census figures as recommended by APOC The LGAs should use the requisition forms developed by the State for their Mectizan allocation. Priority High fndiators for success; o Training report Who to take action . LOCT and PHC Coordinator Dea dl lne for com p letio n . End of next distribution Training & HSAM . PHC Coordinator to be involve in all training in LGA and not to be by passed . Training should be at health facilities and targeted o Guideline for training should be constantly reviewed . Indivldual staff s in all health facilities should be trained in all disease (model health center Priority High fndicator ofsudcess: Training report available and sighed by PHC Coordinator Training justifi cations Who to take action PHC Coordinators and LOCT t9 a spearheading). Advocacy and sensitizaUon meeting on CDTI to be conducted by management team once every year. a Deadline for Completion April - May 2003 Finance . CDTI technical repofis should be widely disseminated to the LGAs o LGAs to mobilize funds from other sustainable sources. . Budget in sustainability plan should be realistic and LGA budget Priority High Indicator for Success: Amount of resources mobilized annually Who to take action PHC Coordinator and LOCTs Deadline for completion March 30 (the inception of the new government) Other Resources . Log books to be provided for use of vehicles . LGAs should to develop a clear plan for maintenance and replacement of transport Priority Hiqh Indicator for success: Availability of maintenance plans Who take action PHC-Coordinator and LOCT Deadline for completion April 2002 20 FINDING AT THE HEALTH FACILITY LEVEL Fig.3: Plateau CDTI Project. Sustainability at FLHF Level 4.5 4 3.5g^ vo .9 2.5o3^ oz C')$ 1.5 4 0.5 0 o*$''"" "r.t- *".'-* n..c ^c .c"/ "t"' ..'"' "/ Groups of lndlcators PTANNING (O.66 Slightlyr: There is a written year plan for CDTI in the FLHF area. There was no written plan for CDTI activities prior to 2002. The LOCTs and State met in 2002 for the first Ume to draw up a written work plan for CDTI activities for 2003. This plan has been discussed and shared with the districts and it contains key CDTI activities, duration and budget by actiMty. LEADERSHIP (1.33 Sliohtly): The FLHF management team is taking full responsibility for CDTI at this level, in an integrated manner. FLHFs consider CDTI to be their program, implementation based on directives from LGAs, once they get the drugs. The district political leaders show interest in CDTI and pafticipate in mobllizing communities. Mectizan distribution is integrated into Lymphatic Filariasis and Schistosomiasis control program but not fully into other PHC activities. 1.6 2.5 2 0 2t MONITORING AND SUPERVISION (1.8 Moderately): Routine and necessary data concerning CDTI activities at this level are being transmitted enHrely within the government system. Health staff at this level is routinely and efficiently supervising CDTI activity at the communities in an integrated manner and problems and successes are managed promptly and The DHS goes round to collect treatment repofts and keeps a copy of the community treatment summary forms at the facility and often these are sent to him/her by CDDs. The DHS monitors and superuises officers in charge of faciliUes who in turn superuise and monitor CDTI activities in the communities. There was however no checklist to conflrm this and reports showed that some health facility staff do not supervise their communities. The DHS also use the opportunity of their monitoring visits to carry out other health care activities, suggesting some form of integration. They are also involved in co-treatment of LF and Onchocerciasis in their districts. Supervision is funded by the pCIect (APOC). Sometimes, reports to the LGAs were late due to delays in submission of reports by CDDs. There was no reward system in place to encourage health workers who show outstanding commitment to CDTI in the facilities. Mectizan Procurement and Distribution (2.7 Moderately): Sufficient Mectizan is being ordered annually, and in good time. Mectizan is being collected, stored and administered within the ment at this level. There were no reported shoftages in most communiUes. Mectizan is stored at the drug store in district health facility and issued out almost immediately to the communiUes. Health facility staff collects drugs for their communities, while CDDs collect drugs on behalf of their communities from the health facility closest to them. There was a good Mectizan inventory of drugs received and balances returned. There were no standard and rational means of determining the number of drugs required by communities. Instead drugs supply is based on the previous year distribution. The drugs, in most cases, were not available at the period the community requires them, which is before active farming season (January to April). Training and HSAM (2,5 Moderately) and HSAM are bei and carried out in an efficient manner. The DHS train the Officers in charge of the FLHF who in turn train the CDDs. In most cases training was found to be routine and in instances where it was not routine, emphasis were on training new CDDs. Training was usually carried out at the district health facility. A team made of the LOCT, DHS and CDDs carried out HSAM of communities annually as routine. It was only on few occasions that HSAM was targeted at issues like refusal of treatment due to side effects. Training is integrated with LF and Schistosomiasis control program but not with other PHC activities. Most health workers at the health facilities have poor knowledge of CDTI because the 22 training for health workers was restricted to one per health facility. Training was entirely funded by the project (APOC and NGDO). Where HSAM was jointly done for LF and Oncho, Oncho becomes subsumed under LF because of perceived high burden of LF. Health education materials such as posters and flip charts were provided by APOC but not adequate Financing (0 Not at all) The costs involved in planned CDTI activities at this level are clearly defined and budgeted for. Sufficient funds to cover these costs are fully or increasingly being disbursed from LGA resources. The Local Government has never provided funds for CDTI activities at this level. CDTI activities are funded through project budgets (APOC and NGDO) since the inception of the program. Transport and other Material Resources (2 Moderately): Adequate and appropriate transport and materials are available for necessary CDTI activities at this level and are adequately and appropriately maintained, managed and used, in an integrated way The transports available at this level are used for all PHC activities CDTI inclusive. The LGA carries out major repairs and maintenance when the vehicle is to be used for NID, the DHS is responsible for minor repairs from his salary and proceeds from sale of PHC drugs in the facility. The motorcycles are inadequate and IEC materials such as posters are not provided on an annual basis. Government can pay for maintenance, replacement of parts and repairs if they are committed to the program. There is no guideline for movement or trips for CDTI. No logbook or travel authorization is used at this level. Human Resources (1.6 Sliohtlv): The team at this level is skilled and knowledgeable, regarding the implementation of CDTI in its area of operation. The CHEWs at the lower cadre are more Sable and not regularly transferred out of their stations. DHS are higher officers and are liable to be transferred out of the LGA at any time. Training at this level is restricted to the District Health Superuisor, who is the most senior officer. There are five LOCTs, one in charge of each of the five health districts. There are also five DHS, one in each of the health district also. The LOCTs are often junior in position to DHS they are expected to superuise. 23 4Fu The geographical coverage is satisfactory at this level based on distribution reports. There are however, no REMO results at the FLHF level to verify this. Recommendation at this level The ical in the FLHF area is sati RecommendaUon Implementation Human Resources The role of LOCTs and DHS should be reviewed with a view to ensuring clarity and avoid duplication of duties. Priority: High Indicators of success LOCT members are assigned specific duties and are empowered to perform the assiqned duties Who to take action: PHC Co-ordinator and LGA Oncho. Co-ordinator, Deadline for completion: April, 2003 Training All the health workers at the facility levels should be trained adequately on CDTI Priority: High Indicators of success All health workers at the health facility levels have good knowledge of CDTI. Who to take action: PHC Co-ordinator and LGA Oncho. Co-ordinator. Deadline for completion: June, 2003 Monitoring and Superuision All CDTI supervisors should use superuisory checklists to ensure appropriate supervision is carried out Priority: Medium Indicators of success Completed supervisory checkl ists by supervisors are available. Who to take action: LOCTS, Deadline for completion: October, 2003 24 Fig. 3 Sustainability at the community level 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 o) .Ecc o o_ o_Ea oE(U c)J o)c L o =c o c o .N (J o U)I 6 '6 C(E cir c o EfI o o)(E o oo Group lndicators the take place. Communities are willing to take the drug but some do not know how long they are expected to do so. In some villages, the leadership has not played any effective and proactive roles in solving major problems such as refusals, CDD attrition and inadequate support to CDDs. Monitoring and Suoeruision (2.6 Moderately): CDDs are to the FLHF level CDDs are reporting appropriately to first line health facilities (submltting repofts and drug balances). Community treatment records are available at the communities, but not properly kept in few communitles. In most cases, monitoring was not carried out by the supervisor at the first line health facility and this had negative impact on the quality and timeliness of reports Mectizan Prccurement and Distribution (3'0 Hiohlv): The rlght amount of Mectizan is received and the CDDs or community members themselves fetch the of Mectizan. Drug supply to communities is adequate. The Officer in charge of the FLHF informs the community about the availability of the drug at the central collection point, and CDDs collect the drugs for distribution. FLHF staff sometimes take the drugs to the CDDs. CDDs were not taught n 25 how to determine the quantity of drugs to be requested for their communities. Lack of a rational criterion for determining quantity of drugs needed often result in surplus drugs in some communities and shortage in a few communiUes. Training and HSAM (2,O Moderately't: CDDs and authorities continue to be in HSAM of other commu members HSAM for communiUes is carried out annually before distribution by a team of health workers, CDDs and community leaders. IEC materials are not usually adequate for HSAM. Knowledge of community members and CDDs about Onchocerciasis and Mectizan treatment is poor in communities that are being co-treated for LF and Onchocerciasis. There were repofts of refusals as a result of inadequate health education, mobilization and sensitization at the communiUes. Finance (2.3 Moderately): The community has made arrangements to fund local costs of distribution. Some communities provide money or non-financial incentives such as farming for the CDDS. Community support as mentioned by some CDDs is inadequate in most communities. Communities knew their roles to support the CDDs, but admitted that they had often failed to do so. Human Resources (3.2 Hiohly) There is a satisfactory ratio of CDDs to households and all CDDs have received appropriate training. CDDs are wil to continue their work in CDTI Registration and treatment is usually completed between two weeks to a month. Prior to last year (2002), training was mainly routine, but now training is targeted at new CDDs because of high attrition rate. The DHS at the first line health facility conducts training for CDDs at the health facility. Most CDDs are willing to continue to serve as CDDs in the long term. There was however, indicaUon that lack of adequate incentive could lead to further CDD attrition. The ratio of CDDs to population is less than the recommended ratio of one CDD to 20 households or 2 CDDS to per 250 population. Most CDDs do not travel far from their homes to distribute Mectizan. Posters and CDD information booklets are provided to CDDs after training annually but not adequate. Attrition rate of CDDs is high due to inadequate suppoft and mobile characteristics of CDDs for further education and employment in towns and cities Coverage (4.O Fully) The geographical coverage in the community is satisfactory and the community has a thera rate. Geographic coverage was 100o/o as all the endemic communities, wards and hamlets are being treated. Therapeutic coverage is very high, as most of the communities have attained an increasing rate of at least 650/0. There were discrepancies in therapeutic coverage given by the project and that calculated by the evaluator in some communities 26 Recommendation Implementation Tralnlng More CDDs should be adequately trained and deployed in each community. CDDs should be trained on the use of the census to determine the quantity of drugs required Priority: Hiqh Indicators of success -The recommended ratio of 2 CDDs to a population of 250 people is met. -The demand for incentive is less. - CDDs can determine the quantity of drugs needed in their communities Who to take action: PHC Co-ordinator and LGA Oncho. Co-ordinator, Deadline for completion : Mav. 2003 Human Resources: The Communities should develop means of providing more CDDS and provision of incentives through the use of community based organisations and villaqe health committees HSAM HSAM should be intensified for communities to understand and perform their CDTI activities. Priority: High Indicators of success -Communities understood their roles and support their CDDs. -Communities have good knowledge of Onchocerciasis and Mectizan treatment. Who to take action: PHC Co-ordinator and LGA Onchocerciasis Co- ordinator, Deadline for completion: July, 2003 Recommendation at this level Conclusion and way-forward Using the four-tier schedule for ranking projects evaluated: fully, High, Moderate and low, the evaluation team judges the Plateau State CDTI to be at the level of Moderate. The project is potentially sustainable as evidenced by the excellent coverage, high level of community leadership and approval and availability of mectizan everywhere. But this project requires rethinking and mobilization of high-level decision-makers to provide financial resources to sustain CDn. It is intriguing to the evaluation that at the end of the fifth year the State has not made a single annual contribution on the lines agreed with APOC. Other important issues that need rethinking include: High attrition rate of CDDs and further empowering of LGAs, through training and decentralization of authority from the State level to 27 LGAs and FLHFs. Using the Abuja grading of: Fully sustainable; On the way to sustainability and not on the way to sustainability, the evaluation team concludes that the Plateau program is on the way to sustainability. Two feedback/planning meetings (one day and two days for LGA and SGA management staff respectively) were successfully carried out, Details of issues discussed and the sustainability plans prepared are contained in report of the meeting attached. Way forward State and LGA leadership indicated that rapid action will be immediately taken in a number of interrelated key areas. Foremost is the implementaUon of the one-year plan of action (developed during the briefing and planning meeting) and development of a three year CDTI sustainability plan. The process of developing the three-year plan has potentials for mobilization of all paftners. The evaluaUon team recommends that APOC follows closely and provide support in key areas particularly the development of the three-year sustainability plan, which is key to future collaboration. It is hoped that the target of completing the plan by Mid April and making it available to APOC will be met. 28 Appendix I Plateau State CDTI Planning Meeting AGENDA 1 Openinq prayer 10:00 - 10:05 To be appointed 2 Welcome 10:05 - 10:10 Director PHC 10:10 - 10:15 SOC (Henry Filda)3 Introductions 4 Introduction to the workshop; What are the aims and objectives of the meeting What ls sustainability 10:15 - 10:30 Dr E. Tarimo 5 Tea Break 10:30 - 10:45 Prof.O. Nwaorgo Dr. K. Ogungbemi 6 "Feedback" on achievements, issues and lessons from the Kogi evaluation on sustainability of CDTI 10:45 - 11:15 S. Isiyaku7 Identification of problems identified and solutions 11:15 - 12:30 8 LUNCH 12.30 - 13.00 13.00 - 13.30 Group Discussions9 Feedback from group work 13.30 - 14.00 Dr. K. Ogungbemi10 Rethinking roles of the different levels and partners 14.00 - 16:00 Dr. T. Tarimo11 Planning for sustainability in this project Planning for the next one year 13 Tea Break 16:00 - 16:45 L4 The way forward: implementing self- sustainability (what to do now) 15:45 - 17:15 Director, PHC 15 AOB t7.t5 - 17.30 To be aooointed16 Closinq Prayers t7.30 FacilitatorItem Actrvr Trme 29 Appendix II: Program for Plateau State CDTI Planning Meeting for Local Government Areas "Feedback'/ Planning Meeting 1 To be apoointed1 Openinq prayer 10:00 - 10:05 Director PHC2 Welcome remarks 10:05 - 10:10 10:10 - 10:15 SOC (Henry Filda)3 Introductions Dr. E. Tarimo4 What are the aims and objectives of the meetinq? 10:15 - 10:30 5 Tea Break 10:30 - 10:45 10:45 - 10:55 S. Isiyaku6 Where we are in the evaluation 6 "Feedback" on issues and lessons from the Plateau evaluation on sustainability of CDn in (LGAs) 10:55 - 11:30 Dr. H. Hamisi Dr. K. Ogungbemi Prof. O.Nwaorgu 7 Identification of problems and solutions: Plenary 11:30 - 12:45 S. Isiyaku LUflCH 72,45 - 73,30 8 13.30 - 14.00 Dr. Ogungbemi9 Rethinking roles of the different levels and partners S. Isiyaku10 Planning for sustainability in this project Planning for the next one year: Group Work by LGA 14.00 - 16:00 Day 2 1 Group Work Continue 8:30 - 10:30 10:30 - 10:452 Tea Break Report back from qrouD work 10:45 - 12:003 12:00 - 12:30 LGA Chairmen Directors PHC 4 The way fonruard: implementing self- sustainability (what to do now) 5 AOB 12:30 - 13:30 6 Closinq Pravers 13:30 To be appointed Time FacilitatorItem Actr 30 Appendix III State/LGA Level Planning Meeting Program The participants would look at problems identified by the evaluators under each indicator and agree solutions to them. In planning for the next one year the group would: a Group would then feed back at a plenary session for general input. This would be the frame work within which the flrst plan would be produce i.e. what would be done between now and the end of the year to address the issues raised by the evaluators and to initiate the process of developing a sustainability plan post APOC funding. 3l a a rI Appendix tV Problems and Solution State Level s AT STATE LEVEL PROBLEMS SOLUTIONS Planning No funds released for CDTI activities following request from CDTI focal person in the LGA, even though CDTI is integrated into the PHC plan, Some Chairman absent during last State sustainabil ity meeting The director PHC accepted that sustainability efforts had just been initiated recently. That each LGA is to contribute N300, 000.00 while the State contributes 2.5m as counterpart funds. That the commissioner of health had directed that an account be opened for this purpose. Suggestion n was made that the money be lodge in LGA accounts instead of State. Strong and persistent advocacy especially from APOC and other NGDOs is required to get desired counterpart funds. Interview held with the commissioner of health suggested that the NGDO should handle the money a a a a a a Training raining materials on CDTI were canty at all levels Some staff at FLHF were not knowledgeable about CDTI The SPO promised to intensiff training in the areas where the programs are integrated CDTI activities are included in school curriculum of public health schools e.g. school of health technology. a a Finance Salaries of local government staff are not paid by LGA even though presently they are in 6 months salary arrears. No financial budget for CDTI program by LGAs No CDTI expenditure for the past 5 years at the LGA level The actual accounting for APOC funds falls on the State office. Local Government Councils are not aware of funds available to them for CDTI activities annually. Suggestion made that CDTI accountant be deployed or appointed at State level, That CDDs be empowered economically where the community failed to support. Also that the CDDs be involved in programs that gives incentives like during NPI program. a a Monitoring & Monitoring reports not available at all levels, The director planning said the LGAs are expected to take full control of 32 5 a a supervision In some LGAs some of the supervisory visit by LOCTs at the FLHF level appear to be high. Some LOCTs collect routine CDTI data directly from the FLHF level and communities (CDDs). Processes of data collection are not part of government health management systems. monitoring and supervision. Accepted that there was weak integration system in the State but effort will be put in place to strengthen it. Mectizan Mectizan received from State to LGAs without previous requisition form or request form HLHF or communities. Census figures do not seem to be used in calculation of annual mectizan requirement. Finances for transport of mectizan from LGA to HLHF provided by APOC at the district and LGA levels. The SPO, SOCTS, LOCTS and the HLHF to train and educate CDDs on how to apply for mectizan by using their census figures and multiplying by factor of 3. Transpoft No logbooks, no plans to repair vehicles available, fueling etc. motorcycle spare parts are carried out with APOC fund. Suggestion was made that APOC help in provision of vehicle. Health Education Materials H/education not adequate (posters, flipcharts, leaflets) the last time this was received was in 2001, APOC, LGAs will help in developing education materials. Human resources Instability of staff at FLHF level, some district health officers are only 6 months old at the present post CDTI staffs at LGA level receive allowances including mop-ups after training and supervision of CDTI activities yet do not report to the State office in time. Suggestion was made that LGA staff should be transfened. Not to rely on NGDO for their data management. Coverage Distribution of mectizan is always not on time for some of the communities. Coverage was high. As reported by some LOCTs, coverage has been high by 10olo annuolly due to the introduction of LF program in some LGAs. I 33 Appendir V Problems and Solutions LGA Leve! AT LGA LEVEL PROBLEMS SOLUTIONS Planning No funds released Directives to be given from States MOH and Ministry for Local Govt. & Chieftaincy affairs to LGA to release funds. Chairman absent during last State sustainability meetinq Another meeting is to be convened Leadership 1. Only LOCTS, DHS, HHFS are trained for Oncho program. Why others? None involvement of health staff at FLHF LOCTs depends wholly on NGDO for funding . LGA has continue to depend on State or CDTI activities 1. Workhop for all LGA coordinators, Directors on CDTI 2. Health institutions (CDn) curriculum 3. LOCTs to train DHS/H/HFS and other health staff at FLHF 4. Organize a sort of competiUon LGA by LGA and District by District, DPHC to release little funds for CDTI activities. 1. LGA should come for drugs at Jos, training and other activities. This should be included in the LGA work plan annually Monitoring/ Evaluation M & E don't know what is happening about CDT repofts 1. HOD should ensure that conect reporting procedures are adhered to. 2. Integration into PHC CDTI reporting proqram Mectizan Quantity of mectizan not properly calculated and inadequate or excess Proper training on how to calculate mectizan comparable with the census pop. total pop x 3 Training/Healt h Education No posters on LF/Oncho CDn (IEC materials) inadequate IEC materials 1. HOD, Coordinators should ensure that training be conducted and provide health education materials 2. States, LGAs should print posters and other IEC materials 3. Printed posters, when distribution should be proDerly distributed and monitored. Finance No funds released by LGA, mostly counterpart funding (Not released) Directives to be given by MOH and Ministry of Local Govt. and chieftaincy to LGAs to release funds (counteroart). Transport/othe r logistics materials No logbooks, no plans to repair vehicles available, fueling etc. bicycles Logbooks, repairs, fueling should be done by the LGAs (vehicles could also be given during CDTI activities), Coveraqe Kudus 34 Appendir VI FLHF AND COMMUNITY LEVEL FLHF/COMM UNITY LEVEL PROBLEMS SOLUTIONS Planning DHS/LOCTS are doing the same thing (CDTI activities) Each LGA should soft out and use either DHA/LOCT or decide No funding at district level, no budget 1. The DHS should be allowed to produce their budget 2. LGA should include such CDTI budget at LGA budqet Attrition rate Attrition rate of CDDs Lack of support to CDDs Gettinq work else where 1. Motivation by the communities 2. Reduce the work load of CDDs 3. Traditional leader involvement, district heads, village heads, ward heads. 4. LGA should also contrlbute IEC materials No IEC offices at LGA level Directives be given to LGA to provide offices for IEC Training/Healt h Education No posters on LF/Oncho CDn (IEC materials) inadequate IEC materials 1. HOD, Coordinators should ensure that training be conducted and provide health education materials 2. States, LGAs should print posters and other IEC materials 3. Printed posters, when distribution should be properly distributed and monitored. 35 1. Bar 2. Mr 3, Mr 1. 2. 3. 4. Appendix vii List of Persons Interviewed during the evaluation State Ministry of Health rrister John Magaji . David S. Belin . Henry Filda NGDO Dr. E. S. Miri Dr. Eigege Abel Mr. John Umoru ZOTIAL NOCP Mrs. Pearce Ogbu BOKKOS LGA 1. Mrs. Dorcas Wambula 2. Pai Hamman3 Mrs Nimdom C. Miri 4, Mr. James Dashen 5, John Jik 6. Health facility staff 7. Ezra Ringmen 8. Friday Daniel 9. Elizabeth Gantu 10.Luka John 11. Bitrus Gaza 12. Health facility staff 13. Adamu Job 14. Yoila Modik 15. Elimon Akaya 16. Yusuf Makabi PANKSHIIV LGA 1. LOCT Pankshin 2. PHC Coordinator Pankshin 3. LGA Chairman 4. District Supervisor Ballang Shipang- - Honorable Commissioner for Health, Plateau State - Director PHC and Disease Control, MOH Plateau State - State Onchocerciasis Program Officer - Country Representative GRBP (Cafter Center) - Director, LF and Schistosomiasis Program - ProjectOfficer, Onchocerciassis - GRBP P@ect Accountant Zonal Coordinator, NOCP - Assistant PHC Coordinator - Oncho Coordinator - Director of Finance - The Chairmankos LGA - District Health Supervisor, Ikngnhap - Gara - CDD, Kawel - CDD, Sat - DHS, Daffo - CDD, Toff - CDD, Karfa - Toff - Village leader, Kawel - Village leader, Sat - Village leader, Toff - Village leader Karfa - Josiah Dakon - Mrs. Rifkatu Akins - Diket P. Mrs. Dara Gomet - James Gokok - Anthony Gomwir Goger Godung Rabecca Bulus Gorap Kadyis Community 1. Village member male 2, CDD male Kadyis 3. CDD male Kadyis 4. Village Head Kadyis Nyelleng Community 1. CDD female 2. Village Chief 36 3. Village member female 4. Village Member male 5. PHC FLHF 6. District Health Superuisor Jiblik Tilleng Community 1, Village Member male 2. Village member female 3. CDD female 4. Village Chief Jing Community 1. Village member male 2. Village leader Mrs. Christy Yunana Jacob Dabish - Rosaline Christopher Mrs. Christy Yunana Adamu Dariye Stephen Damara Nanle Park 37 Appendix viii List of Evaluatorc ,l S/N o Name Address 1 Dr, Eleuther Tarimo Box33277, Dar es Salaam, Tanzania +0255 742 605025 eleuther@ud.co.tz 2. Prof. Obioma Nwaorgu Dept. of Parasitology & Entomology Enugu State University of Science and Technology, Enugu, Nigeria Tel: 23442456128(H) 234 42 2s8078 (OfO 234 42 250836 (Fax) onwaorgu@infoweb. abs. net Obinwaorgu@yahoo.com 3 Dr. Kayode Ogungbemi Dept of Psychiatry Ahmadu Bello University Kaduna, Nigeria. Mobile: 234 803 3230018 Tel: 234 62212844 (H)Tel: 234 62248360 m_kayode@hotmail.com 4. Dr. Harriet Hamisi HKI Project Officer Tanga CDTI Project P. O. Box 5547 Tanga, Tanzania Tel: 255 27 26462L7 255 27 2647880 255744 694369 25s 748 381520 lyimoharrieth@yahoo.com 5 Mr. Sunday Isiyaku Sight Savers International 1 Golf Course Road P. O. Box 503 Kaduna, Nigeria Tel:234 62248360 Fax:234 62248973 Tel:234 622312L6 (H) ssino@infoweb.abs. net 6 Dr. Abel Eigege Carter Center, Inc (GRBP) 1 Jeka Kadima StreeL Off Tudun Wada Ring Road Jos, Nigeria Tel:23473 46L861 234 73 463870 Fax: 234 73 460097 q2000@hisen.orq 38 - lnstrument 1: national/ State level NOTE: . This instrument is for national level which coordinates programme implementation in the country. lt could also be used (modified slightly if necessary) for another level of support other than the main implementation level (districU LGA). An example of this level is the 'State' in Nigeria. The focus of this level's activities in CDTI The main function of this level is to develop, revise as necessary, and oversee implementation of CDTI Policy. This level also provides supporf to the level below it: . Providing targeted training, HSAM and monitoring/ supervision. . Arranging for an adequate supply of Mectizan. Country: Plateau State Nigeria Researcher: Team Date: 28 - 03 - 2003 Abbreviations/ acronymsCDD community directed distributorCDTI community directed treatment with ivermectinFLHF first line health facilityHSAM health education, sensitisation, advocacy, mobilisation - i.e. activities that are aimed at getting all the key players to participate wholeheartedly in the programmeNGDO non-governmentaldevelopment organisationNOTF national onchocerciasis task force {. tndtcalorc of actlvltles and proCesses: This indicator assesses whether the programme has become integrated into the seruice, and whether management is accepting ownership of the programme - both good for sustainability. Check whether there is a year plan for onchocerciasis contro!, appearing as part of for the health seruice at this level 1.1 an overallwritten Characteristics of the indicator Sources of information Onchocerciasis control should be integrated into the overall written year plan of the health service at this level. Note that this plan is usually not very detailed. Examination of: * Written plans: yearly, quarterly, monthly etc. * Minutes of planning meetings. lnterview with health service staff at this level. Findings Describe the present situation: Control of onchocerciasis appears as part of MOHs annual development plan; Overhead 423. All disease control units prepare annual plan proposals, which are consolidated by Director PHC. The consolidated plans are fonrvarded in succession to Director Planning MOH, MOF and parliament for review and approval. MOH staff are called upon by MOF to defend their proposals as necessary. lf planning and implementation of CDTI is not part of the overall year plan: ' \Nhy is this? N/A- . VlJhich steps are being taken to improve the situation? N/A Analysis Wren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is to affect sustainabi Your overalljudgement is this indicator for sustainab being achieved? Fully Highly Moderately Slightly Not at all Not applicable T.] 1.2 Check whether there is a year ptan containing details of all activities needed for effective onchocerciasis control activity at this level. This indicatorassesses whether the project is functioning effectively, and whether management accepts ownership of the programme - both good for sustainability. Characteristics of the indicator Sources of information This more detailed plan should make provision for all key elements of onchocerciasis control: Mectizan supply; targeted training; targeted HSAM; targeted monitoring/ supervision. The plan varies from year to year, showing that it is targeted to the specific needs of each year. Examination of written plans: yearly, quarterly, monthly etc. lnterview with health service staff at this level. Findinss Describe the plan for the present year: A yearly plan containing details of activities for effective control of onchocerciasis exists. The plan is based on the Five Year Proposal of 1997. The first step is the development, by SPO of a proposal-outlining plan of action and spending plan for the year in question. The proposal is reviewed by APOC review meeting at the national leveland forwarded to APOC for consideration by the TCC. The May 2002-2003 has been approved by APOC. This plan has key elements for an effective CDTI programme; Training, advocacy, supervision/monitoring and mectizan distribution. Describe the plan for the previous year: Similar to the 2002 plan. lf the plan is incomplete, or simply a re-write of previous plans: . VVhy is this? N/A t Vvhich steps are being taken to improve the situation? N/A Analysis \Mren writing the report you have to summarise:x The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable I I 1.3 Check whether al! partners (government, UN agencies, NGDOs) are meaningfully involved in the overall planning procesa. This indicator assesses whether the programme is functioning effectively and efriciently - if each paftner is clear about its role, this is good for sustainabili$. Characteristics of the indicator Sources of information All partners should contribute to the routine planning of a project. Partners should be clear about their own roles, and those of the other partners. Examination of:x Plans: yearly, quarterly, monthly etc. * Minutes of NOTF meetings. lnterview with staff at this level (government, relevant UN aqencies and NGDOs) I Findings Describe the present situation All partners participated in the development of the in initial 1997 Proposal. The annual plan of action also indicates contributions to be made by different partners. Contribution by MOH is mostly in the form of staff costs. APOC provides financial support for key CDTI activities. GRBP supports a number of elements and backstopping. However support to activities by SOCT and GRBD in such areas, as supervision is not coordinated. lf partners are not meaningfully involved in planning: . VVhy is this? N/A Wrich steps are being taken to improve the situation? N/A Analysis ! \Men writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabiliU. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 1.4 Check whether specific planning for sustainability has taken place, for the period after APOC funding is withdrawn. Ihis assesses whether the programme is functioning effectively, and that management has begun to take ownership of it and can mobilise the resources if needs. Characteristics of the indicator Sources of information ! Staff members have made plans for this period, which will enhance programme sustainabilig. This planning should include: identifying resource gaps; strategies to cut expenditure; and strategies to find dependable sources. There should be written evidence that such planning has taken place. There should be evidence that the plans are being successfully implemented. Examination of the written sustainability plans. lnterviews with staff at this level (government and NGDO). t Findings Describe the present situation Planning for sustainability has not taken place. A start has been made by convening a meeting of LGA Chairmen. Apparently an agreement was reached that each LGA should take ownership of CDTI and provide finding. lnformation provided during interviews on level of funding by LGAs ranged from 1000000 to 30000 per year. Minutes of the meeting were not available. lf there has been little or no planning for sustainability: . \Mry is this? The deficiency was attributed to frequent changes of senior staff and policy makers. . \Mrich steps are being taken to improve the situation? Follow up meeting with LGA chairman is envisaged. The evaluation report is also to enhance action. Analysis t \Mren writing the report you have to summarise:r' The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabiliU. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 2. lndlcators of actlvltles and processes: monltorlngi supCrvlslon 2.1 Check whether staff at thls level is being used appropriately for monitoring/ superuision. This indicator assesses whether the programme is functioning efficiently, Characteristics of the indicator Sources of information Staff members at this level should routinely only supervise the level immediately below them. Staff should not supervise the FLHF or community levels. 'Spot checks' may however be done from time to time. Staff members at this level should have empowered staff members at the level below them to supervise activities at their own level, as well as levels further down. Examination of: * Supervisory checklists, plans and reports. * Visitor's books at all the levels below this one. lnterviews with: * Staff at this level. * Staff at levels below this one. Findings Staff at this level (SOCTs) supervise LOCTS. SOCTs and Global 2OOO staff also carry out spot check on supervisors and FLHFs. LOCTs supervise lower levels. Other related activities include: Analyses of data and reports from lower levels and organization of State review meetings (involving SOCTS, LOCTs and Directors of PHC). Staff make quarterly supervisory visits. During field visits SOCTs appeared wel! informed &out issues at FLHFs and community level. Some LOCTs at times were much less informed about such issues, which raises doubts about the adequacy of their empowerment. lf staff members are not being used appropriately for monitoring/ supervision . Vvhy is this? N/A I \Mrich steps are being taken to improve the situation? N/A Analysis I \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highty Moderately Slightly Not at all Not applicable 2.2 Check whether monitoring/ superuision is being planned and carried out in an efficient and integrated manner. This indicatorassesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information One routine supervision visit per year must be done to each project (as a separate entity, or as a district). Supervision visits for CDTI should be integrated where appropriate with supervision of other programmes. Resources for supervision (human, transport etc.) should be efficiently used: * Using as few staff members as possible. * Planning trips to cut down on distance travelled. * Not spending unnecessarily many nights out etc. Supervision visits should be thorough, using a checklist. I I Examination of: * Supervisorychecklists, plans and reports * Visito/s books at the level immediately below this one . lnterviews with: * Staff at this level. * Staff at the level immediately below this one. Findings Describe the present situation Each of the five SOCTs supervises an LGA. Visits to LGAs adequate. \Mrile supervision of CDTI is integrated with LF and Control of Schistosomiasis at LGA and lower levels, it is not the case at the State level. There is collaboration between SPO and LF desk officer. Supervision from the State to LGAs for CDTI and LF is not integrated. The overall integration of CDTI with PHC is weak. SOCT supervision is not synchronised with GRBP Describe the situation the previous year: Supervision in2002 is more integrated compared to 2001 lf monitoring/ supervision is not being done efficiently . Why is this? N/A Vvhich steps are being taken to improve the situation? N/A Analysis \Mren writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Examine the trend in monitoring/ supervision activity: is it becoming more efficient? I Your overalljudgement is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable I f 2.3 Check whether there ls a routine process of management of problems and successes, which are indicated by the This indicator assesses whether the programme is running efficiently and effectively, and whether management is beginning to accept ownership of the programme. Characteristics of the indicator Sources of information As soon as problems are identified as a result of supervision visits, or from coverage data (i.e. areas with low coverage) the appropriate manager should deal with them. Such problems should usually be passed on to the appropriate managers at the next level below to deal with, with the necessary support - thus empowering these persons. Successes should be noted and reported, and appropriate feedback given. There should be evidence of action taken based on recommendations in previous monitoring exercises. Examination of the following documents: * Plans: yearly, quarterly, monthly etc. * Minutes of staff/ planning meetings. * Reports of previous monitoring exercises. * Letters of commendation. lnterviews with:x Staff at this level. * Staff at the levels below this one. Findings Describe the present situation: Management problems are dealt with promptly. Several examples were provided, including problems in statistics and calculations, combined treatment for CDTI and LF and dosage. Staff at all levels are encouraged initially to communicate problems verbally to officers responsible to enhance prompt response. lf the system of problems/ successes is weak: . Vvhy is this? N/A I Vvhich steps are being taken to improve the situation? N/A Analysis \Men writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 3. lndicatorc of actlvlt[es and processes: Mectizan@ procurement and distrlbution 3.1 Check whether sufficient Mectizan is being ordered, stored and distributed within the overnment m at this level. This indicatorassesses whether the programme is functioning efficiently, tfs processes are simple, and it is becoming more integrated into the government system. Characteristics of the indicator Sources of information The Mectizan supply should be controlled within a government system. This does p! have to be the system routinely used for the supply of other drugs. The system should be effective, uncomplicated and efficient. This system should use dependable, sustainable resources for its operation. lt is desirable that the government should supply these resources. The system should supply sufficient Mectizan for the needs of all the proiects concerned. r Examination of all Mectizan ordering and stock control documentation at this level. lnterviews with staff at this level (government and NGDO). Describe the situation the previous year: System of ordering Mectizan, like in other States in Nigeria is under an NGDO (G The system operates well. Storage and distribution Mectizan is mostly within government system. GRBP delivers Mectizan to SPO who in turndeliver the same to LGA, LGA to FLHF or FLHFs collect from LGA. \Mrile census is carried out yearly, there is no clear indication or evidence that census figures are basis for calculating Mectizan requirement according to the APOC formula. Describe the situation the year before that: Same as above lf the government system is not fully responsible for all sections of the Mectizan supply system . \Mty is this? Same procedure for Nigeria. \Mrich steps are being taken to improve the situation? No specific activities are being taken to use census data to calculate mectizan requirements Analysis Vvhen writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. \Mrat has been the trend in Mectizan supply? I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 4. lndtcatorc of actlvltles and processes: tralnlng and HSAM 4.1 Check whether staff membens at this level are being used as trainers. This indicatorassesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information Staff at the national level should routinely only train staff at the level immediately below it. Staff members at this level should have empowered the level immediately below them to train lower levels. Examination of training materials, plans/ programmes, reports:{' At this level. * At the levels below this one. lnterviews with: * Staff at this level (the trainers). * Staff at the very next level below (the trainees). * Staff at the districU LGA level. * Staff at the FLHF level. Findings Describe the present situation Staff at state level routinely only train LOCT. LOCT staff in turn train supervisors and FLHF staff. FLHF staff train CDDs. Describe the situation the year before: Same as above lf staff members are not being used efficiently as trainers: . Why is this? N/A \Mich steps are being taken to improve the situation? N/A Analysis Wren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Examine the trend in the way in which staff are being used as trainers: I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable l0 -t 4.2 Check whether training is being planned and carried out in an efficient and rated manner. This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information There should be an objective need for each episode of training. This means there should be evidence that staff to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Repeat training of already skilled staff should not happen. lf circumstances permit training for CDTI should be integrated with other training, e.g. in in-service training programmes. Resources for training (human, transport etc.) should be efflciently used: * Using as few staff members as possible, * Using as little time as possible (without sacrificing quality) * Choosing the most cost-effective site etc. Examination of training materials, plans/ programmes, reports: ,' At this level. '8 At the levels below this one lnterviews with:x Staff at this level (the trainers). * Staff at the very next level below (the trainees). * Staff at the districU LGA level. * Staff at the FLHF level. I Findinss Describe the present situation: Evaluators were informed that training is mostly provided for new transferred staff. Training for CDTI is canied out together with that for LF and Schistosomiasis. Describe the situation the year before: Same as above lf training is not being carried out in an efficient and integrated manner: . !\lhy is this? N/A Which steps are being taken to improve the situation? N/A Vvhen writing the report you have to summarise:x The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it and how this is likely to affect sustainability. t Examine the trend in training activities - is it becoming more efficient and integrated? t Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable ll tr 4.3 Check whether staff at this level is planning and carrying out HSAM activities in an efficlent manner. This indicator assesses whether the programme is functioning efficlently and effectively, and whether managers are taking ownerchip of the programme. Characteristlcs of the lndicator Sources of information Staff members identiff situations where decision makers lack information abouU commitment to CDTI, and undertake activities to inform and persuade these persons. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. Such activities should only be carried out at the national level, and at times at the level immediately below (but only when staff at that levelasks for help). There is evidence that these HSAM activities have been effective and have led to action. I ! Examination of HSAM plans/ programmes and reports. lnterviews with: * Staff (programme and management) at this level. * Civil authorities at this level. x Staff and civil authorities at the next leveldown. Findings Describe the present situation (in relation to efficiency and outcome): HSAM activities are well planned. These include high-level advocacy to Governor. SPO has made considerable efforts to mobilize counterpart funding. Shortage of IEC materials reported by LGAs. On inquiry it emerged that IEC materials may be adequate, the problem is LGAs not collecting due to lack of rooms for storage. Describe the situation the year before (in relation to efficiency and outcome): Same as above lf HSAM activities are not being carried out efficiently: . Vvhy is this? \Mich steps are being taken to improve the situation? Analysis \Men writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Examine the trend in HSAM activities - is it becoming more efficient and effective? I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable t2 I r n -t lnstrument 2: districU LGA level NOTE: . This instrument evaluates the level which actually takes responsibility for implementing CDTI in its area of operations. lt is that level where health services are planned and provided. We are going to refer to it as the districU LGA level . . The level below this one is the one where the health centres/ clinics/ dispensaries are located. We are going to refer to this level as the 'FLHF' (front line health facility) level. The focus of this level's activities in CDTI . The main function of this level is lo take responsibility for the implementation of CDTI in its area of operation. . However its function is still largely one of supporl of the FLHF level: * Providing targeted training, HSAM and monitoring/ supervision. * Arranging for an adequate supply of Mectizan. . The FLHF level is the one that is finally responsible for working with the CDDs in the communities. Geographical name of this districU LGA: Project PLATEAU Researcher: THE TEAM Date: 28 - 03 - 2003 Abbreviations/ acronymsCDD community directed distributorCDTI community directed treatment with ivermectinFLHF first line health facility HSAM health education, sensitisation, advocacy, mobilisation - i.e. activities which are aimed at getting all the key players to participate wholeheartedly in the programme NGDO non-governmentaldevelopment organisation l3 1. lndlcators of actlvltles and This indicator assesses whether the programme has become integrated into the health seruice, and whether management is beginning to accept ownership of the programme - both good for Check whether the year plan for CDTI appears as part of an overall written plan for the activities of the dlstricU LGA. 1.5 Characteristics of the indicator Sources of lnformation CDTI should be integrated into the overall written plan (showing that staff at this level consider CDTI to be part of their yearly routine, like any other programme). The plan should make provision for all key activities: Mectizan supply; targeted training; targeted HSAM, targeted monitoring/ supervision. Year plans should be drawn up in a participatory way. Year plans must take into account community requirements for the timing of distribution. lnspection of: * The written year plans * Minutes of planning meetings. lnterviews with: * Staff at this level: managers and others (pharmacist, tra nsport officer etc.). * Staff at FLHF level. Describe the present situation: . CDTI not integrated into the overall PHC written plan, but the five components of the PHC come together to prepare the annual for the LGA. . Year plan not prepared in a participatory way, 2003 plan prepared by the LOCT and submitted to the PHC Coordinator for vetting, prepared plan made provision for all key CDTI activities namely: planning, training, Mectizan supply, monitoring and supervision, reporting and stakeholders meeting. . Community requirements taken into consideration in plan prepared by LOCT eg time of distribution and mobilisation. . PHC Coordinator and LOCT in attendance at the CDTI sustainability meeting held last January in Jos. lssues discussed included advocacy, assistance from LGAs and communities, counterpart funding and the coming of the evaluation team. H planning and implementation of CDTI is not part of the overall year plan . !ryhy is this? . \Mich steps are being taken to improve the situation? Analysis ffien writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable t4 Findings I 2. lndlcators of actlvlties and This indicatorassesses whether management is taking ownership of the programme. 2.1 Check whether the districU LGA health management team is taking ful! responsibility for the i on of CDTI at this leve!. Characteristics of the indicator Sources of information It should be the management team at this level, and not higher levels/ NGDO leadership, which is initiating the key CDTI activities: planning, targeted monitoring/ supervision, targeted training and HSAM, Mectizan ordering/ distribution. There should be a focal person for CDTI activities. lnspection of year plans. lnterviews with: * Management team at this level. * Staff at the project level "' NGDO leadership. * Staff at FLHF level. Findings Describe the present situation: . Presence of LGA management team, only PHC Coordinator, LOCT and District Supervisors involved in CDTI. . Oncho. Coordinator/LOCT focal person for all activities at this level depend on the State for all CDTI activities including financing. lf leadership at this level is not taking the initiative in implementing CDTI . !\,hy is this? \Mich steps are being taken to improve the situation? Analysis \Men writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable l5 3. lndlcatore of actlvltles and processes: monltorlng/ supbrvlsion 3.1 Check whether routine data concerning CDTI activities are being transmitted from this level, entirely wlthin the government system. This indicator assesses whether the programme has become more integrated into the govemment system, and is functioning effectively. Characterlstlcs of the indicator Sources of lnformation The reporting process should take place within the government system, not using other resources. Data being transmitted includes: coverage reports; Mectizan statistics; training reports; distribution reports; financia I reports. Examination of reports and report forms. lnterviews with: * Managers and staff at this level * Staff at the proiect level. I Findings Describe the present situation: . CDDs coverage/distribution reports are submitted to the FLHF and from there to the LGA level. Community and State provides the support to CDDs and staff at FLHF respectively for this purpose. . LOCT sometimes collects directly from both FLHF and CDDs. lf data are not being transmitted within a government system . Vvhy is this? I \Mrich steps are being taken to improve the situation? Analysis \Men writing the report you have to summarise: x The evidence about how well this indicator is being achieved. x Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable t6 3.2 Check whether the responsible persons at the districU LGA level are efficiently supervising CDTI activity at the FLHF level in an integrated manner. This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information Staff at this level should routinely only supervise the FLHF, and not the community level. 'Spot checks' may however be done from time to time. Although one routine supervision visit per FLHF per year is necessary, supervision visits should focus more on FLHFs where there are proven problems - each supervision visit must be justified. Supervision visits for CDTI should be integrated with supervision of other programmes (e.9. through a shared checklist). Transport for supervisory visits should be shared with other programmes. Examination of: * Supervisory checklists, plans, itineraries and reports. * Msito/s books at FLHF level. lnterviews with: * Staff at this level.x Staff at the FLHF level. Findings Describe the present situation: . Supervision is integrated with other programmes eg schisto, LF , malaria. LOCT draw up monthly itineraries for this purpose. . LOCT visits communities when there are problems eg communities not supporting CDDs. . Transportation is shared for supervisory visits for all diseases. lf supervision is not being done in an integrated and efficient manner: . ![/hy is this? \Mrich steps are being taken to improve the situation? Analysis \Men writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable l7 ! 3.3 Check whether there is a routine process of management of problems and successes, which are Indicated the mon This indicator assesses whether the programme is running efficiently and effectively, and whether management is beginning to accept ownership of the programme. Characteristics of the indicator Sources of information I As soon as problems are identified as a result of supervision visits, or from coverage data (i.e. areas with low coverage) the appropriate manager should deal with them. \Mrere relevant such problems should be passed on to the appropriate FLHF staff to deal with, with the necessary support - thus empowering these persons. Successes should be noted and reported, and appropriate feedback given. There should be evidence of action taken based on recommendations in previous monitoring exercises. I Examination of the following documents: * Year plans and annual reports. * Minutes of staff/ planning meetings. * Reports of previous monitoring exercises. * Letters of commendation * Memos. lnterviews with: * Staff at this level. * Staff at the FLHF level. I Findings Describe the present situation: . Problems identified during monitoring are dealt with at each level, CDDs report drug shortage to their village leaders and then to the FLHF. CDDs report minor drug reactions to village heads and take action on major ones by referring them to the FLHF. . Successes are noted and communities or individual praised verbally during stakeholders meeting. . Monitoring reports not available at any level. lf the system of managing problems/ successes is weak: . !ryhy is this? \A/lrich steps are being taken to improve the situation? Analysis \Men writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable l8 4, l4dicatqrs ol qgtilrlt_les and procegsee: Mectlzan@ procurement and distributlgn 4.1 Check whether sufficient Mectizan is being ordered and received yearly, and in good time. This indicator assesses whether the programme is functioning effectively. Characteristics of the indicator Sources of information The order forms for the districU LGA exist, and should be based on FLHF and community requests. The Mectizan should be available at this level in time for distribution at the time requested by the communities. There should be no reports of shortages ancU or late supply. lf there have been shortages, there should be specific plans to remedy them. Examination of all Mectizan ordering and stock control documentation at this level. lnterviews with staff at this level (managers and pharmacist). Findings Describe the situation with Mectizan supply at the last distribution: . Mectizan received from State to LGA without previous requisition order forms or request from FLFH or community. Census carried out annually by CDDs (between November and January) before each distribution process. . Staff at level (LOCT, FLHF and CDDs) unable to calculate quantity of Mectizan required annually based on community census. . SOCT supply Mectizan annually to LOCT at the LGA. Staff at FLHF collects Mectizan for communities from LOCT at LGA level. CDDs collect Mectizan from FLHF for their communities. . Shortage of Mectizan was only reported in 2000 due to presence of refuges from other states and extra Mectizan provided from state level for this. Describe the situation the previous year: Describe the situation the year before that: lf there are problems with obtaining the Mectizan that is required: . Vvhy is this? Mich steps are being taken to improve the situation? Analysis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. \Mat is the trend in Mectizan supply at this level? Your overalljudgement is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable l9 4.2 Checkwhether Mectizan is being collected, stored and effectively administered within the ment at this level. This indicator assesses whether the programme is functioning etriciently, tfs processes are simple, and it is becoming more integrated into the govemment system. Characteristics of the indicator Sources of information The Mectizan should be controlled within a government system. This does p! have to be the system routinely used for the supply of other drugs. The system should be effective, uncomplicated and efficient The districU LGA level should fetch its Mectizan from the project level itself, by means of transport supplied and paid for by government at this level. t I Examination of: * All Mectizan ordering and stock control documentation at this level. * Vehicle log books and/ or trip authority forms. lnterviews with staff at this level (managers, pharmacist, drivers). Findings Describe the present situation: . SOCT supplies Mectizan directly to LGA. Staff at FLHF collect Mectizan from LGA. Finances for transportation of Mectizan from LGA to FLHL provided by state(APOC). ' Mectizan stored in FLHF clinics at district level and CDDs collect Mectizan form there. Mectizan not stored at LGA levels, usually distributed within 2 days. . Bulk receiving forms available at all levels but ordering forms absent. lf the Mectizan supply is not being administered within a government system . Why is this? . \Mrich steps are being taken to improve the situation? Analysis I \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likelv to affect sustainability I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 20 I 5. lndicatore of actlvlties and p;pcesseg: t1aining and HSAM 5.1 Check whether etaff membens at this level are being used appropriately as trainers. This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information Staff should routinely only train staff at the FLHF level, and not at the community level. Staff should have empowered staff at the FLHF level to see to their own training needs as much as possible, and to conduct training activities at the community level independently. \Mrerever possible staff at this level should conduct their own training for CDTI, if they have need for such training. Examination of training materials, plans/ programmes, reports: * At this level. * At the FLHF level. lnterviews with:x Staff at this level (the trainers). * Staff at FLHF level (the trainees). * CDDs. Findings Describe the present situation: ' LOCT is responsible for training District level health personnel, occasionally he assists . training at community level. Staff at the FLHF and LOCT(at that level) train the CDDs Describe the situation the year before lf staff members are not being used appropriately: ' Why is this? \Mich steps are being taken to improve the situation? Analysis I Wren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. x Reasons for poor performance (if any); steps being taken to improve it; and how this is likelv to affect sustainability. Examine the trend in the way in which staff are being used as trainers: I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 2t ! 5.2 Check whether training is being planned and carried out in an efficient and integrated manner. This indicator assesses whether the programme is functioning efficiently. Characteristlcs of the indlcator Sources of information There should be an objective need for each episode of training. This means there should be evidence that staff to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Repeat training of already skilled staff should not happen. lf circumstances permit training for CDTI should be integrated with other training, e.g. in in-service training programmes. Resources for training (human, transport etc.) should be efficiently used: * Using as few staff members as possible. * Using as little time as possible (without sacrificing quality) * Choosing the most cost-effective site etc. Examination of training materials, plans/ programmes, reports: * At this level. * At the FLHF level. lnterviews with: * Staff at this level (the trainers). * Staff at FLHF level (the trainees). * CDDs. I Find Describe the present situation: . Training is conducted annually for new staff and last for about 2 days and refresher retraining for old staff and CDDs. . CDDs are retrained before each distribution. ' Training materials on CDTI scanty at all levels , Health education on CDTI is occasionally integrated with LF. This takes place before, during and after distribution. Describe the situation the year before: lf training is not efficient and integrated ' \Mty is this? \Mrich steps are being taken to improve the situation? Analysis \Men writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabiliU. I Examine the trend in training activities - is it becoming more efficient and integrated? t Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 22 n 5.3 Check whether staff at this level is planning and carrying out HSAM activities in an efficient manner. This indicator assesses whether the programme is functioning efficiently and effectively, and whether managers are taking ownership of the programme. Characteristics of the indicator Sources of information I Staff members identify situations where decision makers lack information abouU commitment to CDTI, and undertake activities to inform and persuade these persons. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. Such activities should only be carried out at this district/ LGA level, and at times at the FLHF level (but only when staff at that level asks for help). There is evidence that these HSAM activities have been effective and have led to action. I Examination of:{' HSAM plans/ programmes and reports. * Year plans and annual reports. * Minutes of planning meetings. lnterviews with:x Staff (programme and management) at this level. * Civil authorities at this level * Staff and civil authorities at the FLHF level. I Findings Describe the present situation: . LOCT at LGA level undertakes activities to persuade LGA management staff on CDTI . Staff at FLHF level occasionally invited by community leaders to carry out HSAM activities before distribution. . Staff at FLHF are not very knowledgeable about CDTI. . Some CDDS carry out HSAM activities before Mectizan distribution. . Community leaders and CDDs carry out HSAM activities during and after distribution especially for refusals and those with adverse reactions. ' Community members are confusing the health education messages for LF and Oncho and they are not sure for how long the drugs will be taken. Describe the situation the year before: lf planning and implementation of CDTI is not part of the overall year plan . Why is this? . \Mtich steps are being taken to improve the situation? Analysis I \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Examine the trend in HSAM activities - is it becoming more efficient? Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 23 l 6. indlcitors of resourtes: flnanclal 6.1 Check whether appropriate amounts are budgeted for planned CDTI activities at this level. This indicator assesses whether the programme is functioning etrlciently. Characterlstics of the indicator Sources of information The costs for each CDTI activity in the year plan at this level should be clearly spelt out in a budget. There is evidence of a cost reduction/ containment strategy (e.9. targeted training, HMAS and monitoring/ supervision; training conducted at FLHF leveletc). Managers at this level should have a clear estimate of the funds that will be available to them for CDTI in the coming year, and should be able to justify this belief. The total amount budgeted for in the year plan should fall within this estimated income. I Examination of the budget documents. lnterviews with health service and local government managers at this level: * Technical managers. * Treasurer/ administrator. Findings The budget and estimated income: , At the previous distribution Salaries of LOCTS and FLHF staff are paid by LGA even though they are presently in 6 months arrears of payment. Management at LGA level are not aware of funds available to them for CDTI annually I For the previous year: For the year before that: lf budgeting has been inappropriate: . Why is this? \Mich steps are being taken to improve the situation? Analysis I \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability Examine the trend in the budgeted amount and the expected income: I Your overalljudgement: is this indicator for sustainabili$ being achieved? Fully Highly Moderately Slightly Not at all Not applicable 24 6.2 Check whether the government at this leve! is budgeting and disbursing increasing amounts for CDTI yearly. This indicator assesses whether the programme is becoming integrated, and whether management is beginning to accept ownership of the programme and can mobilise the resourcelr it needs. Characteristics of the indicator Sources of information The relative budgetary contributions of the local government and other partners to CDTI should be clearly spelt out. The amount that the government has budgeted in one or more specific CDTI budget lines should be increasing yearly. By the end of Year 5 of APOC funding the bulk of CDTlexpenses at this level should be met from local government funds. The amounts actuallv disbursed from such budget lines should be increasing yearly, as a proportion of total expenses. (Note that actual disbursement is more important than budgeting, and is a real sign of political commitment). I Examination of: * Budget documents (government and NGDO) * Records of disbursement and expenditure (ledgers, orders, approvals for expenditure etc.) lnterviews with health service, local government and NGDO managers at this level:x Technicalmanagers. * Treasurer/administrator. I Findings The budget and disbursements: ' For this year: No other financial commitment to CDTI programme at the LGA management level including release of counterpart funding since inception of CDTI programmes in LGA State takes responsibility for payment of per diem to staff involved in CDTI activities during trainings and supervision, mop ups, delivery and collection of Mectizan at LGA levels, and provision of health education materials t For the previous year: lf the government proportion of expenditure is not increasing proportionately: . Why is this? \Mtich steps are being taken to improve the situation? \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Examine the trend in govemment budgeting and disbursements Your overalljudgement is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 25 I I Characteristics of the indicator Sources of information lf there is a shortfall the management should have specific and realistic plans to bridge it. lf it is planned that non-government sources of funding are to be used after APOC funding ends, written commitment for this should have been obtained at the highest level in these donor organisations. lnspection of:x The budget documents (government and NGDO). * Records of expenditure (ledgers, orders, approvals for expenditure etc.). * Letters of agreement. lnterviews with health service and local government managers at this level: * Technicalmanagers. * Treasurer/administrator. 6.3 Check whether in case of a deficit between estimated costs and the amount provided by the de ndable provlsion is bei made to meet it This indicatorassesses whether management is able to mobilise the resources it needs, as well as its commitment to ownership. Findings Describe the present situation: , Allplanned CDTI programmes supported from State level(APOC) Describe the situation the previous year: lf the shortfall cannot be met: . \Nhy is this? \A/hich steps are being taken to improve the situation? Analysis \Men writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. . Examine the trend in shortfall and how it is to be supplemented Your overalljudgement: is this indicator for sustainability being achieved?I Slightly Not at all Not applicableFully Highly Moderately 26 I 6.4 Check whether funds disbursed for CDTI from the budget at this level are efficiently managed. This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information The budget holder should be using a control system with the following elements: * Approval of each item of expenditure. x Allocation of expenditure against specific budget headings. * Regular calculation of residual amounts under budget headings. All the funds released yearly should be spent as budgeted. lnspection ot * The budget documents (government and NGDO). * Financialcontrolrecords(ledgers, orders, approvals for expenditure etc.). lnterviews with health service and local government managers at this level: * Technicalmanagers. * Treasurer/administrator. I Findings Describe the present situation . Approval of expenditure: I No CDTI related expenditure in the past five years was approved at the LGA level, all activities are funded through the state (APOC). Allocation of expenditure: Regular insight into budget line balances: lf the funds are not being well managed: . VVhy is this? Which steps are being taken to improve the situation? \Mren writing the report you have to summarise:t The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgemenl is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 27 7. tndlcators of resources: transport and other ma-te-rlal resouicei 7.1 Check whether adequate and appropriate transport and materials are available for necsssary CDTI actlvities at this level. This indicator assesses whether the programme is functioning effectively, and whether it is able to mobilise the resources needs. Sources of informationCharacteristlcs of the indicator lnspection of: * Each vehicle in the pool: its source; its functional status. * Stocks of materials for training and HSAM. lnterviews with managers at this level: transport officers, programme managers. There are adequate numbers of appropriate, functional vehicles available for necessary CDTI activities. The running costs for these vehicles are met by the government. There are sufficient materials available for training and HSAM. Find Describe the adequacy of the present vehicles and materials, considering the work still to be done in the coming 5-10 years: . Health education materials not adequate, last time received was in 2001 . Transport adequate for CDTI activities, if continuously maintained. . Replacements of vehicle spare parts carried out by State(APOC) and minor repairs carried by individuals. Source * les 3 7 No. maintained in the next two ars * APOC, MoH, NGDO, other lspecify) * * ls it working, and is there enough of it for Motor bike situation: No ln condition2 APOC 2LF 1LF 5APOC conditionln Vehicles in good condition if adequately Source * State the Describe the Type of vehicle Functional status and adequacy for CDTI tasks * * Health education materials not adequate, last time received was in 2001 Functional status and adequacy for CDTI tasks* * Training/ HSAM material Various H/Ed materials lf transport and materials are inadequate and funded from non-government sources: . t /hy is this? . \Mich steps are being taken to improve the situation? \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is to affect sustainabil Your overalljudgement: is this indicator for sustainability being achieved?t Slightly Not at all Not applicableFully Highly Moderately 28 tr 7.2 Check whether at this level is and maintained Th is indicator assesses the programme is effective I y a n d effi c i e ntly. Characteristics of the indicator Sources of information There is a routine maintenance schedule for each vehicle, which is adhered to and recorded. This includes weekly driver maintenance, scheduled garage servicing, and replacement of worn tyres. The costs for vehicle and equipment maintenance and repair are met by the government. Repairs are rapidly and efficiently done. Staff have ways of coping when vehicles break down or are not available, so that CDTI activities are not disrupted. lnspection of: * Vehicle and equipment maintenance schedules. * Vehicle and equipment maintenance records. lnterviews with managers at this level: transport officers, programme manaqers. Findinge Describe the present situation Transport adequate for CDTI activities, if continuously maintained. Replacements of vehicle spare parts carried out by State(APOC) and minor repairs carried by individuals using the transport. Transportation at the LGA and FHLF are used in an integrated manner for all programmes. Make particular enquiries about the ability of the government to pay for maintenance, repairs and tyre replacement: lf the vehicles are not being well maintained, and/ or the government is not paying . \A/hy is this? ! \Mtich steps are being taken to improve the situation? Analysis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. ! Your overalljudgement is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 29 I 7.3 Check whether the transport avaitable at thls level is appropriately managed and used, in an lntegrated way indicator assesses whefher the programme is functioning efficlently. Characteristics of the indicator Sources of information I Transport is used at this level, and to undertake support activities at the FLHF level. lt should not be used for CDTI implementation activities at the community level. Trips made for CDTI purposes should be properly authorised in writing by the relevant official. Each trip undertaken should be recorded in a log book. Transport provided for CDTI, and that provided for other programmes, should be combined as a poolto be used for legitimate activities of all programmes at this level. lnspection of vehicle control documents: * Copies of trip authorities (also noting destination and purpose) * Log books. * The supervision plan/ matrix. lnterviews with managers at this level: transport officers, programme managers. Findings Describe the present situation: . Trips made for CDTI purposes at LGA level are authorised by the PHC Coordinator and by the District Health Supervisor at the FLHF level. . ltinerary for planned activities available even though no log book was seen. . CDTI and other programme (eg LF) provided transports are used in an integrated manner for all programmes activities at the LGA and FLHF levels. lf the transport is not being well managed: . Vvhy is this? Wrich steps are being taken to improve the situation? Analysis Vvhen writing the report you have to summarise: x The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 30 I -t l 7.4 Check whether there are appropriate and realistic plans for the replacement of transport and materials when APOC support comes to an end. This indicator assesses whether the programme managers are taking ownership of the programme, and are able to find resources for it. Characteristics of the indicator Sources of information Management should know that replacements will be needed before the end of the programme, and have specific, realistic plans to meet the need at that time. It should be planned that the govemment will: * Provide replacements for vehicles. x Maintain existing vehicles. * Provide stationery and materials for training and HSAM. lf it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in these donor organisations. ! lnspection of letters of agreement. lnterviews with: * Programme managers at this level: administrators, technical managers. * NGDO project managers. * High-ranking local qovernment officials. Findings Describe the present situation: . No opinion expressed on ability of LGA management to replace available vehicles after APOC lf the plans for replacing vehicles and materials are unsatisfactory: . !\lhy is this? LGAs involvement in boarder clashes has resulted in extensive refuges problems being experienced and thus financial constrain. . \Mrbh steps are being taken to improve the situation? They are hoping things will improve in the next 2 years. Analysis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affec{ sustainability. ! Your overalljudgement is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 3l I I 8.1 Check whether staff at this level is skilled and knowledgeable, regarding the implementatlon of CDTI in lts area of operatlon. 8. Indlcatorc of resourcee: human resources assesses whether the programme has been able to develop sufficient resources for itself. Characteristics of the indicator Sources of information Staff should have enough knowledge and skill to undertake all the key CDTI activities themselves: planning, training, HSAM, ensurin g Mectizan supply, monitorin g/ supervision. Staff at this level should remain in one post for at least five years. There should be immediate training (in CDTI) of new, unskilled project staff members who have CDTI responsibilities. I lnspection of: * Staff files. * Monitoring reports. * Activity reports. lnterviews with * Managers and other staff at this level. x Staff at the project level * Staff at the FLHF level. Findi . lnformation about staff stability and in-service training: , Senior LOCT also the CDTI focal person has been holding position for 5 years, has acquired enough skill for CDTI activities, remaining 4 LOCTs have been in office for 2-3 years. , LOCT staff in Ballang Shippang(FLHF) appear un skilled . Better stability of staff at the LOCT level compared with FLHF where District Health Officer is only 6months old . Unskilled staff at LGA level trained annually 5 5 5 5 Describe the present Area of skill Planning Training and HSAM Mon itoring/ su peruision No. of skilled persons Level of skill: is it adequate to perform the job? . Particulars of current staff Mectizan ordeing/ distibution lf the staff at this level lack skills, and are often transferred . \ /hy is this? \Mich steps are being taken to improve the situation? Analysis \A/tren writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 32 I 8.2 Check whether staff memberc at this level are committed to their CDTI work. This indicator assesses whether the programme has been able to develop sufficient resources for itself. Characteristics of the indicator Sourcee of information I Staff members express satisfaction with their present responsibilities There is evidence of specific motivational practices and rewards within the programme: awards, financial incentives, compensation in cash or kind. Salaries/ wages and allowances are paid regularly. Staff members mention non-financial rewards inherent in CDTI work. There is evidence from partners and workers at the FLHF level that staff members are committed to their CDT work. lnspection of: * Staff files (for performance records and awards). * Documentation about incentive schemes. lnterviews with * Managers and other staff at this level. * Local government officials. x NGDO project staff. * Staff at FLHF level. Findings Describe the present situation: . Staff members expressed satisfaction with their responsibilities, Senior LOCT received verbal praise for his contribution toward CDTI programme. . LGA staff involved in CDTI owed 6 month salary arrears by LGA management. . CDTI staff at LGA level receive allowances including mop ups after training and supervision of CDTI activities. . Staff at LGA level show good knowledge of CDTI activities. lf staff members appear to have little commitment to CDTI work . \Mry is this? \Mtich steps are being taken to improve the situation? Analysis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 33 I 9. lndlcators of lmpact: coverage 9.1 Check whether the districU LGA has a hicalcoverage rate. This indicator assesses whether the programme is effective - if the rate is poor the project is cleady struggling, and /ess sustainable. Characteristics of the indicator Sources of information a All sub-districts and communities identifled by the latest REMO should be under treatment (i.e. the geographical coverage rate is 100%). This rate should be stable or increasing. lnspection of: * Distribution reports and statistics at community level, for this districU LGA, for the past 3 years. * REMO list of endemic communities for this districU LGA. lnterviews with: * Staff at districU LGA level. * Staff at FLHF level. Findinss The geographical coverage situation in the districU LGA: ' At the last distribution: At least 90% of communities geographically covered, CDDs are selected to cover each hamlet The year before: The year before that: lf the geographical coverage rate is poor: . Why is this? t Vvhich steps are being taken to improve the situation? Analysis Vvhen writing the report you have to summarise the reasons for poor performance (if any);I steps being taken to improve it; and how this is likely to affect sustainability ! Vvhat is the trend in geographical coverage? I Your overall iudqement: is this indicator of sustainabiliW being ach ieved? Fully (100% of communities are doing cDTr) Highly (95-99o/o of communities - stable or increasing) Moderately (90- 94o/o of communities - stable or increasing) Slightly (85-89o/o of communities - stable or increasing) Negligibly (<85% of communities) 34 9.2 Check whether the districU LGA has a satisfactory therapeutic coverage rate. This indicator assesses whether the programme is effective - if the rate is poor the project is clearly struggling, and /ess sustainable. lnstrument 3: first line health facility (FLHF) level Characteristics of the indicator Sources of information Allcommunities in the districU LGA should have a therapeutic coverage rate of 65% or higher. These rates should be stable or increasing. t lnspection of: * Distribution reports and statistics at community level, for this districU LGA, for the past 3 years. x REMO list of endemic communities for this districU LGA. lnterviews with: * Staff at districU LGA level. * Staff at FLHF level. Findings The therapeutic coverage situation in the communities in the districU LGA: . At the last distribution: . Therapeutic coverage about 90% since 2001, no shortages reported, CDDs move form house to house for distribution and drugs reserved for at least 5 weeks for absentees. . Community heads ensure that all community members take the drug annually. . Coverage has been increasing by 10o/o due to introduction of LF programme The year before: The year before that: lf the therapeutic coverage rate is poor: . VVhy is this? Wrich steps are being taken to improve the situation? Analysis \Mten writing the report you have to summarise the reasons for poor performance (if any); steps being taken to improve it: and how this is likely to affect sustainabilitv. t \Mrat is the trend in therapeutic coverage? Your overalljudgement is this indicator of sustainability beins achieved?I Fully (100% of communities have a therapeutic coverage rate >65% - stable or increasing) Hishly (e0- 99% of communities) Moderately (80- 897o of communities) Slightly (70- 79o/o ol communities) Negligibly (<70o/o of communities) 35 -l NOTE: . This instrument evaluates the !eve!, which finally interacts with the villages and communities, in ensuring that CDTI takes place in all the communities in its area of operations. This level has different names in different countries. ln most countries there is a health centre- a clinic, or health centre, or dispensary, which we call a'first line health facility'. The health workers who work there are the ones who are responsible for training and supporting the CDDs in the villages. . When we speak of a FLHF we therefore mean: * The health facility and its staff. * The accompanying political/ administrative mechanisms between the districU LGA and community levels. . By 'FLHF team' is meant the group of persons working in the first-line health facility and in its catchment area. . The level below this one is of course the community, the villages. Here the CDDs -'community directed distributors'- live and work. We refer to this level as the'community' level. Geographical name of this FLHF Project: PLATEAU STATE, NIGERIA Researcher: The Team Date: 28- 04 - 2OO3 Abbreviations/ acronymsCDD community directed distributorCDT! community directed treatment with ivermectinFLHF first line health facilityHSAM health education, sensitisation, advocacy, mobilisation - i.e. activities which are aimed at getting all the key players to participate wholeheartedly inthe programmeNGDO non-governmenta! development organisation lndicitors of activities and proceises: ptanning The focus of this level's activities in CDTI The main function of this level isto work with the village communities, so that CDTI is established in them: . Mobilising them to become involved in CDTI, by selecting CDDs. . Training the CDDs and supporting them in their work.t Arranging a dependable supply of Mectizan for them, at the right time each year. .He them to collect and fonryard the data for their commu 1. 36 1.1 Check if there is a written an for CDTI in the FLHF area. This indicator assesses whether the programme is being planned in an effective and integrated manner, and whether management is beginning to accept ownership of the programme. Characteristics of the indicator Sources of information t There should be a written plan or timetable in existence, for the most recent round of CDTI (this is recommended). ldeally the plan should be integrated into the overall year plan for the FLHF area. ldeally CDTI should form part of the'minimum' or 'recommended' package for this level !nspection of: * The written year plans. * Minutes of planning meetings lnterviews with staff at this level.I Findings Describe the present situation: . There was no any written plan for CDT! activities prior lo2002. The LOCTs and State met in2002 for the first time to draw up a written work plan for CDT! activities for 2003. This plan has been discussed and shared with the districts. . The plan contains key CDTI activities, duration and budget by activity. o Mectizan distribution is integrated into Lymphatic Filariasis and Schistosomiasis control programme but not fully into other PHC activities lf there is no written plan for CDTI . VVhy is this? The officers did not know they were to plan for CDTI activities and saw CDTI implementation as a top to bottom-structured programme where allthe plans were made and forwarded to the LGAs for implementation. a a ! Are any steps being taken to improve the situation? The DHS had already been sensitised on the need for a plan and were involved in developing an LGA plan for 2003 Analysis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 2. lndicatorC of aitivities 37 leadership 2.1 Check whether the FLHF management team is taking full responsibility for GDTI at this level, in an integrated manner. This indicator assesses whether the CDTI proiect is integrated into the health sysfem, and whether management is beginning to accept ownership of the programme. Characteristics of the indicator Sources of information The FLHF management team and all health staff at this level consider the program as theirs and are initiating the key CDT! activities: planning, monitoring/ supervision, training, HSAM, Mectizan ordering/ distribution. The political head/ senior politician at this level should know about CDTI and have participated in some CDTI activities. . lnspection of year plans. . lnterviews with: x Management team at this level. * Senior political figures at this level.x Staff at the districU LGA level. Findinqs Describe the present sttuatton a The FLHF consider CDTI to be their programme and usually initiates and implements activities, once they get the drugs. The district political leaders show interest in CDTI and participate in mobilizing communities under them for Mectizan distribution. lf DMT is not taking full responsibility for CDTI . !ryhy is this? I Are any steps being taken to improve the situation? Analysis When writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability I Your overall judgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 38 i3. lndicatorc of activities and procesqqgi-!!o_0!to!!rl!U-sqIe!'t1stell 3 3.1 Check whether routine and necessary data concerning CDTI activities at this level are being transmitted entirely within the government system. This indicator assesses whetherthe programme is becoming more integrated intothe gove rn me nt he alth sysfem. Gharacteristics of the indicator Sources of information The reporting process should be within the government system, not using other resources. Data being transmitted includes: coverage reports; d istri buti on reports; Mectizan statistics; trai n ing reports. ' Examination of reports and report forms. . lnterviews with: * Managers and staff at this level. * Staff at the districU LGA level. Findings a a a a a Describe the present situation: The DHS goes round to collect treatment reports and keeps a copy of the community treatment summary form at the facility. There are times when the reports to the LGAs are late due to delays in submission of reports by CDDs. The DHS monitors and supervises officers in charge of facilities who in turn supervise and monitor CDTI activities in the communities. There are however no checklist to confirm this and reports indicate that some health facility staff do not supervise their communities. The DHS also use the opportunity of their monitoring visits to carry out other health care activities, suggesting some form of integration. The DHS are also involved in co-treatment of LF and Onchocerciasis in their districts !f CDTI data at this level are not being processed within government system: . \Mry is this? CDTI data is well processed at this level, but the delays in submitting reports by CDDs affect the system. This is attributed to late distribution, which is affected by the rains. a a t Are any steps being taken to improve the situation? There are plans to provide Mectizan early to communities to ensure that communities treat early and reports submitted in good time. Analysis \A/hen writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overalljudgement is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 39 3.2 Check whether health staff at this level is routinely and efficiently supervising CDTI activ at the communities on site in an rated manner This indicatorassesses whetherthe CDTI programme is being implemented effi cientl y. Characteristics of the indicator Sources of information Although one routine supervision visit per community per year is necessary, supervision visits should focus more on communities where there are proven problems - each supervision visit must be justified. During visits to communities FLHF staff should turn their attention to as many health related programmes and problems as possible. t Examination of: * Supervisorychecklists, plans, itineraries and reports. * Log books. lnterviews with: * Staff at this level. * Village heads and CDDs. I Findings Describe the present situation: . Monitoring and supervision is mainly routine and is canied out only during distribution. Facilities are visited with spot checks to communities. . Supervision is integrated at this level but is mainly funded by the project lf health staff persons are not routinely and efficiently supervising CDTI . VUhy is this? Are any steps being taken to improve the situation? Analysis \Mren writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. 'r Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 40 i T 3.3 Check whether there is a routine process of management of problems andsuccesses, which are indicated by the monitoring system (coverage data, visits and reports) This indicator assesses whetherthe programme rs being implemented efficiently and effectively, and whether managemenf is beginning to accept ownership of the Sources of information t Examination of the following documents: * Year plans and annual reports. * Minutes of staff/ planning meetings. * Reports of previous monitoring exercises. 'r Letters of commendation. lnterviews with: * Staff at this level: in-charge and others. * Community leaders and CDDs. Characteristics of the indicator As soon as problems are identified through supervisory visits, coverage data etc. (e.9. communities with low coverage) health staff at this level should deal with them. \Mrere relevant such problems should be passed on to the relevant community to deal with, with the necessary support - thus empowering communities to make decisions on CDTI and cope with problems. Successes should be noted and reported, and appropriate feedback given to communities. There should be evidence of action taken based on recommendations in previous monitoring exercises. I ! I Findings a Describe the present situation: There are no guidelines for dealing with problems identified during monitoring and supervisory visits. There are no checklists or written reports. Such problems are communicated verbally to the officer in charge of the facility but there is no feedback mechanism. There is no reward system in place to encourage health workers that show outstanding commitment to CDTI in the facilities. a a !f there is no routine process of managing problems and successes: . \Nhy is this? There are no explanations why there is no routine process of managing problems and successes, but it seems this has not been the practice in the LGA. Are any steps being taken to improve the situation? There are no definite steps are in place or being proposed to improve the situation. Analysis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overalljudgement is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 4t r :'t n4. lndicators of activities and processes: Mectlzan@ procurement and distribution 4.1 Check whether sufficient Mectizan is being ordered annually, and in good time. This indicator assesses whether the programme is functioning effectively. Characteristics of the indicator Sources of information The order forms for the FLHF area exist, and orders should be based on the requests from the community. The Mectizan should be available at this level in time for distribution at the time requested by the communities. There should be no reports of shortages and/or late supply. lf there have been shortages, there should be specific plans to remedy them. Examination of all Mectizan ordering and stock control documentation at this level. lnterviews with: * Staff at this level (managers and pharmacist). * Mllaqe leaders and CDDs. Findinqs \Mat happened at: . The last round of treatment? a a a There are no standard and rational means of determining the number of drugs required by communities. lnstead drugs supply is based on the previous year supply and in few case census registration figures. The drugs, in most cases, are not available at the period the community require them, which is before active farming season (Aprilto December). There were no reported shortages in most communities. The round of the year before? Same as above The round the year before that? Same as above I lf sufficient Mectizan is not being ordered annually: . \Nhy is this? Are any steps being taken to improve the situation? Analysis \A/lren writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. \Mrat is the trend in Mectizan ordering and suppky? Your overalljudgement is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 42 4.2 Check whether Mectizan is being collected, stored and effectively administered within the government system at this level. This indicator assesses whether the programme is functioning efficiently, ifs processes are simple, and it is becoming more integrated into the govemment system. Sources of information . Examination of: * All Mectizan ordering and stock control documentation at this level. * Vehicle log books and/ or trip authority forms. . lnterviews with staff at this leve! (managers, pharmacist, drivers). Gharacteristics of the indicator The Mectizan should be controlled within a govemment system. This does not have to be the system routinely used for the supply of other drugs. The system should be effective, uncomplicated and efficient The FLHF level should fetch its Mectizan from the districU LGA level itself, by means of transport supplied and paid for by government at this level. Communities should fetch their Mectizan from the FLHF themselves. However in situations where villages are very far from health centres, or where it is easy/ practicable for FLHF staff to deliver the Mectizan, they may help with the delivery. I Findings Describe the present situation: a Mectizan is stored at the drug store in district health facility and issued out almost immediately to the communities. a Health facility staff collect drugs for their communities, while CDDs collect drugs on behalf of their communities from the health facility closest to them There is a good Mectizan inventory of drugs received and balances returned.a lf Mectizan is not being received and stored within the government system . \Nhy is this? Are any steps being taken to improve the situation? Analysis \A/hen writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overall judgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 43 I lndicators of activities and This indicator assesses whether the trai and HSAM is Sources of information . lnterviews with: * Staff at this level (the trainers). * CDDs (the trainees). . Examination of training materials, plans/ programmes, at this level 5.1 Check whether trainin ts ned and carried out in an efficient manner Characteristics of the indicator There should be an objective need for each episode of training means there should be evidence that CDDs to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Resources for training (human, transport etc.) should be efficiently used: Using as few staff members as possible. Using as little time as possible (without sacrificing quality) Choosing the most cost-effective site e!q. * rf * I This Findings Descrrbe the present srtuation . The DHS train the Officers in charge of the FLHF who in turn train the CDDs. ln most cases training was found to be routine. ln instances where this is not routine, emphasis are on training new CDDs. . Training is usually carried out at the district health facility. . Training is not integrated at this level . Training of health workers is restricted to one per health facility while other health workers are not directly or indirectly trained in CDTI . Training is entirely funded by the project (APOC and NGDO) Describe the situation the year before: Same as above lf training is not efficiently done in an integrated manner: . !ryhy is this? !ntegrated training has never been part of the control programme and thus never practiced. Are any steps being taken to improve the situation? o There are no deliberate plans to commence integrated training a Analysis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likelv to affect sustainability. Wrat is the trend in the way training is done - the method and content? Your overalljudgement: is this indicator for sustainabili$ being achieved?t Fully Highly Moderately Slightly Not at all Not applicable 44 I 5.2 Check whether staff at this level is planning and carrying out HSAM activities in an efficient manner. This indicator assesses whether the programme is functioning efficiently and effectively, and whether managers are taking ownership of the programme. Characteristics of the indicator Sources of information Staff members identifu situations where decision makers lack information abouU commitment to CDTI, and undertake activities to inform and persuade these persons. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. There is evidence that these HSAM activities have been effective and have led to action. lnterviews with:x Staff at this level * Givil authorities at this level * Community leaders and CDDs. Examination of: * HSAM plans/ programmes and reports.x Year plans and annual reports. t Findings Describe the present situation: o A team made of the LOCT, DHS and CDDs carries out HSAM of communities annually as routine. lts not targeted at issues. . lt is on only few occasions that HSAM is targeted at issues like refusal of treatment due to side effects. . HSAM is completely supported by the project. o HSAM is not properly planned i.e. integration of PHC . \Mrere HSAM is jointly done for LF and Oncho, Oncho becomes subsumed under LF because of perceived gonid burden of LF Describe the situation the year before: Same as above lf staff is not engaged in HSAM: . Why is this? Are any steps being taken to improve the situation?I Analysis I \Men writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved.x Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Examine the trend in HSAM activities - is it becoming more efficient? Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 45 I 6. lndlcators of resoutces: flnancla! 6.1 Check whether the costs involved in planned CDTI activities at this level are defined and for. This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information The costs for each CDTI related activity in the year plan should be clearly spelt out in a budget. These activities include monitoring/supervision, training, HSAM, and arranging Mectizan supply. The staff should be able to justiff the amount they plan to use. There should be evidence of cost containment (e.g. by targeting training, HMAS and supervision). Examination of the budget documents (government and NGDO). lnterviews with: * FLHF team managers (leader, treasurer). * Local government managers at this level (chairperson, admi nistrator, treasu rer). I Findings \l1/trat happened at: . The last round of treatment? a There have not been budgeted plans for CDTI activities at this level. Planning is top to bottom and just implemented at this level. The input of the DHS has not always been sought and so could not justify the amount available for CDT! at this level. a The round of the year before? lf costs involved in CDTI related activities are not clearly defined . !ryhy is this? a Budgeted plans for CDTI are regarded as a responsibili$ for the LGA and DHS are not involved. Are any steps being taken to improve the situation? a The DHS were invited to participate in developing a budgeted plan for the LGA for 20003 but not specific to the level. Analvsis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. x Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Wrat is the trend in the way costing for CDTI related activities is done? Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 46 I 6.2 Check whether sufficient funds to cover these costs are fully or increasingly disbursed from FLHF and/or districU LGA resources. This indicator assesses whether the programme is becoming integrated, and whether management is beginning to accept ownership of the programme and can mobilise the resources it needs. Characteristics of the indicator Sources of information ! Funding disbursed is enough to enable targeted, essential CDTI activities at this level to be carried out. The relative contributions of all sources of funding should be clearly spelt out. The proportion provided by the government (FLHF and/ or districU LGA levels) should be the maior one by now. Examination of: * Budget documents (government and NGDO) * Records of disbursement and expenditure (ledgers, orders, approvals for expenditure etc.) lnterviews with: * DistricU LGA level managers (technical and administrative). * NGDO managers at this level.{' FLHF team managers (leader, treasurer). * Local government managers at this level (chairperson, administrator, treasurer). Findings How much was provided by the government: , Last round of treatment? . The Local Government has never provided funds for CDTI activities at this level. CDTI activities are completely donor funded since the inception of the programme (APOC and NGDO) . The round of the year before? . The round the year before that? !f the proportion supplied by the government is not the major one by now . \Mry is this? The LGA has not been forth coming in provision of funds because of the assumption that APOC has sufficient funds for the programme a a Are any steps being taken to improve the situation? A budgeted plan has been forwarded to the LGA for consideration for 2003. Some LGAs have approved and are making efforts for the first time to release qqgnlerpart funds. Analysis \Mren writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. \Mtat is the trend in the relative proportion of resources contributed by the official health service? Your overalljudgement: is this indicator for sustainability being achieved?! Fully Highly Moderately Slightly Not at all Not applicable 47 It tr 7 lndlcators of resources: transport and other materi al resources 7.5 Check whether adequate and appropriate transport and materials are available for n GDT! activities at this level This indicator assesses whether the programme ls functioning effectively, and whether it is able to mobilise the resources if needs. Characteristics of the indicator Sources of information ! There are adequate numbers of appropriate, functional vehicles (of any type) available for necessary CDTI activities. The running costs for this transport are met by a dependable source (e.9. the government). There are sufficient materials available for training and HSAM. lnspection of: * Each vehicle being used: its source; its functional status. * Stocks of materials for training and HSAM. lnterviews with the FLHF management team. I Findinqs Describe the situation: Type of transport No. Source* Functional status and adequacy for CDTI tasks* M 1 APOC Fu functional es 1 APOC functional es 4 GoWNID functional Training/ HSAM material No. Source* Functional status and adequacy for CDTI tasks* Posters APOC CDD information booklet APOC Adequate * APOC, MoH, NGDO, other (specify) ** /s rf working, and is there enough of it for the Describe the adequacy of the present vehicles and materials, considering the work still to be done in the coming 5-10 years: . The vehicles are inadequate and materials are provided on an annual basis. . The transports available at this level are used for all PHC activities CDTI inclusive lf transport and materials are inadequate and funded from non-government sources: ' \Nhy is this? . Could not give reasons. . Wtrich steps are being taken to improve the situation? Analysis \Mren writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability Your overalljudgement: is this indicator for sustainability being achieved?! Fully Highly Moderately Slightly Not at all Not applicable 48 7.6 Check whether transport at this level is adequately and appropriately maintained. This indicator assesses whether the programme is functioning effectively and Sources of information !nspection of: * Vehicle and equipment maintenance schedules. * Vehicle and equipment maintenance records. lnterviews with the FLHF team Gharacteristics of the indicator . There is a routine maintenance schedule for vehicles (where relevant), which is adhered to and recorded. . The costs for vehicle and equipment maintenance and repair are met by dependable sources (e.9. the government). . Repairs are rapidly and efficiently done. ' Staff have ways of coping when transport breaks down or is not available, so that CDTI activities are not disrupted. Findings Describe the present situation a The LGA canies out major repairs and maintenance when the vehicle is to be used for NlD, the DHS is responsible for minor repairs from his salary and proceeds from sale of PHC drugs in the facility. Make particular enquiries about the ability of the government to pay for maintenance, repairs and tyre replacement. Government can pay for maintenance, replacement of parts and repairs if they are committed to the programme. a a lf the vehicles are not being well maintained, and/ or the government is not paying . \Mry is this? a The LGAs complain of lack of funds and are barely able to pay salaries I \Mich steps are being taken to improve the situation? o This has been included in the plan for 2003 to 2007. o Maintenance and repairs can rapidly and efficiently be done by the LGAs if CDTI is fully integrated in PHC activities and LF/Schistosomiasis control programme Analysis \Mren writing the report you have to summarise: * The evidence about how we!!this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 49 -t (Check whether the transport available at this level is appropriately managed and used, in an integrated way. This indicator assesses whetherthe programme is functioning efficiently. Characteristics of the indicator Sources of information Trips made for CDTI purposes should be properly authorised in writing by the relevant official. Each trip undertaken should be recorded in a log book. Transport provided for CDTI, and that provided for other programmes, should be combined as a pool to be used for legitimate activities of all programmes at this level. lnspection of vehicle control documents: * Copies of trip authorities (also noting destination and purpose) * Log books. x The supervision plad matrix. lnterviews with the FLHF management team. Findinqs Describe the present situation a There are no guidelines for movement or trips. No log book or travel authorization is used at this level. a Transport provided for CDTI is used for other PHC activities at this level lf the transport is not being well managed: . Vvhy is this? a Trip authorization and use of vehicle control documents has never been the practice at this level. . \Mrich steps are being taken to improve the situation? None Analysis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overall judgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 50 ,//a 7.7 Check whether there are appropriate and realistic plans for the replacement of transport and materials when APOG support comes to an end. This indicator assesses whether the programme managers are taking ownership of the programme, and are able to find resourc* for it Characteristics of the indicator Sources of information I Management should know that replacements will be needed before the end of the programme, and have specific, realistic plans to meet the need at that time. !t should be planned that the government will:x Provide replacements for vehicles. * Maintain existing vehicles. * Provide stationery and materials for training and HSAM. !f it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in these donor organisations. lnspection of letters of agreement. lnterviews with: * The FLHF management team. * NGDO project managers. * High-ranking local government officials. Findinqs Describe the present situation: a There are no definite plans to replace the vehicles and materials at the end of APOC support. The level is not involved in the plan to replace and maintain transport by the LGAs a !f the plans for replacing vehicles and materials are unsatisfactory: . !ryhy is this? ! \Mich steps are being taken to improve the situation? Analysis \Mren writing the report you have to summarise: x The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overalljudgement is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at al! Not applicable 5l I I \ L\ 8. lndicators of resources: human resources 8.1 Check whether the team at this level is skilled and knowledgeable, regarding the implementation of CDTI in its area of This indicator assesses whether the programme has been able to develop sufficient resources for itself. Sources of information . lnterviews with * Managers and other staff at this level x Staff at the districU LGA level. * Mllage leaders and CDDs. . lnspection of: * Staff files.x Monitoring reports. * Activity reports. Gharacteristics of the indicator Staff should have enough knowledge and skill to undertake allthe key CDTI activities themselves: planning, training, HSAM, ensuring Mectizan supply, monitoring/ supervision. Staff at this level should remain in one post for at least five years. There should be immediate training (in CDT!) of new, unskilled project staff members who have CDTI responsibilities. Findings Describe the present situation . Particulars of current staff Area of skill No. of Level of skill: is it adequate to perform skilled the job? Planning The DHS and Officers in charge of FLHF are Training and HSAM 25 adequately skilled in CDTI activities. are all Monitoing/ superuision 25 trained health personneland have been trained Mectizan ordering/ distibution 25 in CDT! implementation . lnformation about stability and in-service training: The CHEWs at the lower cadre are more stable and not regularly transferred out of their stations. Most DHS are higher officers and could be transferred out of the LGA at any time. However, training at this level is restricted to the Diskict Health Supervisor, who is the most senior officer. The roles of LOCT as distinct from DHS are not clear and amount to duplication of duties and/or activities. a a a lf the staff at this level lack skills, and are often transferred: ' \ /hy is this? . The LOCT has not been able to supervise the use DHS effectively because they are junior to the DHS ' \Mrich steps are being taken to improve the situation? Analvsis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved.t' Reasons for poor performance (if any); steps being taken to improve it: and how this is likely to affect sustainability. Your overall judgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 52 I \l 10. lndicqtors of impact: coyqrage 9.1 Check whether the geographical coverage in the FLHF area is satisfactory This indicator assesses whether the project is effective - if the rafe is poor the project is clearly struggling, and /ess sustainable. Characteristics of the indicator Sources of information I All villages identified by the latest REMO should be under treatment (i.e. geographical coverage should be maintained at 100%). The rate should be stable or increasing. lnspection ot * Distribution reports and statistics at community level, for this FLHF area, for the past 3 years. * REMO list of endemic communities for this FLHF area. lnterviews with staff at FLHF leve!.I Findings The geographic coverage situation: . At the last distribution: a The geographical coverage is satisfactory at this level based on distribution reports. There are however no REMO results at the FLHF level to verify this. The year before The year before that: lf geographical coverage is poor: . Why is this? Are any steps being taken to improve the situation? Analvsis \Mren writing the report you have to summarise the reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. What is the trend in geographical coverage?T ! Your overall iudqement is this indicator of sustainabilitv beinq achieved? Fully (100% of communities are doing cDTr) Hishly (9$99% of communities - stable or increasing) Moderately (90- 94o/o of communities - stable or increasinq) Slightly (85-89% of communities - stable or increasing) Negligibly (<85% of communities) 53 lnstrument 4: community level NOTE: . This instrument evaluates the CDTI programme at the level of villages and communities, where the actual distribution of Mectizan takes place. We use the term 'community' to refer to both villages (in societies where there are we!!-defined villages) and communities vfiere family groups are fairly isolated from each other, and do not live in a'village' in the accepted geographical sense of the word. . ln these communities we find the following persons are involved in the CDTI programme: * The community or village leader*rip - both traditional and elected. * The community directed distributors (CDDs) -the persons who have been selected by the community to do the distribution of Mectizan. * The other, 'ordinary'community members, who take the Mectizan yearly. !n this document these persons wi!! be referred to as 'community members'. . When collecting information from 'ordinary'community members discussion groups should be conducted. The focus of this level's activities in CDTI The main function of this level is to distribute lhe Mectizan yearly to the community members: . Communities select CDDs, who are supported by the leadership and the other community members. . The CDDs update the community census every year; distribute the Mectizan appropriately; and send a report on the distribution to the FLHF level. Geographical name of this community/ village: Project: PLATEAU STATE, NIGERIA Researcher: THE TEAM Date: 28 - 04 - 2003 Abbreviations/ acronymsCDD community directed distributorCDTI community directed treatmentwith ivermectinFLHF first line health facilityHSAM health education, sensitisation, advocacy, mobilisation - i.e. activities which are aimed at getting all the key players to participate wholeheartedly in the programmeNGDO non-governmental development crganisation 54 \ 1. lndicators gf activitjes atld processee: planning and management 1.1 Check whether GDDs are planning and managing their CDTI work efficiently. This indicator assesses whether the programme is efficient and simple. The more streamlined and tim*efficient the job, the higher its sustainability. Characteristics of the indicator Sources of information CDDs should plan their work efficiently, e.g. by: . Carrying out census and distribution during the same visit (using this census data for the following year's order). . Arranging with the community leadership for help with specific problems, such as families who are not willing to participate in the programme. . Choosing visiting times and routes which will make the work less burdensome. Community treatment registers. lnterviews with: * CDDs. * Community members. * Community leaders.t FLHF staff. Findings Describe the present situation: . There was evidence of informal planning of Mectizan distribution by the village leaders and CDDs. . Census is carried out annually before distribution in most communities. . CDDs and community leaders decide on times and mode of distribution and inform community members. . The CDDs meet with their community leaders to plan registration and census. lf CDDs are not working efficiently: . \ /hy is this? Are any steps being taken to improve the situation? Analysis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved.x Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 55 ! 2. lndicators of activities and processes: leadership and ownership 2.1 Check whether community leadership is managing problems with the distribution. This indicator assesses whether the programme is effective, and whether the community is taking ownership of it. Characteristics of the indicator Sources of information The communig leadership should be taking responsibility for the distribution of Mectizan within the community. lf coverage (geographical and therapeutic) is not adequate or not being maintained, the leadership should understand the reasons for this. Together with the community at large, the leadership should identify and solve problems related to the distribution. lnspection of minutes of community/ council meeti ngs (where available). lnterviews with: * CDDs. * Community members. x Community leaders. * FLHF staff. I t I I Findings a Describe the present situation The community leaders and CDDs jointly take responsibility for distribution of Mectizan within their communities as evident in their participation in community mobilization, regular consultation with CDDs and support for CDDS . Geographic coverage is maintained at 100%. There were discrepancies in therapeutic coverage given by the project and calculated by the evaluator. There were reports of refusals as a result of inadequate health education, mobilization and sensitization at the communities. The leadership has not played any proactive roles in solving major problems such as refusals, CDD attrition and inadequate support to CDDs a a a lf the community leadership is not involved in the distribution: ' \ /hy is this? Are any steps being taken to improve the situation? a Community health education, mobilization and sensitization is being intensified and this has reduced the number of refusals. Analysis \Mren writing the report you have to summarise: * The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 56 TI lr 2.2Checkwhether the community at large has been involved in taking decisions on the distribution process. This indicator assesses whetherthe community is taking ownership of the programme, Characteristics of the indicator Sources of information The community should have taken responsibility for decisions such as: . The selection/ changing of CDDs, . The timing and mode of distribution. lnspection of minutes of community/ council meetings (where available). lnterviews with: * CDDs. * Community members. * Community leaders. * The persons supervising CDDs: FLHF staff, lay supervisors etc. Findings Describe the present situation . Community leaders and community members selected CDDs. o Times and mode of distribution are agreed by community leaders and CDDs and communities are only informed. . The community is not directly involved in taking decisions on Mectizan distribution, but through the communig leaders lf the community is not sufficiently involved in taking decisions: . \ /hy is this? Are any steps being taken to improve the situation? Analysis \Mren writing the report you have to summarise:x The evidence about how wellthis indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. t Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 57 ,\ 2.3 Check whether the community members value and accept long-term annua! treatment. This indicator assesses whether the community is taking ownership of the programme. Characteristics of the indicator Sources of information Community members should be able to mention one or more advantages of taking Mectizan. Community members should express the need for annual treatment with Mectizan. People should show understanding of the need for, and express interest in long term treatment with Mectizan. I lnterviews with: . CDDs. . Community members. . Community leaders. . The persons supervising CDDs: FLHF staff, lay supervisors etc. Findinqs Describe the present situation: . Community members testify to the benefits of the drug, as one which controls and prevents blindness and expulsion of worms. . Community members usually request for Mectizan annually . Communities are not aware of the need for long-term treatment and for how long treatment should be. lf community members do not value and accept the treatment: . \Mry is this? Are any steps being taken to improve the situation? Analvsis Wren writing the report you have to summarise: * The evidence about how well this indicator is being achieved.r, Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. T Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 58 t TI 3. lndicators of activities a[d processes: monitoring 3.1 Check whether CDDs are to the FLHF level This indicator assesses whether the programme is effective. lf such repofting is not taking place Mectizan supply willbe compromised, which rc bad for sustainability. Gharacteristics of the indicator Sources of information Reports to the FLHF level should get there on time. Reports may be summary reports, or the original community distribution record, depending on the level of skill of the CDD. Adequate transport should have been arranged for distribution records/ reports to be handed to the appropriate person. ! nspection of community distribution reports. lnterviews with: * CDDs. * The persons supervising CDDs: FLHF staff, lay supervisors etc. Findings Describe the present situation: . Monitoring by the supervisor at the first line health facility is not carried out and this had negative impact on the quality and timeliness of reports. . CDDs are reporting appropriately to first line health facilities (submitting reports and drug balances) . No arrangements are made for transportation for the CDDs by the communities . Community treatment records are available at the communities, but properly kept at a few communities lf the reporting by CDDs is poor: . Why is this? Are any steps being taken to improve the situation? Analysis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likelv to affect sustainability. Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 59 tr 4. lndicators of actlvities and processes: obtaining and managing Mectlzan 4.1 Check whether the right amount of Mectizan is received This indicator assesses whether the programme is effective. lf the ight amounts are received it will foster community ownership, which is good for sustainability. Characteristics of the indicator Sources of information All community members who were eligible for treatment got it, and some Mectizan was left over for absentees and those who were temporarily noneligible. There should be a rational explanation about how the amount ordered for the community is calculated (on the basis of population). lnspection of treatment register for the community (held by CDDs; or at higher levels) lnterview with: * CDDs. * Community members. * Community leaders. * The persons supervising CDDs: FLHF staff, lay supervisors etc. Findings \Mat happened at: . The last round of treatment? a Drug supply to communities is adequate. Absentees and temporary-non eligibles are later treated CDDs and communities do not know how to determine amount of drugs to be requested. This is often determined by repeating previous years supply of Mectizan. And sometimes arbitrarily o a The rounds before that? Drug supply had always been adequate lf the wrong amount of Mectizan was received ' Wry is this? Are any steps being taken to improve the situation? AnalYsis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likelv to affect sustainabilitv. \Mrat is the trend in Mectizan supply? Your overall judgement: is this indicator for sustainability being achieved?I Fully Highly Moderately Slightly Not at all Not applicable 60 4.2Checkwhetherthe GDDs or community members themselves fetch the yearly supply of Mectizan. This indicator assesses whetherthe project fosfers community ownership. Characteristics of the indicator Sources of information The CDDs or communi$ members fetch the Mectizan they need every year, from a designated and mutually acceptable place. Adequate transport should have been arranged for Mectizan to be collected from such a place. ln the case of remote communities, the districV LGA, in collaboration with FLHFS, should ensure that supplies reach such qroups. lnterview with: , CDDs. . Community members. . Community leaders. . The persons supervising CDDs: FLHF staff, lay supervisors etc. . DistricU LGA staff. Findinss \Mat happened at: . The last round of treatment? . The Officer in charge of the FLHF informs the community about the availabili$ of the drug at the central collection point, and CDDs go to collect the drugs for distribution. . lnformation about the availability of Mectizan is either sent directly to the CDDs or through the community leaders. . FLHF sometimes take the drugs to CDDs. The rounds before that? lf community members or CDDs have not been collecting the Mectizan: . \Nhy is this? Are any steps being taken to improve the situation? Analysis \A/hen writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. \Mrat is the trend in CDDs or community members fetching the Mectizan they need? Your overall judgement: is this indicator for sustainability being achieved?! Fully Highly Moderately Slightly Not at all Not applicable 6t TI n I 5. lndlCators of activltles and processes: HSAM 5.1 Check whether CDDs and community authorities continue to be engaged in HSAM of other community members. This indicator assesses whether the project is effective, and whether the community has taken ownership of it. Characteristics of the indicator Sources of information CDDs/ community authorities identify situations where community members require information. CDDs/ community authorities take necessary steps to provide req ui red i nformation ; encou rage community members to provide resources; promote acceptance and ownership (meetings, sanctions, community by-laws). lnterview with: . Community leaders. . Community members. . CDDs. . The persons supervising CDDs: FLHF staff, lay supervisors etc. Findinqs Describe the present situation: . HSAM for communities are carried out annually before distribution by a team of health workers and the CDDs . IEC materials are not usually adequate . The knowledge of community members and CDDs about onchocerciasis and Mectizan treatment is poor especially in communities that are being co-treated for LF and onchocerciasis. . Community leaders are involved in HSAM. . There were reports of refusals as a results of inadequate health education, mobilization and sensitization at the communities. lf CDDs and community leaders are not involved in HSAM . VUhy is this? Are any steps being taken to improve the situation? Analysis \Mren writing the report you have to summarise:x The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. ! I Your overall judgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 62 76. lndicatorg of resourg;es: financing 6.1 Check whether the community has made arrangements to fund !oca! costs of distribution. This indicator assesses whetherthe project can mobilise the resources it needs, and fosfers co m m u n ity ow n ersh i p. Gharacteristics of the indicator Sources of information I The community should support individuals who are providing CDTI services for them. The community should make provision for the supply of record books, pencils, transport and other expenses incuned during CDTI. lnterview with: . Community leaders. . Community members. , CDDs. . The persons supervising CDDs: FLHF staff, lay supervisors etc. Findings Describe the present situation: . Community support to CDDs in terms of incentives is inadequate in a few communities. . The communities and/or their leaders provide record books and writing materials. . The communities are aware of their roles to support the CDDs and initially agreed to provide support but failed to do so. lf the community is not supporting or helping to defray costs: . \ /hy is this? Community believes the CDDs are their childrenlwards who are mainly offering community service Are any steps being taken to improve the situation? Analysis \Mren writing the report you have to summarise: * The evidence about how well this indicator is being achieved.x Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overalljudgement is this indicator for sustainability being achieved?t Fully Highly Moderately Slightly Not at all Not applicable 63 I 7 -lndicators of resources: human resources 7.1 Check whether there is a satisfactory ratio of CDDs to households This indicator assesses whether the project can mobilise the resources it needs, and whether the community has taken ownership of the programme. ( I Gharacteristics of the indicator Sources of information I A ratio of at least one CDD to 20 households (or 2 CDDs per 250 population) is recommended. The households for which CDDs are responsible should be close to their own homes. !nterview with: . Community leaders. . Community members. . CDDs. . The persons supervising CDDs: FLHF staff, lay supervisors etc. Findings . The present ratio in the community: The ratio of CDDs to population is less than the recommended ratio of one CDD to 20 households or 2 CDDs to 250 population. Registration and treatment is usually completed between two and four weeks The average distances that CDDs have to walk to get to homes: Most CDDs do not travel far from their homes to distribute Mectizan lf the ratio of CDDs is too low: . \Nhy is this? t Are any steps being taken to improve the situation? Analysis \Mren writing the report you have to summarise:x The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Your overalljudgement: is this indicator for sustainability being achieved?I Fully Highty Moderately Slightly Not at all Not applicable 64 e7.2 Check whether a!! CDDs have received appropriate training This indicator assesses whetherthe project is effective, Gharacteristics of the indicator Sources of information CDDs should be skilled at their work: doing the census; giving the right dose; knowing who is not eligible; knowing what to do with sideeffects There should be a plan in place for training CDDs to replace those who drop out, or when new ones are elected for other reasons. lnterview with: * Community leaders. * Community members.x CDDs. * The persons supervising CDDs: FLHF staff, lay supervisors etc. Observinq CDDs at work. Findinqs Describe the present situation: . Prior to last year (2002), training was mainly routine , but now training is targeted at new CDDs because of high attrition rate. . The officer in charge of FLHF conducts training for CDDs at the health facility. o Posters and CDD information booklets are provided to CDDs after training but no poster was seen in the communities. lf CDDs appear unskilled, or if there is no proper plan for training replacements: . \Mry is this? Are any steps being taken to improve the situation? Analysis t Wren writing the report you have to summarise: * The evidence about how well this indicator is being achieved.x Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. I Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 65 Characteristics of the indicator Sources of information CDDs should express willingness to continue with distribution in the long term, given the conditions which prevail in the community. Few CDDs in this community have dropped out from the distribution work. lnterview with: . Community leaders. . Community members. ' CDDs. . The persons supervising CDDs: FLHF staff, lay supervisors etc. 7.3 Check whether CDDs are wi!!i to continue their work in CDTI This indicator assesses whether the programme has mobilised the resources if needg and whether it fosters community ownership. l-r Findinqs Describe the present situation: a Most CDDs are willing to continue to serve as CDDs in the long term. There are however indications that lack of adequate incentive could lead to CDD attrition. Communities replace and train CDDs that left the programme.a lf some CDDs are doubtful or unwilling to continue, or have dropped out: . \Nhy is this? Are any steps being taken to improve the situation? Analysis ! \Mren writing the report you have to summarise:x The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likelv to affect sustainabilitv. ! Your overalljudgement: is this indicator for sustainability being achieved? Fully Highly Moderately Slightly Not at all Not applicable 66 /1 8. lndicators of j4pact: coverage 8.1 Check whether the geographical coverage in the community is satisfactory This indicator assesses whether the project is effective - if the rate is poor the project is clearly struggling, and /ess sustainable. Characteristics of the indicator Sources of information T All households and areas in the community are being treated. This includes the hamlets for which the community has agreed to be responsible. lf this geographical coverage is not 10Oo/o, it should be improving. lnspection of:x CDDs'treatmentregisters. * Yearly distribution reports for that community lnterview with: * Community leaders. * Community members. * CDDs. * FLHF level staff. I Findings The geographic coverage situation: . At the last distribution: Geographic coverage was 100% as all endemic communities, wards and harmlets are being treated. The year before . The year before that: Same as above lf geographical coverage is poor: . \Nhy is this? t Are any steps being taken to improve the situation? Analysis \Mten writing the report you have to summarise the reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. ffiat is the trend in geographical coverage? Your overalljudgement: is this indicator of sustainability beinq achieved?I Fully (100o/o coverage of all households) Highly (only nomads in the surrounding area were missed) Moderately (the outlying hamlets were also missed) Slightly (some wards of the community were also missed) Negligibly (no- one got treated, or only a few families) 67 \8.2 Check whether the comm has a rate. This indicator assesses whether the programme is effective - if the rate is poor the project is clealy struggling, and /ess sustainable. r Gharacteristics of the indicator Sources of information The community overall has a therapeutic coverage rate of 65% or higher. This rate should be stable or increasing. !nspection of: x CDDs'treatmentregisters. * Yearly distribution reports for that community lnterview with: * Community leaders. * Community members. * CDDs. * FLHF levelstaff. I Findings The therapeutic coverage situation in the community . At the last distribution: Therapeutic coverage is very high as most of the communities have attained an increasing rate of at least 65%. . The year before: Same as above The year before that: lf the therapeutic coverage rate is poor: . VUhy is this? \Mich steps are being taken to improve the situation? Analysis \Mren writing the report you have to summarise the reasons for poor performance (if any); steps beinq taken to improve it; and how this is likely to affect sustainability. \Mrat is the trend in therapeutic coverage?! I Your overall iudqement is this indicator of sustainability being achieved? Fully (the community has a therapeutic coverage rate >65% - stable or increasing) Moderately (the therapeutic coverage rate is >65%, but it is unstable or decreasinq) Negligibly (the therapeutic coverage rate is <65%) 68 ! (