Assessment of the sustainability of the Kaduna CDTI project, January 2002

World Health Organisation African Programme for Onchocerciasis Control Assessment of the sustainability of the Kaduna CDTI project January 2002 Mariam AIIy Oladele Kale Chukwu Okoronkwo Detlef Prozesky 2 3Index Page Abbreviations/ acronyms and acknowledgements Executive summary lntroduction and methodology Findings and recommendations 1. State level 2. LGA level 3. DistricV health centre level 4. Village level 5. Overall sustainability grading for the project Detailed findings 1. State level 2. Health district/ LGA level 3. Sub-districU first line health facility level 4. Community level 4 5 7 9 t2 15 t7 t9 t9 25 3l 37 I 4Abb reviations/ acronyms African Programme for Onchocerciasis Control community directed distributor (of ivermectin) community directed treatment with ivermectin community health extension worker local government authority LGA onchocerciasis control team non-goveflrmental development organisation national onchocerciasis task force Primary Health Care State onchocerciasis co-ordinator State onchocerciasis control team World Health Organisation zonal onchocerciasis co-ordinator APOC CDD CDTI CHEW LGA LOCT NGDO NOTF PHC SOC SOCT wHo ZOC Acknowledgements We would like to thank the following persons for their help: . The staff at APOC Headquarters in Ouagadougou: Dr S6k6t6li, Dr Amazigo, Mr Aholou . Staff of the Ministry of Health in Kaduna: Dr Fayomi and his zonal team; Mr Sankwai and his State team. . Health workers and community members in the Birnin Gwari and Kauru LGAs. 5Executive summary The Kaduna CDTI project has been supported by APOC for the past 4 years, and is in its last year of agreed funding from APOC. An evaluation of the sustainability of the project was carried out in January zooz,by a team of four evaluators from three African countries. The evaluators were charged with two tasks: . Evaluating the sustainability of the project. . Pre-testing an instrument to evaluate sustainability of CDTI projects, which had been developed by a team ofexperts. The evaluation was carried out over a period of a week. Information was collected by document study, interview and observation, at sampled sites at four levels of the health service: State, LGA, health centre and village. The overall judgement of the team is that the Kaduna CDTI project is not far from being sustainable. Regarding the five elements of sustainability, the situation is broadly as follows: . Effectiveness: The project is effective at all levels. . Efr.ciency/financing: Many activities are not properly targeted, resulting in inefficient use of scarce resources. A limited amount of funding will be available after APOC withdraws. . Simplicifl: Routines are generally simple and easy to carry out. . Integration:The project is bynow well integrated into the official health system. . Attitude: Although stakeholders have accepted the project as part of their routine work, some key players have not yet accepted the fact that they will have to cope without outside resources in the near future. Regarding the position of the different levels of the project, the CDTI activities of the most peripheral level (the village) appear to be sustainable, even now. The three higher levels however will have to sort out the key issues of planning, funding and transport if they are to be sustained. The LGA level particularly needs attention. Detailed recornmendations were drawn up, based on the findings of the evaluation at the four levels. The recommendations were prioritised, and indicators and deadlines were suggested for each. The most important recommendations concem determining the exact funding that will be available, and mobilising additional sources of funding if necessary; tailoring activities (mainly training and supervision) to fit the budget; and therefore rationalising activities to achieve maximal efficiency. National, zonal and State level staff have a year in which to implement the necessary changes. Regarding the pre-testing of the instrument to evaluate sustainability, the team found that it was in the main workable. Some changes were found necessary, but these were not substantial. In addition the team developed a guideline document for future evaluators: Using the instrument to determine the sustainability of APOC projects'. Finally it became clear that the time needed for the exercise should be at least 10 days, to accommodate the field work, writing the report and planning for sustainability with the local leadership. B m 6 7Introduction and methodology I Introduction The Kaduna CDTI project was one of the first group of projects to be approved by APOC, and to begin receiving funding. Its 5 year period of funding is due to end in March 2003. In 2001 a decision was taken by APOC management that all projects should be assessed, to find out how sustainable they are. The Kaduna project is the first one to be assessed for sustainability. Accordingly the aim of this first assessment was twofold: . To test the sustainability instrument that had previously been developed by an expert team, meeting in Ouagadougou. . To assess the level of sustainability of the Kaduna project. 2. Methodology This is glven in detail in the document: 'Using the instrument to determine the sustainability of APOC projects'. Research question: How sustainable is the Kaduna CDTI project? Design: Cross-sectional, descriptive. Population: The Kaduna project, including: its SOCT; its NGDO partner; its LGAs with their LOCTs; the project villages and their CDDs. Sampling: * Two LGAs (one with high coverage, one with low coverage) - Birnin Gwari and Kauru * ht each LGA two sub-districts/ health centres (one with high, one with low coverage) - Rendegi and Kamfani Doka; Garmadi and Kusheka * In each sub-district two villages (one with high, one with low coverage): Ung. Alhaji Hassan, Ung. Wakili Rendegi, Ung. Ciwa Kamfani Doka and Ung. Yola Nariyo; Tudun Wada Garmadi, Raguta, Kusheka and Tishen. Instrument: * Questionnaire (see appendix: 'Detailed findings') structured as a series of indicators of sustainability. The indicators are grouped into 9 categories. * The instrument assesses sustainability at 4levels of operation. * The instrument guides the researcher to collect relevant information about each indicator. Source of information;Yerbal reports from persons interviewed, supplemented by documentary evidence. Analysis: * Based on the information collected, each indicator is graded on a scale of 0-4, in terms of its contribution to sustainability. * The average'sustainability score' for each goup of indicators is calculated, for each level.{' A qualitative description of problem areas is given. I l 8Limitations of the study were found to be the following: x The monitoring data did not contain aggregations for the sub-district level, which made sampling for this level difficult. x The documentation required was not ready - the respondents were not informed of this need in advance. * Two LGAs were sampled but this proved to be too little - in future three will be needed. On the other hand it was found that only two sub-districts per LGA are needed (not three as planned) and only two villages per sub-district. * Having project staff with the evaluation team was initially problematical, since they lacked the necessary background. By the end of the week this was no longer a problem. 3. Adapting the instrument Overall the instrument (as designed in Ouagadougou and subsequently operationalised in Pretoria) worked well. The interviews and document study were easy to carry out and did not take long. The instruments were highly effective in identifying problem areas. A fair amount of additional work had to be done, to make the instruments more user friendly: . Changes were required in cases were wording was found to be unclear/ ambiguous. This was the major group of changes needed, and proved to be very time-consuming. . The 9 categories of indicator for the different levels had to be harmonised with each other. For example, if CHEWs train CDDs, is that dealt with at sub-district or community level? There were several other cases of duplication as well. r { system had to be developed, to help the researcher make decisions about the level of sustainability (the 5 point scale described above). A 3 point scale was first tried out but didn't work well - the 5 point scale was much better and had a high level of reliability in the group. As well as improving the instrument, the goup also worked out a method of applying it. These guidelines are given in the document mentioned above: 'Using the instrument to determine the sustainability of APOC projects'. 9Findings and recommendations 1. State level 1.1 Overall grading (on a scale of 04) Kaduna project self-sustainability at State level 4 3.5 A3ttI ,.u .9g2 o B ,.uo .8 1 0.5 0 .""* -""" $"""'.s :}"1.-*::{d group of indicators 1.2 Main findings The situation regarding sustainability is reasonably good overall. The major problem areas are: Training and mobilisation/ sensitisation are carried out routinely every year, without considering whether it is needed. As a result it is overly expensive. Planning takes the form of repeating what was done before, and there is no specific written plan which identifies problem areas and deals with them. In this way time and money are wasted. There is no plan for sustainability. Transport is sufficient for the moment, but no plan exists for future replacement - when the vehicles currently in use are beyond repair. The use of transport is poorly controlled. There is some (more or less) guaranteed funding from the State for the future, but it is much less than the current budget. No plan has yet been made to tailor future activities to the limited future budget. Supervision and monitoring are done routinely, in stead of being targeted, and are therefore not as effrcient as they should be. I TI l0 1.3 Recommendations Recommendation Implementation Plannine: r I plan and budget must be drawn up, for the 2003 round of distribution. Each activity has to be fully costed, and the total budget has to fall within the amount that will be available (see below). . This kind of plan and budget must in future be drawn up yearly. Priority: HIGH Indicators ofsuccess; r I complete plan and budget for 2002, as described on the left. Who to take action: . SOC - Mr Sankwai . ZOC - Dr Fayomi . NGDO partner Deadline for completion: , December 2002 Financins/ fundine: . The funds that will be available for the 2003 distribution must be accurately determined (from the State; from remaining APOC funding; possibly from the NGDO partner). . If there is not going to be enough money for the minimum number of activities that need to be carried out during 2003, firm promises of additional funds from other sources must be obtained as soon as possible, in writing. ' The same must be done for each of the remaining years ofthe project. Priority: XIIGI-l Indtcators ofsuccess: l. Written breakdown of assured tunds for 2003. 2. Official documents stating amount and duration of future financial support from outside. Who to take action: ' SOC - Mr Sankwai . ZOC - Dr Fayomi . NGDO partner Deadline for completion:1. March2002 2. December 2002 Transport: r I system of strict control of transport needs to be instituted: authorisation of every trip and every expenditure (fuel, tyres and repairs); a log book; and monthly reconciliation of trip authorisations and logbook entries. r I specific, realistic, dependable plan must be made for transport, for when the present project vehicle comes to the end of its life. This could be finding funds for a new vehicle; using public transport; using Ministry pool vehicles etc. Priority; MEDIUM Indicators ofsuccess; 1. Up-to-date vehicle logbooks. 2. Trip authority slips for every trip 3. A realistic written plan for assuring transport after the present vehicle is written off. Who to take action: ' SOC - Mr Sankwai . ZOC - Dr Fayomi Deadltne for completion: 1,2 March2002 3. December 2002 Supervision and monitorine: . The 200112002 statistics must be used to identifu LGAs with low coverage. . When planning for the 2003 distribution these LGAs should receive particular supervision, and not those which are going well. . Supervision trips should be planned in such a way Priority: MEDIUM Indicators of success; r I supervision timetable for 2003, which takes account of all the requirements on the left. . This timetable should be part of the overall year plan. ll Note that some of these recornmendations could already start being implemented during the 2002 round of distribution. This will save money, leaving a greater cushion to top up the State contribution in the coming years. that they are as economical as possible - short of duration, sharing transport, integrated with supervision of other pro gralnmes. Who to take action: . SOC - Mr Sankwai . ZOC - Dr Fayomi Deadline for completion; . December 2002 Trainine and mobilisation: . Training must be targeted. Only CHEWs who clearly lack skills, or who are new to the job, should be trained. CHEWs who are known to be performing well do not need further training. ' Training should move to the LGA level. One SOCT member travels to a given LGA to supervise the training there - this will be cheaper. . The CDTI coordinator at LGA level should take charge of the training, with the SOCT member assisting only. Priority: MEDIUM Indicators of success: r I training timetable for 2003, which takes account of all the requirements on the left. This timetable should be part of the overall year plan. Who to take action: . SOC - Mr Sankwai . ZOC - Dr Fayomi Deadline for completion: December 2002 12 2. LGA level 2.1 Overall grading (on a scale of 0-4) Kaduna Projecfi self-sustainability at LGA level 4 3.5 A3tI ,.u .9 o ,z o CD rrlE r.v ol1 0.5 0 "".", o"""" ""%""".,'. :$"$-{"..."":"/ a"t' group of indicators 2.2 Main findings The overall situation regarding sustainability is less satisfactory. Major problem areas are: Training is not carried out at this level at all - it is bypassed, and health centre CHEWs (LOCT members) go to Kaduna to be trained by State level staff. As a result it is overly expensive. It is not integrated with any in-service training. In some LGAs no budgetary allocation whatsoever is made, for CDTI activities. In others funds are allocated when the need arises, but there is no specific allocation for CDTI. It may be refused or withdrawn at any time. Transport is problematical. [n some cases there are no funds for repairs or fuel. There are no real plans for replacement of existing vehicles. Transport is poorly controlled and not used in an integrated way. The leadership (bureaucratic and political) has little understanding of CDTI, and in some cases takes little interest in it. Planning is not really done systematically at this level, and as a result teams depend on the State to start them on the yearly round of distribution. Data obtained through supervision and monitoring are not systematically used as a management tool. t3 2.3 Recommendations Leadership: . The leadership at LGA level needs to be informed more fully about CDTI, and motivated to take a more active interest in it. ('Leadership'means political - the health councillor, and technical - the PHC co-ordinator). . Such sensitisation should be targeted to those LGAs where leadership is known not to take an active interest in CDTI. Priortty: HIGH Indicators ofsuccess; . LGA leadership ensures that CDTI is properly resourced and supported. Wo to take actton: . SOC Mr Sankwai and the SOCT members LGA councillor responsible for health care Deadline for completion: . December 2002 Plannine: . LGA PHC co-ordinators and onchocerciasis co-ordinators should draw up a detailed plan and budget for CDTI in the LGA, for 2003. . The plan contains a timetable of all activities, as well as details of all resources needed (human, material, financial). Priority: HIGH Indtcators ofsuccess: ' The plan exists, as detailed on the left. Who to take action: . SOC Mr Sanlavai and the SOCT members . LGA PHC co-ordinator Deadline for completion: . December 2002 Financine/ fundins: . Those LGAs which are not yet allocating funds yearly for CDTI, must be strongly motivated to do so - even a small amount. Priority; HIGt-l Indicators ofsuccess: . All LGAs budget some funds yearly for CDTI (mainly for transport) Who to take action: . SOC Mr Sankwai and the SOCT members LGA councillor responsible for health care Deadline for completion; . December 2002 Trainine: . The LGA level onchocerciasis co-ordinators should be enabled to train the CHEWs at the next level. SOCT members should only support and supervise. . Such training should be targeted - i.e. particularly focused on those CHEWs who really need it. Priority: I\4EDIUIVI Indicators ofsuccess; 1. SOCT conducts haining for LGA onchocerciasis co- ordinators, to enable them to become trainers. 2. There is a training plan for the year, which fulfils the requirements set out on the left. 3. LGA onchocerciasis co-ordinators conduct training for those CHEWs in their LGAs who really need il Who to take action: . SOC - Mr Sankwai Deadline for completion: 1. December 2002 2. Immediately before the 2003 distribution MEDIUMPriSupervision and monitoriug: t4 Note that some of these recommendations could already start being implemented during the 2002 round of distribution. This will make staffused to the new way of doing things, before the cut in funding begins to bite. I Statistics must be aggregated in such a way that it is possible to get coverage figures for the catchment area of each health centre (i.e. each sub-district). The200ll2002 statistics must be used to identiff sub-districts with low coverage. When planning for the 2003 distribution these sub-districts should receive particular supervision, and not those which are going well. Indicators ofsuccess: r A, supervision timetable for2003, which takes account of all the requirements on the left. . This timetable should be part of the overall year plan. Who to take action: . SOC Mr Sankwai and the SOCT members . LGA onchocerciasis co-ordinator Deadline for completion: . December2002 Transport: r I system of strict control of the prograrnme motorcycle needs to be instituted: authorisation of every trip and every expenditure (fueI, tyres and repairs); a log book; and monthly reconciliation of trip authorisations and logbook entries. r I specific, realistic, dependable plan must be made for transport, for when the present prograrnme motorcycle comes to the end of its life. This could be finding funds for a new vehicle; using public transport; using LGA pool vehicles etc. Priority: iVIEDIUN4 Indicators ofsuccess; 1. Up-to-date vehicle logbooks. 2. Trip authority slips for every trip. 3. A realistic written plan to assure transport after the present motorcycle is written off. lVho to take action; . SOC Mr Sankwai and the SOCT members . LGA councillor responsible for health care Deadline for completion: 1,2 March2002 3. December 2002 I 15 3. Health centre/rdistrictr level 3.1 Overall grading (on a scale of 0-4) 3,2 Main lindings The overall sustainability situation is fair. The major problem areas are: . Supervision and monitoring are done routinely, without prioritising areas of specific need. As a result this activity is more expensive and time-consuming than it need be. . Re-training of all CDDs is done routinely, and is therefore more expensive than it need be. CHEWs at the health cenfie level are in some cases not fully involved in the training, which is done by staffcoming from the LGA level. r fu some cases staffat this level receive no financial support at all from the LGA level, for CDTI activities - CtIEWs even have to pay for transport and stationery themselves. . There is no written year plan for CDTI. The process of implementing CDTI every year depends on its being initiated by the State level. Kaduna Projecfi selfsustainability at district level "'"- .-"' ".*""c $"$-{"{d ""t' group of lndlcators 4 3.5 3 2.5 2 .5 1 0.5 0 tt 2 o o ' o ID a! o a! 1 l6 3.3 Recommendations Note that some of these recommendations could already start being implemented during the 2002 round of distribution. This will make staffused to the new way of doing things, before the cut in funding begins to bite. Plannine: ' CHEWS in each health centre should draw up a detailed plan and budget for CDTI in their catchment area, for 2003. . The plan contains a timetable of all activities, as well as details of all resources needed (human, material, financial). Priority: l-llGH Indicators ofsuccess; . The plan exists, as detailed on the left. Who to take action; . CHEWs in charge of CDTI at each health centre . LGA PHC co-ordinator and onchocerciasis co- ordinator (to help CHEWs) Deadline for completion; . December 2002 Supervision and monitorine: . CHEWs in charge of CDTI at health centres must analyse village coverage figures for200ll 2002, and identify those that are struggling. . When planning for the 2003 distribution these villages should receive particular supervision, and not those which are going well. Priority: MEDIUN4 Indicators ofsuccess; r { supervision timetable for 2003, which takes account of the requirements on the left. . This timetable should be part of the overall year plan. Who to take action: . CHEWs in charge of CDTI at each health centre . LGA onchocerciasis co-ordinator (to help CHEWs) Deadline for completion: . December 2002 Trainine: ' Training of CDDs should be conducted by CHEWs working in the health centres, and not by higher levels (e.g. SOCT members). The LGA onchocerciasis co-ordinator may be present to support and supervise. . Yearly training of CDDs should be targeted - i.e. particularly focused on those who really need it. Priority: NIEDIUI\l Indicators of success: 1. There is a training plan for the year, which fulfils the requirements set out on the left. 2. Sub-district CHEWs only conduct training for those CDDs who really need it. Who to take actton: . CHEWs in charge of CDTI at each health cenffe . LGA onchocerciasis co-ordinator (to help CHEWs) Deadline for completion: 1. December 2002 2. Immediately before the 2003 distribution t7 4. Yillage level 4.1 Overall grading (on a scale of 04) Kaduna Project self-sustainability at village level tt s .9 o , o ED a! o o 4 3.5 3 2.5 2 't.5 1 0.5 0 ./ -"""""C ^"""'-* .."-t ::."!$4d group of indicators 4.2 Main findings The sustainability situation at this level is high. There appears to be only one problem area: r fu planning the year's work a number of communities planned two visits per distribution - one for the census and one for the actual distribution. This doubles the CDDs'work. 4.3 Recommendations Plannine: . CDDs undertake only one visit to each family during each distribution - i.e. the census and ivermectin distribution are done at the same time. . This way of doing CDTI is suggested to the CDDs and villagers, for their decision - since in the spirit of CotI they have to make the decision. Priority: HIGH Indicators ofsuccess: . Villagers and CDDs decide on one visit per home per distribution. Wo to take action: . SOC Mr Sankwai; LGA PHC co-ordinators; LGA onchocerciasis coordinators (to set in motion) . CHEWs (to suggest to villagers and CDDs, and discuss with them how to do it) Deadline for completion: . December 2002 l8 5. Overall sustainability grading for the Kaduna project The project scores as follows for its different levels: Scores for each tevel (/50) State LGA Health centre Village 31.6 20.s 3t.7 40.1 Overall total for the proiect (1200) 123.8 The project is doing fairly well - there is a lot of effort being expended at all levels. The CDTI activities of the most peripheral level (the village) appear to be sustainable, even now. The three higher levels however will have to sort out the key issues of planning, funding and transport if they are to be sustained. The LGA level particularly needs attention. National, zonaland State level staff members have a year in which to implement the necessary changes. 19 Detailed findings 1 State level 1. Coveyage Not relevant at this level. 2. Pla4ni4g 2.r Check if the year planfor CDTI appears as part of an overall written planfor the health service at this level (it may appeil under another narne, e.g. 'onchocerciasis indicator of and effectiveness 2.2 Check if the plan contatns all the elements neededfor CDTI to workwell (indicator of effectiveness 2.3 Check if all partners (government, UN agencies, NGDOs) are meaningfully involved in the overall indicator of and effecti 2.4 Check if specific planningfor sustainability has taken place, for the period after APOC is withdrawn of effectiveness Findinss: Planning for CDTI is completely separate from that for other programmes. The only plan used is the one prepared for APOC. The administration of CDTI is equally separate. CDTI is integrated into the overall year plan at this level (showing that the management considers CDTI to be part of ix yearly routine, like any other programme). It is not separate. CDTI is adminktered like other, similar disease control programmes. There is written evidence of such integrated planning and management. Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Findines: The plan (in APOC format) is comprehensive The plan should make provisionfor all kqt elements. Checklist of kqt activities: Mectizan supply; targeted training ; targeted mo bilisationl s ensitis ation ; monitoring anel supervis ion Fully Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Findines: Each partner makes a plan and shows the others for comments. Finally it is anralganrated. . All partners contribute to routine planning. . The plan gives all pamters specific tasl<s to perform.t Partners are clear about their own roles, and those ofthe other partners. Fully Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Findines: There is a vague idea with some specific points - but it is not specific and safe. Staffreport that they have made plans for this period which will enhance programme sustainability. There is written evidence that such planning has taken place. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? 20 lS. --i,-iovitin g leadership 3.1 Check if there is an oficial, within the normal civil service organogram, charged with onchocerciasis control of and effectiveness 3.2 Check if there is evidence of political commitment to CDTI at this level (indicator of and and supervision 4.1 Check if routine data concerning CDTI activities at this level are collected and transmitted within the indicator of 4.2 Check if only necessary information is betng collected (i.e. necessary for making decisions of and 4.3 Check if the relevant person at the national/ State level is routinely supervising the CDTI the LGA/ health district on site of effectiveness Findinss: There are l0 persons in the SOCT. This is far too many - they are completely under-utilised and spend most of their time doing other things. They have more than enough time for this activity. There is at least one fficial who has clearly been allocated the task of running the CDTI programme. There is a document available which conJirms thk appointment. If this person/s has other responsibilities as well, s/he has sfficient timefor her/ his onchocerciasis control activities. r Not applicableFully Highly Moderately Slightly Not at allIs this good for sustainability? The senior person (politician/ civil servant) in charge of health matters at this level knows about CDTI and appears committed to it. There is evidence ofspecific past budgetary allocations and disbursements for CDTI, and the amounts are increasing. I I Findines: They know about CDTI and appear committed to it. They know that APOC money is corning to an end. They have allocated quite a substantial sum, but say that this will not be increasing nexl year. Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Moderately Findinss: It all happens entirely within the government system. The reporting process takes place within the government system, not using other resources. Not applicableFully Highly Moderately Slightly Not at allIs this good for sustainability? Findines: The information is all necessary All the information included in the reports is potentially useful at this level, as a lool in planning and decision making Not applicableFully Highly Moderately Slightly Not at allIs this good for sustainability? Findinss: There are about three visits per LGA per year There is evidence that each LGA/ district is visited at least once around the time of distribution. There are not unnecessarily many visits - there is clear iustificationfor each one. Fully Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? fI 2t 4.4 Check if supervisory visits are being planned and executed as efi.ciently as possible indicator of and 4.5 Check whether there is a routine process of management of problems and successes, which are indicated by the monitoring system (coverage dat4 visits and reports) of effectiveness and attitude 5. Mectizan procurement and distribution 5.1 Check if Mecttzan is being stored and admintstered within the government system at this level of 5.2 Check if the alternattvesfully dependable, in cases where parts of the system are not mainstreamed of effectiveness and . NOC/ SOC routinely only supervises the next level, and empowers the next level to supervisefurther down . Supervision visits are integrated where possible. . Resources (human, transport etc.) are fficiently used. . Each visit has a specific goal and / or checklist. Findines: SOCT members visit all levels (down to the village) when they go out. All visits are routine - not targeted. There is a supervision checklist. SOCT nrembers go out alone. Visits are for CDTI only. Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Moderately Findines: The informatiorr or-r coverage is not used, it's just filed. Supervisors meet together at times to plan according to what they have found during visits. When the SOC identifies problems he deals with them. There is no feedback on successes. As soon as problems are identified in this way, the appropriate manager deals with them. Planning includes activities which will improve coverage in areas where it is unsatisfactory. Successes are noted and reported, and appropriate feedback given. The CDTI co-ordinator is empowering the next level to cope with problems. Checklkt of kqt activities: sensitisation/ mo bilis ation ; training ; improving M ectizan supply ; effectiv d s upportiv e sup ervis io n I Fully Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? The Mectizan: . All costs of this activity are met in the government budget. . The Mectizun is stored in a room made available by the government at this level - preferably in the same room as the other dntgs. . The Mectizan is controlled within the government system, preferably using the same stock control system as for other drugs. . No stage of the ordering / distribution requires input from APOC / NGDO. Findines: For distributing: government system, fully Supply: SSI State takes to LGAs - luel paid by State - special visit Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Findinss: The SSI leg: they promise they will continue this activity - this is believable therefore reliable There is a specific systemfor Mectizan distribution in such cases. This system uses sustainable resources.I Fully Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Findines: The Federal government has agreed There is a written agreement allowingfree importation. The importation of Mectizan is not delayed by bureaucracy. Not at all Not applicableFully Highly Moderately SlightlyIs this good for sustainability? 22 5.3 Check if the clearance of Mectizan is assured, when it is imported (indicator of effectiveness and 16. _ Tlaining and sensitisation/ mobilisation NOTE: This refers to training and sensitisation activities performed by officers working at this level, for the next level. In Nigeria officers at the level of the State train the whole LOCT (the LGA oncho coordinator plus his/ her team of CHEWs); in other countries these higher level officers only train officers at the health district level. 6.1 Check if ,s done of and 6.2 Check that training conducted at this level (i.e. training of district level staff) is betng done the indicator of efficienc and 7. Financial resources 7.1 Check whether the cost implications of each CDTI related activity (monitoring/ supervision, training/ mobilisation, Mectizan distribution) are quantified in the yearly the State/ indicator o f effectiveness Findines: Training is done routinely. There is no targeling. The duration is too long. There is an objective needfor each episode of training: * There is evidence that staffto be trained lack lenowledge and skills to perform the job. * Training to motivate staffis not a sfficient reasonfor training. The evaluator is satisfied that the duration of training was justified. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? Findines: State level trains LOCT. Note tl-rat zonal level staff members help with this training. Staf of this level only train the level immediately below it, and notfurther down. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? Findines: All are budgeted for The costs for each CDTI relatecl activity in the year plan is clearly spelt out in the budget. . Recurrent and capital costs (ifany) are separated in the budget. . ChecHist of key activities: Mectizan supply; targeted training targeted mobilisation/ sensitisation ; monitoring and supervision. Not at all Not applicableFully Highly Moderately SlightlyIs this good for sustainability? u 23 7.2 Check whether appropriate and adequate amounts are budgetedfor the planned CDTI related activities of effectiveness and 7.3 Check whetherfunds to cover these costs are increasingly being suppliedfrom State/ national resources of 7.4 Check if in case of a deficit between estimated costs and the amount provided by the government, dependable provision is being made to meetir (indicator of effectiveness and 8. 8.1 related activities of effectiveness and efficienc 8.2 Check if the transport made availablefor CDTI is being adequately managed of efficienc Transport and other material resources Check if there is sfficient transportfor necessary visits to the district levelfor CDTI Findines: Adequate but not appropriate - APOC is supplying what is needed, but the expenses are admitted to be high for the outcome. The amounts budgeted are neither excessive nor too small - tnanagers are able to justify the needfor each amount, in relation to: . The outcomes resultingfrom the previous year's CDTI expenditure . The cost of similar activities for other programmes. Fully Highly Moderately Slightly Not at all Not applicableIs this good for sustainabilitfl Findines: The State contributior-r is spelt out in detail. It is not the nrajor contribution yet, but it is increasing. The relative contributions of the district/ LGA and other partners are clearly spelt out. The proportion provided by the oficial health service is the major one by now. This proportion of funds is increasing yearly. Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Management is aware of the size of the shortfall, and has specific and realistic plans to bridge il. If it is planned that non-government sources of funding are to be used after APOCfunding ends, written commitmentfor thk has been obtained at the highest level in these donor organisations (e.g. a Memorandum of Undersnnding). Findines: 2003: planning to see NGDOs/ clubs - e.g. for T-shirts Planning to cnt activities. They have no other sources of funding in mind at preselrt. They have not spoken to SSI about their future contribulion. BLrildin-e up a cusl.tion in the'Counterpart' fund. Moderately SfighAy Not at all Not applicableIs this good for sustainability? Fully Highly There are no reports offailure to undertake necessary CDTI activities at district level, due to transport not being available or affordable. Checklist of activities : monitoring/ supervision, trqining/ mobilisation, Mectizan clistribution Findines: At present there is arnple transporl available - two vehicles and enough money for fuel and maintenance. Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Moderately Trips madefor CDTI purposes are properly authorised in writing by the relevant fficial. Each trip undertakenfor CDTI purposes is recorded in a log book. Transport provided by the central level is no longer being usedfor routine CDTI activities at LGA/ district level. I I a Findines: There is no written control system at all. The systenr mrls on trust and intuition. This transporl is being used a sub-district level, lor supervisiorr and training. Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Moderately I ft 24 8.3 Check if there are sfficient other material resources avatlablefor all key CDTI activities at this level indicator of 8.4 Check if there is a dependable plan to replace transport and equipment supplied by APOC, when this becomes of effectiveness and atti '9. Human resources 9.1 Check whether staff at this level is skilled and knowledgeable, regarding the CDTI in its area indicator of effectiveness 9.2 Check whetherpersonnel at the health district/ LGA level is stable, and whether provision ts madefor passing on CDTI skills when a trained person moves away of effectiveness Findines: They have everything they need - contputer, printer, furniture, photocopier There are no reports offailure to undertake necessary CDn activities, due to material resources not being available at this level Checklist of activities : monitoring/ supervision, training/ mobilis ation, Mectizan distribution I Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? Findinss: Olfice equipment is there - extemally funded; no plan for replacement. 2 cars - cornpletely externally funded, no plan to replace. The SOC is not making enough effort - he does get something from he State when he asks; needs more proactive planning for it - depending on APOC funding. Management knows that replacements will be needed before the end ofthe programme, and has specific, realistic plans to meet the need thm. It is planned that Government will provide the replacements, and maintain existing vehicles and equipment. If it is planned that replacement will befrom non- government sources, written commitment for this must have been obtained at the highest level in these donor organisations (e.g. a Memorandum of Understanding). Checklist of equipment: vehicles, computer/ printer, photocopier. t Slighfly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? Findines: Stalf (especially the SOC) know exactly whal to do. Team members have enough lotowledge and skill to undertake all the key CDTI qctivities themselves, without help: . Planning . Training and sensitisation/ mobilisation . Mectizan ordering/ distribution . Monitoring/superviston. Not applicableHighly Moderately Slightly Not at allIs this good for sustainability? Fully Findines: To date staff has been very stable. There are 10 colleagues in the SOCT; there is SSI to give advice and provide continuity; there is the zonal team next door. Staffshould remain in one postfor at leastfive years. There should be immediate training (in CDTI) of new, unskilled district/ LGA staffmembers who have CDTI responsibilities. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? I I 25 f 1. 2. Health districU LGA level I Coverag Check whether the geographical coverage in the health district/ LGA is satisfactory of effectiveness 1.2 Check if all sub-districts and vtllages have a satisfoctory therapeutic coverage rate of effectiveness , 2.t Plan4ing Check if the year planfor CDTI appears as part of an overall written planfor the health dtstrict's acttvities (it may appear under another name, e.g. 'onchocerciasis indicator of and effectiveness 2.2 Find out if theyearly plans were drawn up in a participatory way (the overall districU LGA and the CDTI of effectiveness Findines: l: LGA: didn't know which are covered and which not - some problem with Fulani Even so it appears good. 2: LGA: had an idea of what the endemic areas were - not specific; thinks fully covered All sub-districts and villages identified by the latest REMO are under treatrnent (i.e. the geographical coverage rate is I 00?5 and stable). This should be conJirmed by comparing coverage records with the REMO list of endemic villages Slightly (many villages/ hamlets not covered) Is this good for sustainability? Fully (100% coverage) Highly (problem withnomads only) Moderately (a few villages/ hamlets not covered) The therapeutic coverage rate should be: . 6596 or higher . stable or increasing.. Findings: Both: very good everywhere. Higbly (80-89% coverage) Moderately (65-79% coverage) Slightly (less than 65% coverage) Is this good for sustainability? Fully (>90% coverage) Findines: 1: Tl-rere is a plan for l0 priority problems. but oncho is not one of these. 2: There is a 3 year rolling plan for capital expenditure - nothing specific for oncho. Both cases: they don't doubt that CDTI is part of their work. CDTI should be integrated into the overall plan (showing that the DMT members consider CDTI to be part of their yearly routine, like any other programme). It should not be separate. Fully Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? In both cases planning should involve all relevant staff (e.9. sub-district, pharmaceutical staff) preferably in a workshop situation. Findines: l: Oncho coordinator says he takes part fully in the yearly overall planning exercise. No participation of CHEWs in the oncho plan. 2: No participation in either case. Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable Findines: 1: No written plan, all is in the head - complete. 2: Sanre - a routine, not a plan. The plan should make provisionfor all key elemenls. Checklist of key activities: Mectizan supply; targeted training ; targeted mobilisatian/ s ensitis ation ; monitoring and sup ervis ion Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully 2.3 Check if the plan contatns all the elements neededfor CDTI to workwell (indicator of 26 13. Providing 3.1 Check if the district management team @MD is takingfull responsibilityfor CDTI of and attitude 3.2 Check if there is evidence of polttical commitment to CDTI in the district/ LGA indicator of and attitude t4. _ $upgrvision and monitoring 4.1 Check if the relevant person at the LGA/ distrtct level is routinely and efficiently supervising the CDTI activtty of the sub-districts on site (indicator of effectiveness and 4.2 Check if routine data concerning CDTI activities at this level are collected and transmitted within the of Check whether there ts a routine process of management of problems and sltccesses, which are indicated by the monitoring system (coverage data, visits and reports) I I Findines: l: Knows it's going on and is happy; does feel responsible, but responsibility delegated to onchocerci asis coord inator 2: DMT didn't know what was happening - is aware, doesn't initiate it. It should be the DMT which is initiating the kqt CDTI activities : planning, monitoring/ supervision, training, M ectizan ordering/ distribution. The DMT should no longer depend on the State/ regional/ NGDO leadership to carry out CDTI. Not at all Not applicableFully Highly Moderately SlightlyIs this good for sustainability? Findines: l: Supervisory councillor attended the interview - knew what was going on. LGA secretary knew what was going on. Regular disbursements. 2: Chairman knows, not really interested. Never released lunds. The senior person (politician/ civil servant) in charge of health matters at district/ LGA level knows about CDTI and appears committed to it. There is evidence of spectfic past budgetary qllocations and disbursements for CDTI, and the amounts are increasing. Not at all Not applicableFully Highly Moderately SlighflyIs this good for sustainability? Findines: l: Regular routine visits - not focused, no justification 2: Same There should be evidence that each sub-district is visited at least once around the time of distribution. There should not be unnecessarily many visits - here should be clear justificotion for each one. Only the sub-district is routinely visited, not the villages. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? Findinss: l: Collected and transmitted by oncho coordinator - within the system. 2: Sarne. The reporting process should be within the government system, not using other resources. Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Findines: l: They do manage problems, but not based on the data - or-rly what the coordinator brings. They don't 'manage' successes. No As soon as problems are identified in this way, the appropriate manager deals with them. Planning must include qctivities which will improve coverage in areas where it is unsatisfactory. 4.3 of effectiveness and attitude 27 i5. Ordering and distributing Mectizan 5.1 Check if sfficient Mectizan is being ordered yearly, and tn good time (indicator of effectiveness 5.2 Check if Mectizan is betng stored and administered within the government system at this level of 5.3 Check if Mectizan is being distributed appropriately and effectively to the sub- districts of effectiveness Successes are noted and reported, and appropriate feedback given. The CDTI co-ordinator should be empowering the next level to cope with problems. ChecHist of key activities: sensitisation/ mobilisation ; training; improving Mectizan supply ; effec tiv e/ s upportiv e s up ervis io n empowenrrent 2: Same. Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable . The orderformsfor the health district/ LGA exist, and are based on the sub-district orders. . The Mectizan is availablefor the LGAs in timefor distribution at a time which is convenienl to villagers. . There are no reports ofshortages. t lfthere hsve been shortages, there are specific plans to remedy them. Findines: State person orders for LGA, based on previous year's population as at previous treatment - it is enough. The LGA person doesn't make an order. There are no shortages though. They can get nrore in Kaduna if they need. Only one year has been late so far. Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Moderately The Mectizan: . Should be orderedfrom the nart level usingforms supplied by the governmenl. . Should be stored in a room made available by the government at this level - preferably in the same room as the other drugs. . Should be controlled within the government system, preferably using the same stock control system as for other druss. Findines: l: A separate system, oncho coordinator is accountable. Hand-written control system. They believe this is safer. It is less efficiently though. Kept in a separate room. 2: Kept in the coordinator's office - the rest is the same. Slightly Not at all Not applicableIs this good for sustainability? Fully Hiehly Moderately Findines: l: Vehicle supplied by outside donor, but fuel/ maintenance paid for by the district. Sub- district usually fetches the drug. Use of private transporl paid for by district. 2: CHEWs fetched it when informed by LGA level. They use bikes supplied by a donor. The sub-districts should obtain the Mectizan by means oftransport supplied and paidfor by the government at the district/ LGA level Preferably the sub-district shouldfetch the Mectizanfrom the district/ LGA level store. r I Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully 28 16. and sensitisation/ mobilisation NOTE: This refers to training and sensitisation activities performed by officers working at this level, for the next level. In Nigeria the LGA oncho coordinator forms part of the LOCT team. S/tre does her/ his raining at the next level, together with the other CHEWS. done of6.1 Check if ,s and 6.2 Check that training conducted at this level (i.e. training of sub-district staff) ts being done the indicator of and 6.3 Check if in-service trainingfor CDTI is being integrated with other health at this level indicator of and '7. Fin funding 7.1 Check whether the cost impltcations of each CDTI related activity (monitoring/ supervision,traininglmobilisation, Mectizan distribution) are quanttfied in the yearly the health district/ LGA of effectiveness 7.2 Check whether approprtate and adequate amounts are budgetedfor the planned CDTI related activities of effectiveness and effi There should be an objective needfor each episode of training: * There must be evidence that staffto be trained lack knowledge and skills to perform the job. * Training to motivate staffis not a sfficient reasonfor ffaining. The evaluator must be satisfied that the duration of trainins was iustilied. I I Findines: The LGA onchocerciasis co-ordinator does not train the CHEWs at all - in stead, s/he is trained with them by the State team. Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Moderately Findines: The LGA oncho coordinator trains at tl-re sub- district level only (this is the accepted Nigerian systenr ).. Staffof this level only trains the level immediately below it, and notfurther down. Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Highly Findines: This is not l-rappening. Trainingfor CDTI should be integrated into the yearly planfor in-service training in the health district. Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Findines: l: Nothing to indicate oncho specifically - although some general budget items are said to be utilised for CDTI. 2: No plan, no yearly budget. The costs for each CDTI related activity in the year plan should be clearly spelt out in the budget. Recurrent and capital costs (ifany) should be separated in the budget. Not at all Not applicableFully Highly Moderately SlightlyIs this good for sustainability? Findines: l: Not too big, not too small; outcome was good. 2: Not applicable. The amounts budgeted should be neither excessive nor too small - managers should be able to justify the need for each amoun4 in relation to: . The outcomes resultingfrorn the previous year's CDTI expenditure . The cost of similar activitiesfor other programmes. Not applicableFully Highly Moderately Slightly Not at allIs this good for sustainability? 29 7.3 Check whetherfunds to cover these costs are increasingly betng suppliedfrom district/ LGA resources of 7.4 Check if in case of a deficit between esttmated costs and the amount provided by the government, dependable provision is being made to meet ir (indicator of effectiveness and attitude 7.5 Check whether CDTIfunds in the health district/ LGA budget are fficiently managed of effectiveness and 8. Tlq4qport and qlher material lesources 8.1 Check if there is sfficient transportfor necessary visits to the sub-district levelfor CDTI related activities of effectiveness The relative contributions of the district/ LGA and other partners should be clearly spelt out. The proportion provided by the fficial health sertice should be the major one by now. This proportion offunds is increasingyearly. t I Findines: i: Donor: vehicle and repairs; rest: LGA; contributions are not spelt out; both contributions substantial; LGA stable. 2: No lunds being given. Is this good for sustainability? Fully HighIy Moderately Slightly Not at all Not applicable Management is aware of the size of the shortfall, and has specific and realistic plans to bridge il. Ifit k planned that non-government sources of funding are to be used after APOCfunding ends, written commitmentfor this must have been obtained at the highest level in these donor organisations (e.9. a Memorandum of Understanding). Findines: l: PHC chaimran and LGA secretary: aware of the shortlall; no real plan to meet it - say: they could buy a motorbike - not very dependable. 2: APOC and the private pocket are providing transport; management aware of this; no plan. Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable The budget holder uses a control system with the following elements: . Approval of each item of expenditure . Allocation of expenditure against speciJic budget headings . Regular calculation of residual amounts under budeet headinss. Findines: 1 : Only approval of expenditure is done by the budget holder - no control. 2: Not applicable. Is this good for sustainability? Fully Highty Moderately Slightly Not at all Not applicable There are no reports offailure to undertake necessary CDTI activities at sub-district level, due to transport not being available or affordable. Transport provided by the State/ central government level is no longer be usedfor routine CDTI activities at LGA/ district level. Checklist of activities : monitoring/ supervision, training/ mobilis atian, Mectizan distribution I Findines: l: Suflficient transport, paid for by LGA; when staff r-rse their own nloney they recover it. 2: Transport not available - broken down and not repaired; no LGA contribution. Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable I 8.2 30 Check if trips/ journeys undertakenfor routine CDTI purposes are integrated with those made for other programmes or purposes (indicator of integration and of 8.3 Check if the transport made availablefor CDTI is being adequately managed 8.4 Check if adequate provision ts being madefor thefuture, to replace existing means of when it wears out of effectiveness tr Ifuman resources9.1 Check whether the health dtstict/ LGA team is skilled and lvtowledgeable, regarding the CDTI in its area of effectiveness 9.2 Check whether personnel at the health district/ LGA level ts stable, and whether provision is madefor passing on CDTI skills when a trained person moves away indicator of e ffectiveness Findines: l: Trips are not integrated, but the vehicle is used for other programmes as well - occasionally. 2: Trips are not integrated. Ilhere CDTI is involved. trips are authorised/ undenakenfor more than one purpose. Means of transport provided by specific programmes are usedfor other programmes as well. Checklist of activities : monitoring/ supervision, training/ mobilisation, Mectizan distribution Not applicableSlightly Not at allFully Highly ModeratelyIs this good for sustainability? Findines: 1: Authorisation is verbal only. No log books are kept. 2: The same.. Trips madefor CDTI purposes are properly authorised in writing by the relevant fficial. Each trip undertakenfor CDTI purposes is recorded in a log book. Not at all Not applicableHighly Moderately SlightlyFullyIs this good for sustainability? Findines: 1: LGA secretary says they are able to pay for repairs even now, and will have enough funds to replace the motorbike themselves when it finally breaks down. They will however first try to use other donated vehicles. This is believable because they are already paying for the running expenses. 2: No plan at all. The district awhortQ/ LGA has a clear planfor replacing the present vehicle used for CDTI work, when eventually it becomes necessqry to do so. Not applicableModerately Slightly Not at allFully HighlyIs this good for sustainability? Findinss: 1: Knows it all. 2: Fairly knowledgeable. LGA/ health district team members have enough knowledge and skill to undertake all the key CDTI activities themselves, without help : . Planning . Training and sensitisaton/ mobilkation t Mectizan ordering/ distribution . Monitoring/supervision. Not applicableHighly Moderately Slightly Not at allFullyIs this good for sustainability? Findines: l: 3-5 years; provision is made; there is a deputy oncho co-ordinator who can take over for one year; yearly training at Kaduna. 2: 2-3 years - but will be replaced by a person experienced in CDTI Staffshould remain in one postfor at leastfive years. There should be immediate taining (in CDTI) of ncw, unskilled district/ LGA staffmembers who have CDTI responsibilities. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? 31 3. Sub-districU first line health facility level 1. Coverage 1.1 Check whether the geographical coverage in the sub-district is satisfactory (indicator of effectiveness 1.2 Check if all villages have a satisfactory therapeutic coverage rate (indicator of effectiveness 2. Plannlng 2.1 Check if CDTI is an fficial part of health service activities at this level, or is included tn the'basic activities' it indicator of and 2.2 Check if there is a written year planfor CDTI in the sub-districr (it may appear under another name, e.g. 'onchocerciasis progftrrnme') (indicator of integration and effectiveness All villages identified by the latest REMO are under treatment (i.e. the geographical coverage rate is l00ok and stable). This should be contirmed by comparing coverage records with the REMO list of endemic villases Findines: Very good in all 4 Moderately (a few villages/ hamlets not covered) Slightly (rnany villages/ hamlets not covered) Is this good for sustainability? Fully (100% coverage) Highly (problem with nomads only) The therapeutic coverage rate should be: . 65%or higher . stable or increasing. Findinss: Very good in all 4 - above 80%o Highly (80-89% coverage) Moderately (65-79% coverage) Slightly (less than 65%o coverage) Is this good for sustainability? Fully (>90% coverage) There should be an official document locally available, making clear that CDTI is a sub-district responsibility. Sub-district staffshould perceive CDTI to be one of their routine respons ibilities. Findines: l: It is really part of their work, but there is no official document about it. 2: The sarne Not applicableIs this good for sustainability? Fully Highly Moderately Slightly Not at all There should be a written plan or timetable in existence,for the most recent round of CDTI. Ideally the plan should be integrated into the overall year plan for the sub-district. It should not be separate. Findines: 2: no written plar-r, lor any disease control activity 1: there is no written plan for one; the other has a tinretable that he makes for all his programmes Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Highly E 32 2.3 Check if the plan contains all the elements needed for CDTI to work well (indicator of effectiveness .3. Providing leadership Not applicable at this level. ',4. lupervision and monitoring 4.1 Check if members of the sub-district team are routinely and eficiently supervising the CDTI the vil, on site of effectiveness and 4.2 Check if routine data concerning CDTI activities at this level are collected and transmitted wtthin the of Check whether there is a routine process of management of problems and successes, which are indicated by the monitoring system (coverage data, visits and reports) 4.3 of effectiveness and attitude Findines: 3 of them know about it - really in detail; I has also written it The plan should make provisionfor all key eletnents. Checklist of kqt activities: Mectizan supply; targeted training (for CDDs); targeted mobilisation/ sensitisation ; monitoring and supervision Not applicableModerately Slightly Not at allFully HighlyIs this good for sustainability? Findines: 1: Visits are routine, bicycle or motorbike; LGA pays petrol; also calls CDDs to come to him (by letter). At least two per distribution. 2: Routine and excessive. There should be evidence that each village is visited at least once around lhe time of distribution. There should not be unnecessarily many visits - here should be clear justification for each one. Not at all Not applicableFully Highly Moderately SlightlyIs this good for sustainability? Findines: l: CHEW cllecks CDD work - LGA coordinator fetches tl-re data - LGA pays for it - a bit wasteful 2: a f-ew CDDs do the summary sheet - rest done by CHEW The reporting process should be within the government system, not using other resources. Not at all Not applicableFully Highly Moderately SlightlyIs this good for sustainability? Findines: Everything (training, visiting) is done routinely, with no specific attention to weaknesses or problems. As soon as problems are identified in this way, the appropriate manager deals with them. Planning must include activities which will improve coverage in areas where it is unsatisfoctory. Successes are noted and reported, and appropriate feedback given. The CDTI co-ordinator should be empowering the next level to cope with problems. Checklist of key activities: sensitisation/ mobilis ation ; training ; improving M ectizan supp ly ; effectiv e/ s upportiv e s up erv is io n I Not applicableHighly Moderately Slightly Not at allFullyIs this good for sustainability? r 33 5.1 Check if sfficient Mectizan is being ordered yearly, and in good time (indicator of effectiveness 5.2 Check if Mectizan is regarded as afree but essential drug at the sub-district level of and attitude 5.3 Check if Mectizan is being stored and administered within the government system at this level of 5.4 Check if Mectizan is being distributed appropriately and effectively to the villages of effectiveness .t The orderforms for the sub-district exist, ancl are based on the village orders. The Mectizan is availablefor the sub-districts in timefor distribution at a time which is convenient ta villagers. There are no reports of shonages and/ or late supply. Ifthere have been shortages, there are speciJic Dlans to remedy them. Findinss: 2: Mectizan is delivered, not ordered; enough usually; in time; letch more if too little l: sornetinres a bit late Not applicableIs this good for sustainability? Fully Highly Moderately Slightly Not at all There should be an oficial document locally available, making clear that Mectizan is an essential drug at this level. Sub-district staffshould perceive Mectizan to be an essential dras at this level. a Findines: l: Not written but he thinks it is; the villagers value it 2: Same Is this good for sustainability? Fully Hiehly Moderately Slightly Not at all Not applicable The Mectizan: . Should be orderedfrom the next level usingforms supplied by the government. . Should be storedin a room made available by the goventment at this level - preferably in the same room 0s the other drugs. . Should be controlled within the government system, preferably using the same stock control system as for other drugs. Findines: 1,2: sarne room, different shelves; no order forms; stock control separate but same in/out systenr as for others Not at all Not applicableIs this good for sustainability? Fully Highly Moderately Slightly The villages shouldfetch the Mectizanfrom the sub- district health facility themselves Findines: l: village makes written requests; delivered by CHEWI another lets CDDs fetch 2: villagers do not request - come and collect Not applicableIs this good for sustainability? Fully Highly Moderately Slightly Not at all - 34 lc rraintne qn_a sensitisation/ mobilisation NOTE: This refers to training and sensitisation activities performed by officers working at this level, at this level, for the next level. 6.1 Check if is done indicator of and 6.2 Check that training conducted at this level (i.e. CDD training) is done by the ofeffici and 6.3 Check if adequate and appropriate materiqls are availablefor the necessary health education and mobilisation (of villagers) and training (of CDDs), related to CDTI indicator of effectiveness 1,L_ Elqancing/funding 7.1 Check whether adequate provision is madefor the costs tnvolved in each CDTI related at this level of effectiveness I There should be an obiective need for each episode of training: * There must be evidence that CDDs to be trained lack lcnowledge and skills to pedorm thejob. * Training to motivate CDDs is not a sfficient reasonfor training. The evaluator must be satisJied that the duration oJ trainingwas justified. I Findines: 2: training of all CDDs routinely yearly l: lor new developments; for reunion; a planning function Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable Findines: 2: CHEW gets invited to the training, but LGA organises it - State OC is present as supervisor l: CHEW takes responsibility - LGA supervisor comes to check; timing is dictated by the CDTI process Only sub-district staffshould be involved in training of CDDs Not applicableModerately Slightly Not at allFully HighlyIs this good for sustainability? Findines: l: all available, relevant, satisfactory - LGA pays - takes place at local health centre 2: not sufficient but relevant The materials should be easy to use. The materials should be locally relevant. There should be sfficient materials. It should be possible to get new stocl<s of materials when neecled. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? Findines: 2: no funds provided 1 : has inputs into preparation of LGA budget - receives what he requests The costs for each CDTI related activity in the year pan should be clearly spelt out in the budget. The staffshould be able to justify the amount they plan to use Checklist of items to be costed: monitoring/ supervision, training/ mobilisatton ; arranging Mectizan supply I Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? r 35 7.2 Check whetherfunds to cover these costs arefully or increasingly being supplied own and/ or district/ LGA resources of 8. 8.1 Transport and other material resources Check if there ts sfficient transportfor necessary visits to the sub-district levelfor CDTI related activities indicator of effectiveness Check if trips/ journeys undertakenfor routine CDTI purposes are integrated with those madefor other programmes or purposes (indicator of integration and Human resources Check whether the sub-district team is skilled and knowledgeable, regarding the CDTI in tts area of effectiveness Check whether personnel at the sub-dtstrict level ts stable, and whether provision ts madefor passtng on CDTI skills when a trained person moves away (indicator of 8.2 I 9, 9 9.2 7fn (suD-a$lrrct The relative contributions of all sources offunding should be clearly spelt out. The proportion provided by the fficial health service (district or sub-district) should be the major one by now. This proportion offunds is increasing yearly.I Findines: l: all from LGA 2. none at all - even has to pay for some experlses personally Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable Findines: l: APOC pays for the bike and repairs - LGA should be able to replace; LGA pays all other expenses 2: LGA provided tyre once - but CHEWs often use conrmercial transpotl There should be no reports offoilure to undertake necessary CDTI activities at village level, due to transport not being available or affordable. Checklist of activities : monitoring/ supervision, training/ mobilis ation, arranging M ectizan s upp ly I Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Were CDTI is involved. trips are authorised/ undertakenfor more than one purpose. Means of transport provided by specific programtnes are usedfor other programmes as well. Checklist of activities : monitoring/ supervision, tratning,/ mobilisation, arranging Mectizan supply I Findines: 2: vehicle use is integrated - but many trips are purely made for oncho I : vehicle use are integrated, and trips as well Not at all Not applicableIs this good for sustainability? Fully Highly Moderately Slightly Sub-district team members have enough knowledge and skill to undertake all the key CDTI activities themselves, without help. Checklist of activiites : planning, monitoring/ supervision, training / mo bilisation, arranging Mectizan supply. Findines: l: skilled 2: one not officially trained but skilled Highly Moderately Slightly Not at all Not applicableIs this good for sustainability? Fully Staf should remain in one postfor at leastJive years. There should be immediate training (in CDTI) of new, unskilled sub-district staffmembers who have C D TI res p ons ib ilities. I Findinss: 2: I and2 years in position - will be trained if needed l: 7 and l2 years; other sub-district staff will help hinr; also LGA will train Not applicableIs this good for sustainability? Fully Highly Moderately Slightly Not at all effectiveness 36 9.3 Check whether the working conditions of sub-district staffare sattsfactory (indicator of effectiveness Findines: I : satisfactory staffing situation; stationery short; enjoys work 2: salary sufficient; stationery they have to buy; nredical supply system works; shortage of equipntent - bil coping The staffshould at least receive their salaries regularly. Also to be considered are stafing level, medical equipment (for vital signs), medical supplies, stationery I Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? 37 1.1 4. Commu level ,1. Cov_erqge Check that the is 100or4 1.2 Check that adequate therapeutic coverage is being maintained (indicator of effectiveness 2; Planning - - 2.1 Check if the census update is routtnely done at the same time as the distribution of efficienc and 3. Providing leadership 3.1 Check if community leadership gets involved in managing problems and successes with distribution of attitude and effectiveness of effectivenessgVX' U All quarters/ wards and hamlets are being covered satisfactorily. Findine: In all villages all wards were covered - in one case there some nomads were missed Is this good for sustainability? Fully (100% coverage) Highly (problem withnomads only) Moderately (a few villagesi hamlets not covered) Slightly (many villages/ hamlets not covered) Therapeutic coverage of 65% should be maintained or improving Validate the responses from the register/summary forms. Findine: All over 80, renraining stable Fully (>90% coverage) Highly (80-89% coverage) Moderately (65-79% coverage) Slighfly (less than 650/o coverage) Is this good for sustainability? CDDs are asked to describe the steps of the distribution. This should make clear whether thq do the census and the distribution at the same visit. The CDDs should visit eachfamtly only once a yeor (except to follow up absentees/ refusals). Also ask about previous years - what is the routine?I Findines: ln 5 of the 8 villages they did 2 visits Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable The community leadership takes responsibility for the distribution of Mectizan within their village. Ifcoverage (geographical and therapeutic) is not adequate or not being maintained, the leadership understands the reasons for this. The leadership identi/ies and solves problems with distribution at this level. I Findines: Aware of CDD about to give up; using town crier when drugs came late. Another takes responsibility but doesn't help to solve the problern. In sonre no problerns were perceived. Not applicableIs this good for sustainability? Fully Highly Moderately Slightly Not at all 38 4.1 Check whether village level distribution registers are betng kept accurately (indicatot of attitude and effectiveness 4.2 Check whether CDDs are reporting to the sub-district approprtately (indicator of effectiveness 5. _ Meqtizan supply 5.1 Check that the red amount Mectizan is received indicator of 5.2 Check that calculations for the required quantities of Mectizan are acc'urare (indicator of effectiveness 5.3 Check that the required amount of Mectizan is received on time (indicator of effectiveness Findines: Accurate, complete, completed by CDDs Register should be complete and accarate. The register may beJilled in by the CDD and/ or a village helper. Verify by examining the register. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? Findinss: Fornrs were submitted in tinte Reports to sub-district should get to the sub- district on time. Reports may be summary reports, or the original village record, depending on the level ofskill of the CDD Not applicableHighly Moderately Slightly Not at allIs this good for sustainability? Fully Findines: 7 out of 8 got enough; one failed to supply some latecomers The CDDs / villagers report that all villagers who were eligible for treatment got it, and that there was little or no stock lefi over after completing treatment round Moderately Slightly Not at all Not applicableFully HighlyIs this good for sustainability? Findines: They used household summary forms to calculate; also did an update immediately before distribution. ln some cases calculations were based on the previous year's supply. There must be a rational explanation about how the qmount ordered for the village is calculated (on the basis of population). The calculation may be done by the CDD or the health centre nurse, depending on the CDD's level of skill. Not applicableHighly Moderately Slightly Not at allIs this good for sustainability? Fully Findines: 7 out of 8 villages received it on time. The other received it late. one year only. The CDD and the villagers report that the Mectizun arrives at a convenient and acceptable time ofthe year. Not applicableModerately Slightly Not at allFully HighlyIs this good for sustainability? l.- _Uqoitoring TI 16._ 39 Training Not applicable at this level - CDDs are trained, but do not conduct training. Financ_[ng 7.1 Check that the villagers appreciate the financial and non-financial costs involved in CDTI of 7.2 Check that the community has made affangements tofund local costs of distribution of i8. Transport and other material resources Not applicable at this level. 9. Human resources 9.1 Check that all CDDs have received train indicator of effectiveness 9.2 Check that there is a system in placefor new CDDs to acquire the appropriate skills Briefly tell villagers (including the village head) what the component activities of CDTI are: mobilisation; triining; collecting drugs; the actual dis tributio n, s up ervis io n. Then ask them where they think resources for these activities comefrom. Thqt should show an understanding that the resources comefrom outside anclfrom the work of the CDD, and etpress their appreciation. Findines: Tlrey are aware and are contributing in various ways. Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable The community makes provisionfor the supply of record bool<s, pencils, transport and other expenses incurred (including possible remuneration for CDDI) Findines: ln most villages they did, in one way or anotlrer: paid for things, or lent bicycles, or helped in their fields. Not at all Not applicableIs this good for sustainability? Fully Highty Moderately Slightly CDDs report that they have received training and demonstrate understanding of the work. Checklist: doing the census; doing the reports, giving the right dose; knowingwho is not eligible; knowingwhat to do with side-effects Findines: All had been trained; all except one appeared skilled. Is this good for sustainability? Fully Highly Moderately Slightly Not at all Not applicable There should be a plan in placefor training CDDs to replace those who drop out, or when new ones are elected for other reasons. Findinss: There is yearly training, so they can be trained then. Or-re village intends to have backup CDDs Fully Highly Moderately Slightly Not at allIs this good for sustainability? Not applicable of effectiveness 40 9.3 Check whether CDDs are to continue with CDTI of 9.4 Check whether villagers value annual treatment (indicator of attitude and 9.5 Check whether villagers accept the needfor long term CDTI (indicator of attitude and effectiveness Findine: The great majority are willing; one expressed doubts CDDs shoulcl express willingness to continue with distribution in the long term, given the conditions which prevail in the community. Not applicableSlightly Not at allFully Highly ModeratelyIs this good for sustainability? Findine: Tl-rey know several benefits. The community members should mention one or more advantage ofMectizan, and express the needfor the annuul treatment. Slightly Not at all Not applicableFully Highly ModeratelyIs this good for sustainability? Findins: They are willing to continue, even keen on it People show understanding and express interest in long term treatment with Mectizan. Not at all NotHighly Moderately SlightlyFullyIs this good for sustainability?