Appendix 9: intruments 1-4 (instruments used for evaluation of the sustainability of Bench-Maji CDTI project), January-February 2006

Appendix 9: Instruments l-4 (Instruments Used for Evaluation of the Sustainability of Bench-Maji CDTI Project) January-February 2006 filr)- " i1 I_ lnstrument 1: national and project level .- NOTE: . ln countries like Malawi, Uganda and Chad it is the NOC and her/ his team that is actually implementing CDTI projects - while also having the national role of interacting with the NOTF and the Ministry of Health. This instrument collects information about everything these ' i people do. All its indicators are therefore relevant to them. . ln countries like Nigeria, Cameroon and DRC these functions are shared by two groups:! " The project managers (State levelin Nigeria, p0ect levelin Cameroon and DRC). - A headquarters group which oversees the several projects as they go about their work. Some indicators in this instrument are only relevant to one of these groups, not both. Others are relevant to both. The focus of this level's activities in CDTI The main function of this level is to develop, revise as necessary, and oversee implementation of CDTI policy. This level also provides supportto the level below it: . Providing targeted training, HSAM and monitoring/ supervision. . Arranoino for an adeouate suoolv of Mectizan. Researcher: Dr E Nnoruka , MrW Musolo, Mr Hamus Date: 25' January - 15'n February2006 Respondents: Zonal Director of Health, Zonal Oncho Cordinator, Zonal Director Public Health u Diseases, NGDO Project Officer, ZonalAccountant, Capacity Building Desk, ZonalTransport Desk, Abbreviations/ acronymsrJ CDD community directed distributorCDTI community directed treatment with ivermectinr1 FLHF first line health facility HSAM health education, sensitisation, advocacy, mobilisation - i.e. activities which are aimed at getting allthe key players to participate wholeheartedly in the r \ programmeNGDO non-governmentaldevelopment organisation .- NOC nationalonchocerciasis coordinator NOCT nationalonchocerciasiscontrolteam _ NOTF national onchocerciasis task force Sustainability evaluation instrument no.1 - national level - September 2004 version 2fi!r{!":H kYffiffi*'#i," 1.1 Check whether there is adequate yearly planning for onchocerciasis control in the health service at this level. This indicator assesses whetherthe programme has become integnted intothe heafth seruice; whether management is accepting ownerchip of the programme; and whether the project is functioning effectively in an integrated manner Characteristics of the indicator Sources of information a. ln the overall written year plan of the health service at this level there should routinely be a section on onchocerciasis control. Note that this plan is usually not very detailed. b. There should also be a more detailed plan, specifically for onchocerciasis control, which makes provision for all key elements of onchocerciasis control: Mec{izan supply; targeted training ; targeted HSAM ; targeted monitoring/ supervision. (Note that in the case of a c,ountry which contains many projects, the NOTF plan will be less detailed). c. The plan varies from year to year, showing that it is targeted to the soecific needs of each vear. . Examination of: * Written plans: yearly, quarterly, monthly etc. * Minutes of planning meetings. . lnterview with senior health service staff at this level: + Project director, her/ his team, her/ his superiors.r NOC, her/ his team, her/ his srrneriors Describe the present situation: There are yearly written activity plan for onchocerciasis control (200212003 - 2005/2006) containing all CDTI activities at the project office. Plans do not vary from year to year and is not targeted to specific needs. lnputs are obtained from mainly the Woreda Health offices. Similarly at the Zonal Health office there were Bugdet plans ,which lumped Onchocerciasis under Vector control for similar periods for Bench Maji health office. These were not detailed and did not reflect key elements of Onchocerciasis control. Plans were essentially the same without targeting areas of specific needs. ls there a practice of integration? Plans did not reflect any form of integration, despite the fact that vehicles and office equipment are utilized in an integrated manner. lf planning and implementation of CDTI is not part of the overall year plan: . Why is this? Planning for CDTI activities was not taken seriously from inception of the project because it had been perceived as an NGDO project. . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved.i Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overall iudqement is this indicator for sustainabilitv beino achieved? Fully I Highly I Moderately I Sfghty I Not at all I Not applicable a Sustainability evaluation instrument no.1 - national level - September 2004 version 1.2 Check whether all partners (government, UN agencles, NGDOs etc.) are meaningfully involved in the overall This indicator assesses whether the programme is functioning effectively - if each partner is clear about its role, this is good for sustainability. Gharacteristics of the indicator Sources of information a. All partners should contribute to the routine planning of a project. b. Partners should be clear about their own roles, and those ofthe other partners. Examination of * Plans: yearly, quarterly, monthly etc. * Minutes of NOTF meetings. lnterview with senior staff at this level: ' P0ect directors and their superiors. ' NOC and persons in MoH superior to her/ him. . NOTF members. . Other stakeholders: relevant UN aqencies. NGDOs Describe the present situation: NGDO partner plans together with zonal offie and Woredas yearly. Some partners particularly Zonal is not yet clear of its role in regards to CDTI activities in Onchocerciasis control. There were no minutes, no lf partners are not meaningfully involved in planning: . Why is this? Poor understanding and insufficient awareness of CDTI Strategy. . \Mich steps are being taken to improve the situation? Planning to embark on HSAM activity to the Policy makers at the zone. Analvsis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. Your overall iudqement: is this indicator for sustainabilitv beinq achieved? Fully I Highly I ilodffaildy I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version Ihis assesses whether the programme is ', and that management has begun to 1.3 Check whether specific planning for sustalnability has taken place, for the period after APOG fundino is withdrawn. take ownerchip of it and can mobilise the resources it needs. Characteristics of the indicator Sources of lnformation a. Members of NOTF as well as project managers have made plans for this period, which will enhance programme sustainability. This planning should include: identifying resource gaps; strategies to cut expenditure; and strategies to find dependable sources of resources. b. There should be written evidence that such planning has taken place. c. There should be evidence that the plans are being successfullv imolemented. . Examination of the written sustainable plans. . lnterview with senior staff at this level: * P0ect directors and their superiors. . NOC and persons in MoH superior to her/ him. * NOTF members. * Senior NGDO staff. FI Describe the present situation: There are no Sustainabihty plans yet at Zonal prqect office nor at the Zonal Health office. Zonal Director for Health has no specific plans for counterpaft funding from Government, since the amount allocated for health is not much Feels Government is poor therefore the suppofting partners should continue funding CDTI. Cunently NGDO paftner has no Post APOC plans yet but believes they will continue with technical supporl for Bench Maji CDTI propcts. lf there has been little or no planning for sustainability: . Why is this? Despite being informed by NGDO partner, Zone project office has not yet deemed it necessary yet. . Which steps are being taken to improve the situation? None Analysis . When writing the report you have to summarise:I The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustai nabilitv. . Your overalljudgement is this indicator for sustainability being achieved? Fully I Highly I Moderately I Slightly I ltbtdC lNotapplicable Sustainability evaluation instrument no.1 - national level - September 2004 version 2.1 Check whether the various programme support activities are being planned and carried out in This indicator assesses whether the programme is functioning Characteristics of the indicator Sources of informatlon a. Staff should provide o A written work plan o Plan shows how implementation of activities in an integrated manner will be achieved b. Staff combines two or more tasks on a single trip: * Monitoring / supervision for CDTI. * Training for CDTI. * HSAM. * Fetching records. * Delivering Mectizan. b. Staff combines CDTI activities with those of other programmes, where this is relevant (e.9. supervision, lrainino. HSAM. Mectizan deliverv). . Examination of documents: trip authorisations, log books, trip reports etc. . lnterviews with: * Staff from this level (managers, administrators, drivers etc.). * Staff from the next level below. Describe the present situation: There were no written plans for CDTI lntegration with other PHC activities at the Zonal level. All tasks carried out by staff are purely Oncho activities just like other health programmes carry on with their own, however equipment and transport are utilized in an integrated manner. Training, mectizan deliveries or returns of reports are solely done on its own.There are no trip repo( of integrated activities within the field. Describe the situation the year before: Same as above lf integration between support activities is poor: . Why is this? 'Because we are poor in keeping note we have it in our heads says the Zonal oncho Coordinator. . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. Examine the trend in the integration of activities - is it becoming more common? Your overalljudgement: is this indicator for sustainability being achieved? Fully I Higtrty I Moderately I Sfghry | Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2OO4 version I6 ffi{,MMm }Wir_'}*#,i 3.1 Check whether NOCT/ NOTF/ prolect leadee are exercising effective leadership for the at their level. This indicator assesses whether this leadership is functioning etfectively, and has taken ownerchip of the programme. Characteristics of the lndicator Sources of information a. The leaders are fully aware of the progress, successes and problems of the projecU projects for which they are responsible. b. The leadership is up to date with its reports and targeted activities. c. The leadership delegates responsibilities appropriately, to colleagues at this and lower levels. d. The leadership has a sound and collegialworking relationship with junior colleagues, and keeps them fully informed. e. NOTF meets regularly and its members are fully aware of current issues in the CDTI programme. . lnspection ot ' Year plans. . Reports. ' NOTF minutes. . lnterview with senior staff at this level: * APOC HQ managers. ' Staff at project level. ' NOC, her/ his superiors in MoH. ' NOTF members. ' Senior NGDO staff. * DislricJ/ I GA staff Describe the present situation: There is aZonal Focal person in charge of the project. He is aware of the project to some extent and is able to handle some problems in the field,while the Zonal Health Dept is not fully aware of its responsibility to its CDTI project. Leadership is not up to date with reports and targeted activities. No specific training or monitoring/supervisory reports are available however there was a 20031 2004 perfomance report for the project. Leadership of Bench Maji has been subject to a lot of politics and so has accounted for poor documentation of reports. Leadership delegates responsibilities to his colleagues at this level particularly during monitoring and supervision. NOTF/Regional/NGDO partner guide the zonal level from time to time. There are specific NGDO quarterly reports for 2003, lf leadership at this level is not effective: . Why is this? Leadership alZonal level has not fully understood it role in CDTI strategy for Bench Maji. . Which steps are being taken to improve the situation? NGDO has spoken to them on the issue of leadership and has been monrtoring them qurte closdly. . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affecl Your overall judgement: is this indicator Susteinability evaluation instrument no.1 - national level - September 2004 version 4.1 Check whether data and records concerning CDTI activities are being effectively handled this level. This indicator assesses whether the programme is functioning effectively. Characteristics of the indicator Sources of information a. lmportant records are readily available at this level: treatment summary records, inventories of equipment, financial records, periodic reports (e.9. annual technical reports). b. The records are of good quality: the content is clear and seems accurate/ trustworthv. . Examination of the documents mentioned, and their source documents (e.9. districU LGA reports). . lnterviews with: * Managers and staff at this level. ' Staff at the districU LGA level. Sustainability evaluation instrument no.1 - national level - September 2004 version !/ Describe the present situation: There were available treatment summary reports(2003-2005), inventories of equipment, proposal requests for release of funds for CDTI activities for (2003-2005),NGDO annual plan reports. Content of reports are clear Only reports from the zone comprising mainly of treatment reports generated from the Woreda levels are sent to direct to NOTF/MOH at Addis. There were no finacial records and nor annual periodic repofts. lf records are not being handled properly: . Why is this? Unaware of the importance of report writing. . Which steps are being taken to improve the situation? Analysis a When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect suslainabilitv. . Your overall judgement: is this indicator for sustainability being achieved? Fully I Hbhry | Moderately I Slightly | ruot at att I Not appticabte 4.2 Check whether staff at this leve! ls used for This indicator assesses whether the programme is functioning efficiently. Characteristlcs of the indicator Sources of information a. Staff members at this level should routinely only supervise the level immediately below them. Staff should nof supervise the FLHF or community levels. 'Spot checks' may however be done from time to time. b. Staff members at this level should have empowered staff members at the level below them to supervise activities at lheir own level, as well as levels furlher down. . Examination of: * Supervisory checklists, plans and reports. * Visitor's books at all the levels below this one. * Trip authorisations. . lnterviews with: * Staff at this level. * Staff at levels below this one. Describe the present situation: Routine monitoing and supervision has been carried out yearly by the Zonal project office staff.This was carried out to all levels( Woreda, FLHF and communftes)in the guidance of NGDO partner. Staff at the zone remain in the Woredas for monitoring and supervision, but move around with the Woreda staff, FLHF officer, and CDDs within the kebeles until after distribution. They monitor training, census update and the distribution within communilies. Occassionally spot checks are carried out to the inaccessible areas by the woreda staff. Describe the situation the previous year: Same as above. lf staff members are not being used appropriately for monitoring/ supervision: . Why is this? . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affecl suslainabilitv. . Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately | Sftlily I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version 4.3 Check whether monitoring/ supervlslon is being planned and carried out in an eflicient manner. This indicator assesses whether the programme is functioning Gharacteristics of the indicator Sources of infomation a. One routine supervision visit per year should be done to the next level (project or districU LGA). lf more visits are carried out the need should be clearly indicated. b. Resources for supervision (human, transport etc.) should be efficiently used: * Using as few staff members as possible. * Planning trips to cut down on distance travelled. * Not spending unnecessarily many nights out etc. c. Suoervision visits should be thorouoh. usino a checklist. . Examination of: * Supervisory checklists, plans and reports. * Visitor's books at the level immediately below this one. . lnterviews with: * Staff at this level. * Staff at the level. immediatelv below this one. Describe the present situation: There are several monitoring and supervisory visits conducted over a period of 4-6 weeks yearly to all levels by the Zone during distribution. All supervisory and monitoring visits are routine visits not targeted to improving needy aspects. Four zonal project staff in addition to NGDO are involved with supervisory visits and move into the field with all transport available to the project. Supervision is not integrated. Checklisfs are used. lt is routtnely done and there are no reports All superwsory vistts were dependent on APOC funds. Describe the situation the previous year: Same as previous years . lf monitoring/ supervision is not being done efficiently: . Why is this? No previous understandng of exact Zonal role Unaware of the need to target monitoring and supervision according to needs. . Which steps are being taken to improve the situation? Nothing yet. Analvsis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Examine the trend in monitoring/ supervision activity: is it becoming more efficient? Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I Moderately I Slgttfy I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version l0 4.4 Check whether there is a routine process of management of problems and successes, which are Indlcated by the monitoring system. This indicator assesses whether the programme is running efficiently and effectively, and whether management is beginning to accept ownerchip of the programme. Characteristics of the indicator Sources of information a. As soon as problems are identified as a result of supervision visits, or from coverage data (i.e. areas with low coverage) the appropriate manager should dealwith them. b. Such problems should usually be passed on to the appropriate managers at the next level below to deal with, with the necessary support - thus empowering these persons. c. Successes should be noted and reported, and appropriate feedback given. d. There should be evidence of action taken based on recommendations in the reports of previous monitorino exercises. . Examination of the following documents: * Plans: yearly, quarterly, monthly etc. * Minutes of staff/ planning meetings. * Reports of previous monitoring exercises. * Letters of commendation. * Letters with information and feedback. . lnterviews with: * Staff at this level. * Staff at the levels below this one. Describe the present situation: When a problem such as refusals and low coverage are identified, measures are taken by the Zonal Oncho project Co-ordinator to tackle this in the feld and this is even extended to lower levels if he s called upon. There is a reward system in place whereby NGDO rewards peiormance by giving a Cerlificate. Cerlificates were seen Verbal reports are the main system of giving feedback reports. lf the system of managing problems/ sucoesses is weak: . Why is this? . Which steps are being taken to improve the situation? Analysis a When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I moOrd*y I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version ll &..Lltl 4 4.1 Check whether sufficient Mectlzan is belng ordered, storcd and distributed within the at this level. This indicator assesses whetherthe programme isfunctioning efficienily, ifsprocesses are simple, and it is becoming more integrated into the govemment system. Gharacteristics of the indicator Sources of lnformation The Mectizan supply should be controlled within a government system. This does not have to be the system routinely used for the supply of other drugs. The system should be effective, uncomplicated and efficient. This system should use dependable, sustainable resources for its operation. lt is desirable that these resources should be supplied by the government. The system should supply sufficient Mectizan for the needs of all the oroiects concerned. in qood time. . Examination of all Mectizan ordering and stock control documentation at this level. . lnterviews with staff at this level (government, NGDO etc.). Describe the situation the previous year: Zone picks mectizan from Addis once information is passed down by the Region from NOTF( once mectizan is cleared by WHO). Mectizan is then stored in the Zonal Drug store from where it is dispensed to the FLHF. Mectizan is thus controlled within the government system. The system is effective, uncomplicated and efficient. APOC provides funds for collection and distribution to the communities. Mectizan documentation were available however there were no stock cards. Zonal office sends treatmenUpopulation data to Regional Health Offlce who then place orders through NOTF with the result that excessive amounts of Mectizan are ordered yearly. Zonal Oncho coordinator has no logical way of how Mectizan orders are being made. Eg in 2003 1 , 1 09,000 tablets of mectizan were received , 342,555 tablets were used, 1 029 wasted, thereby giving a balance of 765,416 tablets as left overs. ln 2005, 1183,275 were received, 890,469 were utilized, 2455 were wasted leaving a balance of 290,351. Mectizan is recieved in good time for instance Mectizan for 2006 is already available at the zonal drug store. Describe the situation the year before that: Same as above. . Why is this? . Which steps are being taken to improve the situation? Analysis a When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . What has been the trend in Mectizan supply? . Your overall judgement is this indicator for sustainability being achieved? Fully I Hhhly I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version t2 ffi,s's This indicator assesses whether the programme is Characteristics of the indicator Sources of information a. Staff at this level should routinely only train staff at the level immediately below it. b. Staff members at the national level should have empowered the level immediately it to conduct training activities at the levels below it again (i.e. districU LGA or FLHF) independentlv. . Examination of training materials, planv programmes, reports: * At this level. * At the levels below this one. . lnterviews with: * Staff at this level (the trainers). * Staff at the very next level below (the trainees). * Staff at the districU LGA level. * Staff at ihe FLHF level. Findi Describe the present situation: Staff at this level routinely train Woreda staff,FLHF and CDDs. Training manual for CDDs was available but there were no training reports available. Describe the situation the year before: Same as above. lf staff members are not being used efficiently as trainers: . Why is this? . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Examine the trend in the way in which staff are being used as trainers: . Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately I Slglily I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version l3 This indicator assesses whether the programme is functioning efficienily and in an integrated manner. Characteristics of the indlcator Sources of information a. There should be an objective need for each episode of training. This means there should be evidence that staff to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Repeat training of already skilled staff should not happen. b. Resources for training (human, transport etc.) should be efficiently used:r Using as few staff members as possible. * Using as little time as possible (without sacrificing quality) * Choosino the most epst-effeclive site etc. . Examination of training materials, plans/ programmes, reports: * At this level. * At the levels below this one. . lnterviews with: * Staff at this level (the trainers). * Staff at the very next level below (the trainees). * Staff at the districU LGA level. * Staff at the FLHF level. Describe the present situation: Training is routine and not targeted to need. Neither was it imntegrated. I person trains 10 people per training session at the Health center. Training is for 2 days for CDDS, 1 day for Health workers and 1 day for retraining of CDDs. APOC funds were utilized for training Describe the situation the year before: Same as above training is not being carried out in an efficient and integrated manner: . Why is this? . Which steps are being taken to improve the situation? Analysis I \A/hen writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustai nabilitv. . Examine the trend in training activities - is it becoming more efficient and integrated? . Your overalljudgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately I Shn0y I Not at all I Not appticabte Sustainability evaluation instrument no.1 - national level - September 2004 version ai 6.3 Check whether staff at this Ievel is plannlng and carrying out HSAM activities in an efficient manner. This indicator assesses whether the programme is efficiently and effectively, and whether managers are taking ownership of the programme. Characteristics of the indicator Sources of information a. Staff members identify situations where decision makers lack information abouU commitment to CDTI, and undertake activities to inform and persuade these persons. b. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. c. Such activities should only be carried out at the national level, and at times at the level immediately below (but only when staff at that level asks for help). d. There is evidence that these HSAM activities have been effective and have led to action. Examination of HSAM plans/ programmes and reports. lnterviews with: * Staff (programme and management) at this level.t Civil authorities at this level.t Staff and civil authorities at the next level down. Describe the present situation (in relation to efficiency and outcome): There has been no evidence of HSAM activity alZonal level. No HSAM activity reports/nor future plans are available despite the fact that proposals for these activity were made for2003-2005. Describe the situation the year before (in relation to efficiency and outcome): Same as above. lf HSAM activities are not being carried out efficiently: . Why is this? No reason given for this. . Which steps are being taken to improve the situation? Nothing Analysis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affec{ sustai nabilitv. . Examine the trend in HSAM activities - is it becoming more efiicient and effective? . Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I Moderately I Slightty I HotCd lNotappticabte Sustainability evaluation instrument no.1 - national level - September 2004 version l5 ?ir'r'i,Ifffiffi j+*\*,*q$SffiWeq'],'"{{tr,W 7.1 Check whether appropriate amounts are budgeted for planned onchocerciasis control activities at this level This indicator assesses whether the programme is functioning efficienily. Gharacteristics of the indlcator Sources of lnformation The costs for each onchocerciasis control activity in the year plan at this level should be clearly spelt out in a budget. There is evidence of a cost reduction/ containment strategy (e.9. targeted training, HMAS and monitoring/ supervision; training conducted at the next level below etc). Projecl managers should have a clear estimate of the funds that will be available to them for onchocerciasis control in the coming year, and should be able to justiff this belief. The total amount budgeted for in the year plan should fallwithin this estimated income. Examination of the budget documents. lnterviews with the health service managers at this level. The budget and estimated income: . For this year: There are no clearly spelt out budget estimates for CDTI activity for this year in the zone. Estimated incomes are reflected below: There are no evidence of cost reduction /containment strategy. Training is not targeted, HSAM not carried out and M& S is carried out for very long time while in the field. Zonal coordinator is unaware of funds allocated for CDTI activities and has no budget proposal for various CDTI activities for the coming year. For the previous year: Same as above. lf budgeting has been inappropriate: . Why is this? . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affec{ suslainabilitu. . Examine the trend in the budgeted amount and the expected income: . Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately I Slightly I Not et aI I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version l6 7.2 Check whether the government is budgeting and dlsburslng sufflclent amounts for onchocerciasis control yearly. and in qood tlme. indicator assesses whefher the programme is becoming integnted, and whetherthe govemment is accepting ownership of the programme and can mobilise the resources it needs. Characteristics of the indicator Sources of lnformation a. The relative budgetary contributions of the govemment and other partners to onchocerciasis control should be clearly spelt out. b. The amount that the government has budgeted in one or more specific onchcerciasis control budget lines (e.9. current and capital) should be increasing yearly, as a proportion of total expenses. By the end of Year 5 of APOC funding the bulk of onchocerciasis control expenses at this level should be met from government funds; by the end of Year 3 at least half of it. c. The amounts actually disbursed from such budget lines should be increasing yearly, as a proportion of total expenses. (Note that actual disbursement is more important than budgeting, and is a real sion of oolitical commitment). . Examination of: * Budget documents (government, NGDO etc.)t Records of disbursement and expenditure (ledgers, orders, approvals for expenditure etc.) . lnterviews with managers at this level (government, NGDO etc.). The budget and disbursements: . For this year Government has not yet fulfilled its own role in regards to counterpart funding of the CDTI programme The relative budgetary proposed commitments (from letters of agreement) are as follows:Year APOC NGDO Partner Government(Regional/Zonal)2002t3 $283,446 $81,830 $78,8952004 $170,440 $73,872 $106,5642005 $142,184 $69,143 $120,176 Bench Maji currently in its 3'd year of CDTI programme implementations is still heavily dependent on APOC funds and government has just been providing salaries and office space. A zonal account ( as requested by APOC) was only opened this month January 2006. Actual disbursement for NGDO partner were seen. . For the previous year: Same as above, there were no government releases apart from salaries. . For the year before that: Same as above, there were no government releases apaft from salaries. lf the government proportion of expenditure is not increasing proportionately: . Why is this? According to one of the Zonal officals interviewed' we are poor and so external donors should continue funding this programme to help our people'. . VVhich steps are being taken to improve the situation? No steps are being taken yet. Analysis a When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Examine the trend in government budgeting and disbursements: . Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately | S{eh$y I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version t7 7.3 Check whether in case of a deficit between estimated costs and the amount provided by the is belno made to meet it. This indicator assesses whether the govemment is able to mobilise the resources it needs, as well as its commitment to ownership. Characteristics of the indicator Sources of information a. Project management at this level should be aware of the shortfall, if one exists, and of its size. b. Project management should have specific and realistic plans to bridge the shortfall. c. lf it is planned that non-government sources of funding are to be used afier APOC funding ends, wriften commitment for this should have been obtained at the highest level in these donor organisations. Projects in Year 3 of APOC funding should also be well on the way to achieving such commitment. . lnspection ot * The budget documents. (government, NGDO etc.). * Records of expenditure (ledgers, orders, approvals for expenditure etc.). * Letters of agreement. . lnterviews with managers at this level (government, NGDO etc.). Describe the present situation: Zonal Management are unaware of any existing shortfalls, nor of its magnitude. There are no plans to bridge the shortfall. There were no records of budget documents nor expenditures made. Currently because of poor retirement APOC funds have not yet been released for the current year and NGDO had to wade in to help the project. Describe the situation the previous year: Zonal Management are unaware of any existing shortfalls, nor of its magnitude. There are no plans to bridge the shortfall. There were no records of budget documents nor expenditures made. lfffi . Why is this? Becaue they are unaware of source of funding for trheir CDTI projects . Which steps are being taken to improve the situation? Analvsis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affeci sustainabilitv. . Examine the trend in shortfall and how it is to be supplemented: . Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I Moderately I Slgttty I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version l8 7.4 Check whether funds disbursed for onchocerciasis control from the budget at thls leyel are This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of lnformation a. The budget holder should be using a control system with the following elements: * Approval of each item of expenditure. * Allocation of expenditure against specific budget headings. * Regular calculation of residual amounts under budget headings. b. All the funds released vearlv should be sDent as budoeted. . lnspection of: * The budget documents (government, NGDO etc.). * Financial control records (ledgers, orders, approvals for expenditure etc.). . lnterviews with managers at this level (oovernment. NGDO etc.). Describe the present situation: Requests proposals for CDTI activities were available but actual approvals of proposals made for the various CDTI activities each year were not available. There were no finacial disbursments reflecting yearly releases, expenditures or regular calculations of residual amounts seen at the zonal level. However there were queries issued to different woreda health workers requesting retirement of funds given to them for 200312004. . Approval of expenditure: . Allocation of expenditure: . Regular insight into budget line balances: lf the funds are not being well managed: . \ /hy is this? . Which steps are being taken to improve the situation? Analvsis When writing the report you have to summarise:i The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustai nabilitv. . Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I Moderately J Slightly I iaot it.f I Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version t9 &J"'}.'milm :r.3gffiffiF$*! 8.1 Check if adequate and appropriate transport and other material resources are available for CDTI activities at this level. This indicator assesses whether the programme is functioning effectively, and whether it is able to mobilise the resources it needs. Characteristics of the indicator Sources of information a. There are adequate numbers of functional vehicles available for necessary CDTI activities. b. The vehicles are appropriate for the purpose they are intended to fulfil - tough but not luxurious. c. There is sufficient office equipment available, in working order: computers, printers, photocopiers - also stationery and materials for training and HSAM. d. The running costs for these vehicles and equipment are met from deoendable. sustainable souroes. . lnspection ot * Each vehicle in the pool, each piece of equipment its source; its functional status. * Training materials and stationery stocks. . lnterviews with managers at this level (government, NGDO etc.). Describe the availability/ suitability/ functionality of the present vehicles, equipment and materials, considering the work still to be done in the coming 5-10 years: Vehicle is in fairly good order but not suitable for the job. Equipments are available and suitable for the job for the next coming 2-3 years. Will definitely require new vehicle and equipment for the post Post APOC era. IEC material and training manuals are currently grossly inadequate. Running & maintenance of the vehicles are bv APOC. Toyota double Cabin Still working but unsuitable b/c it uses petrol which is scarce in Bench Maji.( Diesel Double cabin should be most aooriooriate next 5 still working, 3 non functioning; suitable for the ls workinq, suitable & enouoh for the iob.. ls workinq. suitable & enouqh for the iob. ls workinq, suitable & enough for the i ls workinq. suitable & enouqh for the ls workinq, suitable & enouqh for the iob. ls workino, suitable & enouqh for the Non funclional from onset because of fluctuation in ' APOC, MoH, NGDO, other (specify) ** ls it working? ls there enough of it for the job? ls it lf transport, equipment and materials are inadequate and/or funded from sources which are not dependable: Why is this? All have been supplied by APOC apart from the IEC materials. . Which steps are being taken to improve the situation? Sustainability evaluation instrument no.1 - national level - September 2004 version . When writing the report you have to summarise:t The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall iudgement: is this indicator for sustainability being achieved? Sustainability evaluation instrument no.1 - national level - September 2004 version 2t 8.2 Check if transport and other material resources in use at this Ievel are adequately and maintained. This indicator assesses whether the programme is functioning effectively and efficiently Characteristics of the indicator Sources of information a. There is a routine maintenance schedule for each vehicle, which is adhered to and recorded. This includes weekly driver maintenance, scheduled garage servicing, and replacement of worn tyres. b. Equipment such as photocopiers and generators is regularly maintained according to a schedule, and this is recorded. c. Staff members have ways of coping when vehicles break down or are not available, so that CDTI activities are not disrupted. d. The costs for vehicle and equipment maintenance and repair are met ftom dependable/ sustainable sources. e. Reoairs to vehicles and eouioment are raoidlv and efficientlv done. . lnspection ot * Vehicle and equipment maintenance schedules. * Vehicle and equipment maintenance records. . lnterviews with managers at this level (government, NGDO etc.), drivers. Describe the present situation: Routine maintenance schedules were not available, nor were log books, but there was a system by which vehicle movement were known. There were no schedule for mainatenanc of vehicles and other equipments. Vehicles are used in an integrated manner and so staff have a way of coping. APOC is responsible for repair and maintenance of vehicle and equipment, but records of such repairs are not documented. Repairs to vehicle and equipment are not efficiently nor rapidly done for rnstance the 3 broken down motor bikes and fax equipment have not yet been repaired because, Zonal co-ordinator has not yet deemed it necessary. Nor has he put up a request or proposed budget for its repair. lf the vehicles and equipment are not being well maintained, and/ or funded from sources which are not dependable: . Why is this? APOC has been the sole source of funding. . Which steps are being taken to improve the situation? Nothing yet. . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement is this indicator for sustainability being achieved? Sustainability evaluation instrument no.1 - national level - September 2004 version 22 Gharacterlstics of the indicator Sources of lnformation a. Transport is used at this level, and to undertake suppoft activities at the next level. lt should not be used for CDTI implementation activities at lower levels. b. The use of transport is properly controlled:r Trips made for CDTI purposes should be properly authorised in writing by the relevant official.r Each trip undertaken for CDTI purposes should be recorded in a log book. * Trip authorities and log book entries are regularly reconciled, and aciion laken if there are discreoancies. . lnspection of vehicle control documents: * Copies of trip authorities (also noting destination and purpose). * Log books. . lnterviews with managers at this level (government, NGDO etc.). Describe the present situation: Transport is used in an integrated fashion, however in most cases it gets to the lower levels(FlHF and Kebeles) during monitoring/ supervision and distribution There are no trip documents,log books nor relevant official document to show vehicle movement. lf the transport is not being well managed: . Why is this? . Which steps are being taken to improve the situation? Analvsis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performancc (if any); steps being taken to improve it; and how this is likely to affect sustai nabilitv. . Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I Moderately I Slightly I NotetC lNotapplicable Sustainability evaluation instrument no.1 - national level - September 2OO4 version 23 8.4 Check if there are appropriate and realistic plans for the replacement of transport and other material resources, when APOC support comes to an end. This indicator assesses whether the programme managers are taking ownership of the programme, and are able to find resources for it. Characteristics of the indicator Sources of information a. Management should know that replacements will be needed before the end of the programme, and have specific, realistic plans to meet the need at that time. b. lt should be planned that the government will: * Provide replacements for vehicles and equipment. + Maintain existing vehicles and equipment. * Provide stationery and materials for training and HSAM. c. lf it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in these donor organisations (end of Year 5), or negotiations should have started (end of Year 3). . lnspection of lefters of agreement. . lnterviews with: * Programme managers at this level (government, NGDO etc.). * High-ranking Ministry officials and other decision makers at this level. Describe the present situation: There are no realistic plans yet for replacement of vehicle, and equipment when APOC comes to an end. lf the plans for replacing vehicles, equipment and materials are unsatisfactory: . Why is this? Unaware of the need for such replacement plans feels donors should carry on. . Which steps are being taken to improve the situation? Nothing yet. Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performancc (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. Your overalljudgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2OO4 version 24 ffi *{ffi ffi ; t' l.ffi ffi,ffi ,ffi $'r$,,*e&iYl 9.1 Check whether staff at this level is skilled, knowledgeable and committed, regarding the of CDTI in its area of This indicator assesses whether the programme has been able to develop sufficient resources for itseff. Characterlstics of the indlcator Sources of lnformatlon The number of staff members in the government health service at this level should be appropriate to the task in hand: not too many or too few. Team members should have enough knowledge and skill to undertake all the key CDTI activities themselves, without help: * Planning * Report writing * Training and HSAM * Monitoring/supervision There should be evidence that the team is commifted to the success of the programme (from the evidence of the partners, as well as workers at the next level below: from wriften reoorts and timetables). * Data management * Computer skills * Mectizan ordering/ distribution lnspection of: * Staff files. * Training reports and timetables. lnterviews with * Managers and other staff at this level (government, NGDO etc.). * Staff at the next level below. lf the staff at this level lack skills and commitment: . Why is this? Staff are very committed. . Which steps are being taken to improve the situation? Describe the present situation: Number of pergg4gylq show that they are committed to their work, and perform it well : No. of persons qualified in this area . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement is this indicator for sustainability being achieved? Not applicable Sustainability evaluation instrument no.1 - national level - September 2004 version 25 9.2 Check whether staff at this level is stable, and whether provision is made for passing on CDTI skllls when a trained person moves away. This indicator assesses whether the programme has been are able to maintain ifs resources. Characteristics of the Indicator Sources of information a. Staff at this level should remain in one post for at least five years. b. There should be immediate orientation (in CDTI) of new, unskilled project staff members. c. During their training new professional health workers in the country (doctors, nurses etc.) are provided with up-to-date knowledge and skills in the field of CDTI. . lnspection of staff files. . The table in 9.1. . lnterviews with: * Managers and other staff at this level. * The person in the Ministry responsible for human resouroe develonmanl Describe the present situation: Staff are relatively stable and stay in a post for a period of upwards of 5-15 years. New staff are rarely oriented/ updated on the CDTI program. Describe the situation two years ago: Staff are relatively stable and stay in a post for a period of upwards of 5-15 years. Staff on ground then were oriented and exhibited reasonable knowledge of the CDTI approach. Describe the situation when APOC funding started being given: During inception of CDTI strategy, initial core staff were oriented on CDTI strategy and exhibited reasonable of its activity. lf the staff is not stable, and new staff is not being trained: . Why is this? . Which steps are being taken to improve the situation? Analvsis I \A/hen writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affec{ sustai nability. . What is the trend in the number and quality of staff? . Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I lloOcracly I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.1 - national level - September 2OO4 version 26 {$i.;-t, tis+sx,ll$lgs,g-qdrJffit}H'iFsr+"# It;lr;*,$iw& 10.1 Check from records whether all projects in the country/state/province have a satisfactory rate This indicator assesses whether the programme is effective - if the rates are poor the projects are clearly struggling and /ess sustain able. NOTE: The denominator in the calculation of GCR is the total number of projects in the country/state/provi nce Sources of information Characteristics of the indicator a. The GCR for all the projects in the country/state/province should approach 1 00% b. This GCR rate should be stable or increasing. . lnspection ot * Distribution reports and statistics at national/ project level, for the past 3 years. * National REMO records. . lnterviews with staff at:t This level. * The next level below. The GCR in the country/ state/province . At the last distribution: 100% . The year before: 100o/o . The year before that: 100o/o lf the GcR is poor, . Why is this? . Which steps are being taken to improve the situation? (if such steps are already being taken that is good for sustainability) Analysis When writing the report you have to summarise lhe reasons for poor performance (if any); steps beino taken lo imorove it: and how this is likelv to affec't sustainabilitv. . What is the trend in geographical coverage? Your overall iudoement: is this indicator of sustainabilitv beino achieved? Futy ({tCRfor0rcoors1tpdcerrbSt0tSclmnhD Highly (rate is 90-95% for country's nroieclsl Moderately (rate is 80- 8qo/^\ Slightly (rate isTO-79%l Negligibly (rate is <70%) Sustainability evaluation instrument no.1 - national level - September 2004 version I therapeutic coverage rate (TCR). I This indicator assesses whether the programme is effective - if the rates are poor the projects are clearly struggling, and /ess sustainable. NOTE: . The in theof Characteristics of the indicator Sources of information All projects in the country/state/province should have a TCR of 65% or higher. These rates should be stable or increasing. a. b. lnspection of: * Distribution reports and statistics at national/ project level, for the past 3 years. ' National REMO records. lnterviews with staff at: * This level. * The next level below. The TCR in the country/state/province: . At the last distribution: 2005 84.9o/o . The year before: 2004 63.8% There was this dip b/c funds were not released ontime for CDTI activities in one woreda . The year before that: 2003 75.1o/o lf the TCR is poor: r \A/hy is this? . Which steps are being taken to improve the situation? Analysis When writing the report you have to summarise the reasons for poor performance (if any); steps beinq taken to improve it: and how this is likelv to affect sustainabilitv. What is the trend in therapeutic coverage? Your overall iudoement: is this indicator of sustainabilitv beino achieved? Frrlu Glr. stlIp TCR brtr cou@r p{cctr b :EOlt - Cebb or lncmdnol Highly (the average TCR for the country's projects is 60-64%) Moderately (the TCR is 55-59%) Slightly (the TCR is 50-540/") Negligibly (the TCR is<5Ool") Sustainability evaluation instrument no.1 - national level - September 2004 version 1.. I lnstrument2: districU LGA level NOTE: . This instrument evaluates the level which actually takes responsibility for implementing CDTI in its area of operations. lt is that level where health services are planned and provided. We are going to refer to it as the districU LGA level. . The level below this one is the one where the health centres/ clinics/ dispensaries are located. We are going to refer to this level as the 'FLHF' (front line health facility) level. Geographical name of this districU LGA( Woreda): E - Researcher: Abbreviations/ acronyms CDD CDTI FLHF HSAM NGDO community directed distributor community directed treatment with ivermectin first line health facility health education, sensitisation, advocacy, mobilisation - i.e. activities which are aimed at gefting allthe key players to participate wholeheartedly in the programme non-governmental development organisation . The main function of this level is to take responsibility for the implementation of CDTI in its area of operation. . However its function is still largely one of support of the FLHF level: * Providing targeted training, HSAM and monitoring/ supervision. * Arranging for an adequate supply of Mectizan. . The FLHF level is the one that is finally responsible for working with the CDDs in the communities. Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version 1.1 Check whether the year plan for CDTI appears as part of an overall written plan for the activities of the dlstricu LGA. This indicator assesses whether the programme has become integnted into the health seruice, and whether management is beginning to accept ownership of the programme - both good for sustainability. Characteristics of the indicator Sources of informatlon a. CDTI should be integrated into the overall written plan (showing that staff at this level consider CDTI to be part of their yearly routine, like any other programme). b. The plan should make provision for all key activities: Mectizan supply; targeted training; targeted HSAM, targeted monitoring/ supervision. c. Year plans should be drawn up in a participatory way. d. Year plans must take into account community requirements for the timinq of distribution. . lnspection of: * The written year plans.r Minutes of planning meetings. . lnterviews with: * Staff at this level: managers and others (pharmacist, transport officer etc.). * Staff at FLHF level. Describe the present situation: Health plans 2004-2005 were available for most of the Woreda Health Dept, while some had no workplans. Most were integrated with CDTI activities. Plans are made in a participatory manner with the heads of various dept planning together. However there were no reports of planning meetings available. lf planning and implementation of CDTI is not part of the overall year plan: . Why is this? . Which steps are being taken to improve the situation? . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement is this indicator for sustainability being achieved? Not applicable Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version 3S'*fi 2.1 Check whether the various programme support activities are being planned and carried out in This indicator assesses whether the programme is functioning and in an integrated manner. Characteristics of the indicator Sources of lnformation a. Staff should provide o A written work plan o Plan shows how implementation of activities in an integrated manner will be achieved b. Staff combines two or more tasks on a single trip: * Monitoring / supervision for CDTI. * Training for CDTI. * HSAM. . Fetching records. * Delivering Mectizan. c. Staff combines CDTI activities with those of other programmes, where this is relevant (e.9. supervision, trainino. HSAM. Mectizan deliverv). . Examination of documents: trip authorisations, log books, trip reports etc. . lnterviews with: * Staff from this level (managers, administrators, drivers etc.). * Staff from the next level below. Describe the present situation: Work plans seen at the Woredas, are integrated showing how CDTI activities are implemented within the field (HSAM and training for CDTI). Plans were integrated alongside other health programmes.There were Log sheets but no reports. Describe the situation the year before: Same as above. lf integration between support activities is poor: . Why is this? . Which steps are being taken to improve the situation? Analvsis \A/hen writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Examine the trend in the integration of activities - is it becoming more common? . Your overalljudgement: is this indicator for sustainability being achieved? Fullv I f I Moderatelv I Sliqhtlv I Not at all I Not apolicable Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version 4B-'" 1 3.1 Check whether the districU LGA health management team is taking full responslbltlty for the implementation of CDTI at this level. This indicator assesses whether management is taking ownerchip of the programme Characteristics of the indlcator Sources of lnformation a. lt should be the management team at this level, and not higher levels/ NGDO leadership, which is initiating the key CDTI activities: planning, targeted monitoring/ supervision, targeted training and HSAM, Mectizan ordering/ distribution. b. There should be a focal person for CDTI activities. . lnspection of year plans. . lnterviews with: * Management team at this level. * Person responsible for CDTI at this level.i Staff at the project level * NGDO leadership. * Staff at FLHF level. Describe the present situation: Woreda Health management has a focal person responsible for Oncho and other health programmes in most of the Woredas visited. They are responsible for initiating CDTI activities which depends on the arrival of Mectizan. lf leadership at this level is not taking the initiative in implementing CDTI: . Why is this? . Which steps are being taken to improve the situation? Analvsis When writing the report you have to summarise:i The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affeci sustainability. Your overalljudgement: is this indicator for sustainability being achieved? Fully I f I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 2 - district/ LGA level - September 2004 version t'- 5 i - '&s' I system, and is functioning effectively. Characteristics of the indicator Sources of information a. The reporting process should take place within the government system, not using other resources. Data being transmitted includes: coverage reports; Mectizan statistics; training reports; distribution reports; fi nancial reoorts- . Examination of reports and report forms. . lnterviews with: * Managers and staff at this level. * Staff at the proiect level. Describe the present situation: Reports are transmitted entirely within the government system and comprises of treatment coverages,mectizan statistics from FLHF. These are moved to the next level which is the zone. Some reports were available. Government & APOC funds are used for transmitting these data. tf . Why is this? . Which steps are being taken to improve the situation? 4.1 Check whether routine data conceming CDTI activities are being transmitted from this level, within the This indicator assesses whether the programme has become more integnted into the govemment Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affecl sustainabilitv. . Your overall judgement: is this indicator for sustainability being achieved? Fully I f I Moderately I Slightly I Not at all I Not applicable i I l I l a- I Sustainability evaluation instrument no. 2 - district/ LGA level - September 2004 version 4.2 Check whether the responsible peraons at the districu LGA level are efflciently supervlslng CDTI activity at the FLHF !eve! ln an Inteqrated manner. This indicator assesses whether the programme is functioning efficiendy and in an integnted manner. Gharacteristics of the indicator Sources of informatlon a. Staff at this level should routinely only supervise the FLHF, and not the community level. 'Spot checks' may however be done from time to time. b. Although one routine supervision visit per FLHF per year is neoessary, supervision visits should focus more on FLHFs where there are proven problems - each supervision visit must be justified. . Examination of: * Supervisorychecklists, plans, itineraries and reports.r Visitofs books at FLHF level. . lnterviews with: * Staff at this level. * Staff at the FLHF level. Fr.r-"tt"a,gul.bunanrrb"r-- FomatEd: Bullets ard Numberi Describe the present situation: ln some cases,staff at this woreda level normally supervise FLHF, while some move lower down to the FLHF. Supervison was routinely done and several visits were carried out per year during distribution. They were targeted to needs such as poor record keeping in a few cases. There were no supervisory checklists nor reports in most of the Woredas. ln some Woreda's because of staff shortage they rely entirely on FLHFs to carry on monitoring, but they occasionally carried out spot checks to these FLHF. lf supervision is not being done in an integrated and efiicient manner: . Why is this? Occassionally, supervison is intergrated but there were no integrated checklists. . Which steps are being taken to improve the situation? Nothing Analvsis \A/hen writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affec{ sustainabi litv. . Your overalljudgement: is this indicator for sustainability being achieved? Fully I I I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version I 4.3 Check whether there ls a routine process of management of problems and successes, which are indicated by the This indicator assesses whether the programme is running efficiently and effectively, and whether management is beginning to accept ownerchip of the programme. Characteristics of the indicator Sources of information a. As soon as problems are identified as a result of supervision visits, or from coverage data (i.e. areas with low coverage) the appropriate manager should dealwith them. b. Where relevant such problems should be passed on to the appropriate FLHF staff to deal with, with the necessary support - thus empowering these persons. c. Successes should be noted and reported, and appropriate feedback given. d. There should be evidence of action taken based on recommendations in orevious monitorino exercises. . Examination of the following documents: * Year plans and annual reports. * Minutes of staff/ planning meetings.t Reports of previous monitoring exercises. * Letters of commendation. * Memos. . lnterviews with: * Staff at this level. * Staff at the FLHF level. Describe the present situation: Problems referred to the woreda level are managed accordingly particularly area in which low coverages is due to refusals because of the fear of side effects of the drug. There is a system in place for acknowledging successes. But there were no reports nor commendation letters seen. lf the system of managing problems/ successes is weak: . Why is this? . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affec{ sustainability. . Your overall judgement: is this indicator for sustainability being achieved? Fully I f I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 2 - districu LGA level - September 2004 version 8&l 5.1 Check whether sufficient Mectizan ls being ordered and recelved yearly, and ln good tlme. This indicator assesses whether the programme is functioning effectively. Characteristlcs of the lndicator Sources of information a. The order forms for the districU LGA exist, and should be based on FLHF and community requests. b. The Mectizan should be available at this level in time for distribution at the time requested by the communities. c. There should be no reports of shortages and/ or late supply. lf there have been shortages, there should be specific plans to remedv lham a a Examination of all Mectizan ordering and stock control documentation at this level. lnterviews with staff at this level (managers and pharmacist). Describe the situation with Mectizan supply at the last distribution: At the Woreda levels order forms fro FLHF were seen. Mectizan was available, sufficient and and ontime. These were stored at the Woreda Health stores. However timing for distribution was not dependent on community. Describe the situation the previous year: Same as above Describe the situation the year before that: Same as above lf there are problems with obtaining the Mectizan that is required: . Why is this? . Which steps are being taken to improve the situation? . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . What is the trend in Mectizan supply at this level? Your overalljudgement is this indicator for sustainability being Sustainability evaluation instrument no. 2 - district/ LGA level - September 2004 version 5.2 Check whether Mectizan is being collected, stored and effectlvely dellvered withln the at this level. This indicator assesses whetherthe programme isfunctioning efficiently, lfsprocesses are simple, and it is becoming more integrated into the govemment system. Characteristics of the indicator Sources of information a. The Mectizan should be controlled within a govemment system. This does p! have to be the system routinely used for the supply of other drugs. b. The system should be effective, uncomplicated and efiicient. c. The districU LGA should ideally fetch its Mectizan from the project level itself (although use of the routine MoH drug supply system to districts is acceptable). ln either case transport should be supplied and paid for by government at this level. . Examination of: * All Mectizan ordering and stock control documentation at this level. * Vehicle log books and/ or trip authority forms. . lnterviews with staff at this level (managers, pharmacist, drivers). Bullets and Describe the present situation: Mectizan is collected from the zone, while the FLHF collects from the woreda. Funds are released from a pooled financial account at woreda level for collection of mectizan at zonal level. Mectizan collected is stored at this level before distribution. lts entire movement is within the government system. lf the Mectizan supply is not being administered within a government system: . Why is this? . Which steps are being taken to improve the situation? . When writing the report you have to summarise: + The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement: is this indicator for sustainability being achieved? Not applicable Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version l0 *rlr.,& aq'Hii4*t&W&" This indicator assesses whether the programme is functioning Characteristics of the lndlcator Sources of informatlon a. Staff should routinely only train staff at the FLHF level, and not at the community level. b. Staff should have empowered staff at the FLHF level to see to their own training needs as much as possible, and to conduct training activities at the communi$ level independently. c. Wherever possible staff at this level should conduct their own training for CDTI, if they have need for such trainino. . Examination of training materials, plans/ programmes, reports: * At this level. * At the FLHF level. . lnterviews with: * Staff at this level (the trainers). * Staff at FLHF level (the trainees). * CDDs. Describe the present situation: Woreda train FLHF staff who inturn train CDDs. They train without support form zone, but occasionally train with Zonal pa(icularly when they are short staffed. Training was routine and not directed to ares of deficiency. Describe the situation the year before: Same as above lf staff members are not being used appropriately: . Why is this? . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Examine the trend in the way in which staff are being used as trainers: . Your overall iudoement is this indicator for sustainabilitv beinq achieved? Fully I f I Moderately I Slightly i ttot at att i Not applicable Sustainability evaluation instrument no. 2 - districu LGA level - September 2004 version l1 il ._t 6.2 Check whether training is being planned and carried out in an efficient and integrated manner. This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information a. There should be an objective need for each episode of training. This means there should be evidence that staff to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Repeat training of already skilled staff should not happen. b. Resources for training (human, transport etc.) should be efliciently used: * Using as few staff members as possible. * Using as little time as possible (without sacrificing quality) * Choosing the most cost-effective site etc. Examination of training materials, plans/ programmes, reports: * At this level. * At the FLHF level. lnterviews with: * Staff at this level (the trainers). * Staff at FLHF level (the trainees). * CDDS. Describe the present situation: Training was done routinely as a refresher training yearly. Training was integrated with other health programmes such as malaria. Training was carried out by 2-3 Woreda Health staff for 2-3 days at the FLHF. There were some training reports available. Describe the situation the year before: Same as before lftr@ . Why is this? . Which steps are being taken to improve the situation? Analysis a When writing the report you have to summarise:r The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. . Examine the trend in training activities - is it becoming more efficient and integrated? Your overalljudgement: is this indicator for sustainability being achieved? Fullv I I I Moderatelv I Sliohtlv I Not at all I Not applicable Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version t2 @D6.3 Check whether staff at this leve! ls planning and carHeadsrying out HSAM activities in an efficient manner. This indicator assesses whether the programme is functioning efiicien{y and effectively, and whether managers are taking ownerchip of the programme. Characteristics of the lndicator Sources of information a. Staff members identiff situations where decision makers lack information abouU commitment to CDTI, and undertake activities to inform and persuade these persons. b. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. c. Such activities should only be carried out at this districU LGA level, and at times at the FLHF level (but only when staff at that level asks for help). d. There is evidence that these HSAM activities have been effec'tive and have led to action. . Examination of:r HSAM plans/ programmes and reports.t Year plans and annual reports. * Minutes of planning meetings. . lnterviews with: * Staff (programme and management) at this level. * Civil authorities at this level. + Staff and civil authorities at the FLHF level. Sustainability evaluation instrument no. 2 - districi/ LGA level - September 2004 version Describe the present situation: Planned advocacy were carried out to Heads of the Woreda administration as well as to areas where needed. lmproved covergaes were subsequently recorded at some Woreda, thus exhibiting uccessful HSAM activity. Woreda Task forces for onch were also involved withHSAM activities. Describe the situation the year before: Same as above. lf HSAM activities are not being canied out efiiciently and effectively: . Why is this? . \A/hich steps are being taken to improve the situation? Analysis \A/hen writing the report you have to summarise: * The evidence about how well this indicator is being achieved.i Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Examine the trend in HSAM activities - is it becoming more efficient? . Your overall judgement: is this indicator for sustainability being achieved? Fully I I I Moderately I Slightly I Not at all 1 Not applicable l3 &rt This indicator assesses whether the programme is functioning Characteristics of the indicator Sources of information a. The costs for each CDTI activity in the year plan at this level should be clearly spelt out in a budget. There is evidence of a cost reduction/ containment strategy (e.9. targeted training, HSAM and monitoring/ supervision; training conducted at FLHF level etc). Managers at this level should have a clear estimate of the funds that will be available to them for CDTI in the coming year, and should be able to justiff this belief. The total amount budgeted for in the year plan should fallwithin this esiimated income. d. . Examination of the budget documents. . lnterviews with health service and local government managers at this level: * Technical managers. * Treasurer/ administrator. The budget and estimated income: . At the previous distribution: Government does not schedule any money for CDTI; but APOC funds are pooled with other health programme funds and used jointly for Health. . For the previous year: Same as above . Forthe year before that: Same as above budgeting has been inappropriate: . Why is this? Staff are unaware of funds released specifically for oncho. Finance and economic planning development office of the woreda, say they are unaware that Government is suppose to fund CDTI activities. . Which steps are being taken to improve the situation? Analysis a When writing the report you have to summarise: * The evidence about how well this indicator is being achieved.t Reasons for poor performance (if any); steps being taken to improve it; and how this is likety to affecl sustai nabilitv. . Examine the trend in the budgeted amount and the expected income: Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I Moderately I I I Not at all I Not applicable Sustainability evaluation instrument no. 2 - district/ LGA level - September 2004 version t4 @D7.2 Check whether the govemment at thls level le budgetlng and disburclng incrcasing amounts for CDTI yearly. and in good time. This indicator assesses whetherthe programme is becoming integnted, and whether govemment is beginning to accept ownership of the programme and can mobilise fl,e ,€sources it needs. Characteristlcs of the indicator Sources of infomation a. The relative budgetary contributions of the local government and other partners to CDTI should be clearly spelt out. b. The amount that the government has budgeted in one or more specific CDTI budget lines should be increasing yearly. By the end of Year 5 of APOC funding the bulk of CDTI expenses at this level should be met from local government funds; by the end of Year 3 at least half of it. c. The amounts actuallv disbursed from such budget lines should be increasing yearly, as a proportion of total expenses. (Note that ac{ual disbursement is more important than budgeting, and is a real sign of political commitment). . Examination of: + Budget documents (government and NGDO) * Records of disbursement and expenditure (ledgers, orders, approvals for expenditure etc.) . lnterviews with health service, local government and NGDO managers at this level: * Technical managers.t Treasurer/administrator. The budget and disbursements: . For this year: No funds are specifically released by Government for CDTI activity in most Woredas while in some woredas staff are unaware of whom releases funds for CDTI activity. However, from the pooled source for Health some funds are often allocated for CDTI activities yearly. . For the previous year: Same as above the government proportion of expenditure is not increasing proportionately: . Which steps are being taken to improve the situation? Analysis r When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. + Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affeci sustainabilitv- . Examine the trend in government budgeting and disbursements: . Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately I f I Not at all I Not applicable Sustainability evaluation instrument no. 2 - district/ LGA level - September 2004 version l5 7.3 Check whether in case of a deficit between estimated costs and the amount provided by the is beino made to meet it. This indicator assesses whether management is able to mobilise the resources it needs, as wel/ as ifs commitment to ownerchip. Characteristics of the indicator Sources of information a. lf there is a shortfall the management should have specific and realistic plans to bridge it. b. lf it is planned that non-government sources offunding are to be used after APOC funding ends, written commitment for this should have been obtained at the highest level in these donor oroanisations. lnspection ot * The budget documents (government and NGDO) * Records of expenditure (ledgers, orders, approvals for expenditure etc.). * Letters of agreement. lnterviews with health service and local government managers at this level: * Technical managers. * Treasurer/administrator. Describe the present situation: Most Woreda staff are unaware of any shortfalls. ln all Woredas visited there are no definite plans to bridge the gap. However, there is an integrated manner of allocating funds from a pooled source of funding available at the Woreda level. ln some cases, staff levy themselves to be able to carry out some health activities such as monitoring and supervision. Describe the situation the previous year: lf the shortfall cannot be met: . Why is this? Government is unaware that they are to allocate funds into the pooled accounts for Oncho. However government does pay salaries. There are no written commitments . Which steps are being taken to improve the situation? Analvsis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabi litv. Examine the trend in shortfall and how it is to be supplemented: Your overalljudgement is this indicator for sustainability being achieved? Fully I Highly I Moderately I f I Not at all ; Not applicable Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version l6 7.4 Check whether funds disbursed for CDTI from the budget at this level are efficiently This indicator assesses whether the programme is functioning efliciently. @s Gharacteristlcs of the indicator Sources of information a. The budget holder should be using a control system with the following elements:t Approval of each item of expenditure. * Allocation of expenditure against specific budget headings. * Regular calculation of residual amounts under budget headings. b. All the funds released yearly should be spent as budoeted. . lnspection of: * The budget documents (government and NGDO). * Financial control records (ledgers, orders, approvals for expenditure etc.). . lnterviews with health service and local government managers at this level: * Technical managers.t Treasurer/administrator. Describe the present situation: . Approval of expenditure: Approvals for APOC funds disbursed for CDTI activities were available in all woredas visited. . Allocation of expenditure: No funds were allocated for oncho by the Government. . Regular insight into budget line balances: Oncho has no budget line. lf the funds are not being well managed: . Why is this? . Which steps are being taken to improve the situation? . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement: is this indicator for sustainability being achieved? Sustainability evaluation instrument no. 2 - district/ LGA level - September 2004 version t7 l,wgi,$ @rrng8.1 Check whether adequate and appropriate transport and materials are available for CDTI activities at this level. This indicator assesses whether the programme is mobilise the resources it needs. Characteristics of the indicator Sources of information a. There are adequate numbers of appropriate, functional vehicles available for necessary CDTI activities. b. The running costs for these vehicles are met by the government. c. There are sufficient materials available for training and HSAM. . lnspection of: * Each vehicle in the pool: its source; its functional status. * Stocks of materials for training and HSAM. . lnterviews with managers at this level: transoort offlcers. orooramme manaoers. Describe the availability/ suitability/ functionality of the present vehicles and materials, considering the work still to be done in the coming 5-10 years: The terrain is most unsuitable for available motor bikes, training materials and posters are highly inadequate considering the amount of work ahead for 5-10yrs. * APOC, MoH, NGDO, other (specify) ** ls it working? ls there enough of it for the job? ls it suitable lf transport and materials are inadequate and funded from non-government sources: . Why is this? Government is unaware. . \A/hich steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affecl sustainabilitv. . Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately I I I Not at all I Not applicable Sustainability evaluation instrument no. 2 - districU LGA level - September 20(X version l8 I This indicator assesses whether the programme is functioning effectively and efficiently. Characteristlcs of the lndicator Sources of Informatlon a. There is a routine maintenance schedule for each vehicle, which is adhered to and recorded. This includes weekly driver maintenance, scheduled garage servicing, and replacement of worn tyres. b. The costs for vehicle and equipment maintenance and repair are met by dependable sources (e.9. the government). c. Repairs are rapidly and efficiently done. d. Staff have ways of coping when vehicles break down or are not available. so that CDTI activities are not disrupted. . lnspection of: * Vehicle and equipment maintenance schedules.i Vehicle and equipment maintenance records. . lnterviews with managers at this level: transport officers, programme managers. Formatted: Bullets and Numbe D_ Fi Describe the present situation: There was no regular repair or maintaince of vehicles, neither were there any schedules for maintaince. Government does not release any funds for repairs. Staff have a coping mechanism of always findi transport for activities. Make particular enquiries about the ability of the government to pay for maintenance, repairs and tyre replacement: lf the vehicles are not being well maintained, and/ or the government is not paying: . Why is this? . Which steps are being taken to improve the situation? . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve lt; and how this is likely to . Your overall judgement is this indicator for Not applicable Sustainability evaluation instrument no. 2 - district/ LGA level - September 2004 version l9 r lrrl 8.3 Check whether the transport available at thls Ievel is appropriately managed and used, in an This indicator assesses whether the programme is functioning efficiently. Characteristics of the indicator Sources of information a. Transport is used at this level, and to undertake suppott activities at the FLHF level. lt should !E! be used for CDTI implementation activities at the community level. b. Trips made for CDTI purposes should be properly authorised in writing by the relevant official. Each trip undertaken should be recorded in a log book. c. Transport provided for CDTI, and that provided for other programmes, should be combined as a pool to be used for leoitimate activilies of all oroorammes at this level. . lnspection of vehicle control documents: * Copies of trip authorities (also noting destination and purpose) * Log books. * The supervision plan/ matrix. . lnterviews with managers at this level: transport officers, programme Describe the present situation: Transport moves up to the FLHF, and requests are made with approvals. Occasionally, they get to comunies. Log books are available. Transport are utilized in an integrated manner. lf the transport is not being well managed: . Why is this? . Which steps are being taken to improve the situation? Analvsis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. Your overall judgement: is this indicator for sustainability being achieved? Fully I f I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version 20 8.4 Check whether there are approprlate and realistic plans for the rcplacement of transport and materlals when APOC auoDort comes to an end. This indicator assesses whether the programme managers are taking ownership of the programme, and are able to find resources for it. Gharacteristlcs of the indicator Sources of lnformatlon a. Management should know that replacements will be needed before the end of the programme, and have specific, realistic plans to meet the need at that time. b. lt should be planned that the government will: * Provide replacements for vehicles. * Maintain existing vehicles. * Provide stationery and materials for lraining and HSAM. c. lf it is planned that replacement will be ftom non-government sources, written commitment for this should have been obtained at the highest level in these donor organisations (end of Year 5), or neootiations should have staried (end of Year 3). . lnspection of letters of agreement. . lnterviews with:i Programme managers at this level: administrators, technical managers. * NGDO project managers. * High-ranking local qovernment officials. J Describe the present situation: Woreda has no plans for replacement of vehicles within their environment and is unaware if NGDO will support them. However, Government maintains the vehicles as much as they can. There are no realistic plans of yet for replacement. However Government provides some stationary lf the plans for replacing vehicles and materials are unsatisfactory: . Why is this? . Which steps are being taken to improve the situation? lncreased advocacy to policy makers for assistance. . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement is this indicator for sustainability being achieved? Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version rl lr 2t ffi*S .'g 9.1 Check whether staff at this level ls skilled and knowledgeable, regardlng the of CDTI in its area of This indicator assesses whetherthe programme has been able to develop sufficient resources for itself. Characteristics of the indicator Sources of information a. Staff should have enough knowledge and skill to undertake all the key CDTI activities themselves: planning, training, HSAM, ensuring Mectizan supply, monitoring/ supervision. b. Staff at this level should remain in one post for at least five years. c. There should be immediate training (in CDTI) of new, unskilled project staff members who have CDTI responsibilities. lnspection of: * Staff files. * Monitoring reports.t Activity reports. lnterviews with * Managers and other staff at this level. * Staff at the project level * Staff at the FLHF level. Bullets and Numbering Describe the present situation: . Particulars of cunent staff No. of persons qualified in this area . lnformation about staff stability Most of staff are in the office for over 25 years. . lnformation about in-service training: They have had series of in service training on courses such as EPl, lf the staff at this level lack skills, and are often transfened: . Why is this? . Which steps are being taken to improve the situation? When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall iudoement is this indicator for Sustainability evaluation instrument no. 2 - districf/ LGA level - September 2004 version 22 9.2 Check whether staff memberc at thls level are commifted to thelr CDTI work. This indicator assesses whether the programme has been able to develop suffrcient resources for itself. Characterlstlcs of the Indlcator Sources of informatlon a. Staff members express satisfac{ion with their present responsibilities b. There is evidence of specific motivational practices and rewards within the programme: awards, financial incentives, compensation in cash or kind. c. Salaries/ wages and allowances are paid regularly. d. Staff members mention non-financial rewards inherent in CDTIwork. e. There is evidence from partners and workers at the FLHF level that staff members are committed to their CDTI work. . lnspection of: * Staff files (for performance records and awards). * Documentation about incentive schemes. . lnterviews withi Managers and other staff at this level. * Local government officials. * NGDO p@ect staff. Staff at FLHF level. r Describe the present situation: Staff are satisfied with their job salaries are paid regularly. Members of staff are committed to their work. Occasionally they given certificates of commendation for outstanding performance lf staff members appear to have little commitment to . Why is this? . Which steps are being taken to improve the situation? Analysis . 'When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affec't sustainabilitv. . Your overall judgement is this indicator for sustainability being achieved? Fully I f I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 2 - district/ LGA level - September 2004 version 23 l ed rudl'#r4'W$ffiS'&"siiY#'sJi'"llr.+4*s-Wffi#& 10.1 Check from records whether the districts/ LGAs being evaluated have a satisfactory rate This indicator assesses whether the programme is eflective - if the rates are poor the project is cleafly struggling, and /ess sustainable. NOTE: . We only investigate the GCR of the districts/ LGAs that have been sampled. . The denominator in the calculation of GCR is the number of FLHFs in the districts/LGAs The GCR in the districU LGA: . At the last distribution: All woredas had a GCR ot lOOo/o. . The year before: Allworedas had a GCR of 100%. . The year before that: Allworedas had a GCR of '100%. Analysis . When writing the report you have to summarise the reasons for poor performance (if any); steps beino laken to imorove it and how this is likelv to affec{ sustainabilitv. . What is the trend in geographical coverage? . Your overall iudqement: is this indicator of sustainabilitv beinq achieved?I Highly (GCR is 90-95%for districts/ LGAssamoled) Moderately (GCRis 80-89%) Slightly (GCRis 70-79%) Negligibly(GCR is<7io/^\ Bullets and Numbennq Characteristics of the indicator Sources of information a. The GCR for the districts/ LGAs sampled should approach 100o/o b. This GCR should be stable or increasing.. . lnspection of: * Distribution reports and statistics at districu LGA level, for the past 3 years. * REMO list of endemic communities for this districU LGA. . lnterviews with: * Staff at districU LGA level. * Staff al FLHF level. Sustainability evaluation instrument no. 2 - districU LGA level - September 2004 version 24 10.2 Check whether the dlstrictsl LGAs being evaluated have a satisfactory therapeutic rate This indicator assesses whether the programme is effective - if the rate is poor the project is clearly struggling, and /ess sustainable. NOTE: . We only investigate the TCR of the districts/ LGAs that have been sampled. . The denominator for calculating the TCR is the number of FLHFs in the districts/LGAs. Characteristics of the indicator Sources of information a. The TCR for the districts/ LGAs sampled should be 65% or higher. These rates should be stable or increasing. . lnspection ot * Distribution reports and statistics at districU LGA level, for the past 3 years. * REMO list of endemic communities for this districU LGA. . lnterviews with: * Staff at districU LGA level. * Staff at FLHF level. Findinqs The TCR in the districts/ LGAs: . At the last distribution: { Overall average TCRs for FLHF visited was 73.4o/ol lf the TCR is poor: . Why is this? Which steps are being taken to improve the situation? Analvsis . When writing the report you have to summarise the reasons for poor performance (if any); steps beins taken to improve it and how this is likely to affect sustainability. . What is the trend in therapeutic coverage? . Your overall iudoement: is this indicator of sustainabilitv beino achieved? - Highly (the TCR for the districtsr/ LGAs is 60-64%) Moderately (the TCR is s5-59%) Slightly (the TGR is 50- 54o/o\ Negligibly (the TGR is<50%) Sustainability evaluation instrument no. 2 - districU LCrA level - September 2004 version I ' i lnstrument 3: first line health facility (FLHF) level NOTE: . This instrument evaluates the level which finally interacts with the villages and communities, in ensuring that CDTI takes place in all the communities in its area of operations. This level has different names in different countries. ln most countries there is a health centre - a clinic, or health centre, or dispensary, which we call a 'first line health facility'. The health workers who work there are the ones who are responsible for training and supporting the CDDs in the villages. . When we speak of a FLHF we therefore mean: ' * The health facility and its staff. * The accompanying political/ administrative mechanisms between the districU LGA and community levels. . , . By'FLHF team' is meant the group of persons working in the firshline health facility and in its catchment area.I . '' . The level below this one is of course the community, the villages. Here the CDDs - 'community directed distributors' - live and work. We refer to this level as the 'community' level. . This level is extremely variable. ln some places FLHFs are large, wellorganised, with their own budgets and transport. ln others they hardly function at all and have no resources. The focus of this level's activities in CDTI The main function of this level is to work with the village communiti*, so that CDTI is established in them: ' Mobilising them to become involved in CDTI, by selecting CDDs. . Training the CDDs and supporting them in their work. . Arranging a dependable supply of Mectizan for them, at the right time each year. a inq them to collect and fonlrard the coveraqe data for their communitv. Project: Date: - Abbreviations/ acronymsCDD community directed distributorCDTI community directed treatment with ivermectinFLHF first line health facility programmeNGDO non-governmentaldevelopment organisation Sustainability evaluation instrument no.3 - FLHF level - September 2004 version 2fi.'1." :i$ffi$,S$rl 1.',| Check if there is a written for CDTI in the FLHF area. This indicator assesses whetherthe programme is being planned in an efiective and integnted manner, and whether management is beginning to accept ownerchip of the programme. Characterlstlcs of the lndicator Sources of information a. There should be a written plan or timetable in existence, for the most recent round of CDTI (this is recommended). b. ldeally the plan should be integrated into the overall year plan for the FLHF area. c. ldeally CDTI should form part of the 'minimum' or 'recommended' oackaoe for this level . lnspection of: * The wriften year plans. * Minutes of planning meetings. . lnterviews with staff at this level. Describe the present situation: There were no specific health plans of CDTI activity at most of FLHF visited. CDTI plans are normally made at woreda level and handed over to them at the FLHF, while in a few FLHF an integrated health schedule was pasted on the wall. CDTI activity mainly included was the distribution of Mectizan lf there is no written plan for CDTI: . Why is this? Unaware of the importance of having a written plan for CDTI activities nor of health ac{ivity at this level . Are any steps being taken to improve the situation? Have promised to improve in the future Analvsis a When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overalljudgement: is this indicator for sustainability being achieved? Fully I Highly I - | Slightly I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version 2.1 Check whether the various programme support activities are being planned and carried out in an anteqrated manner. This indicator assesses whether the programme is functioning efficienily and in an integrated manner. Characteristics of the indicator Sources of information a. Staff combines two or more tasks on a single trip: * Monitoring / supervision for CDTI. * Training for CDTI. * HSAM. * Fetching records. * Delivering Mectizan. b. Staff combines CDTI activities with those of other programmes, where this is relevant (e.9. supervision, training, HSAM, Mectizan daliverv) . Examination of documents: trip authorisations, log books, trip reports etc. . lnterviews with: * Staff from this level * Staff from the next level below. Describe the present situation: Staff at this level combine two or more CDTI activities while in the field ( such a HSAM and training, delvery of mectizan and fetching of reports back for the woredas) Virtually, all staff at the FLHF are involved with CDTI activities and other health programmes such as EPl, Malaria, HIV/AlDs, handling of acute cases in the clincs such as diahhoreas At all FLHF visited there were some reports such as mectizan and the community, treatment summaries at FLHF. Describe the situation the year before: Same as previous years. lf integration between support activities is poor: . Why is this? . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect susiainabilitv- . Examine the trend in the inteqration of activities - is it becominq more common? . Your overall judgement is this indicator for sustainability being achieved? Fully I I I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2OO4 version el,i"H ,$:e.St:Jffi# 3.1 Check whether the FLHF management team is taking ful! responsibility for GDTI at this level, in an integrated manner. This indicator assesses whetherthe CDTI project is integnted into the health system, and whether management is beginning to accept ownerchip of the programme. Gharacteristics of the indicator Sources of information a. The FLHF management team and all health staff at this level consider the program as theirs and are initiating the key CDTI activities: planning, monitoring/ supervision, training, HSAM, Mectizan ordering/ distribution. b. The political head/ senior politician at this level should know about CDTI and have participated in some CDTI activities. . lnspection of year plans. . lnterviews with: * Management team at this level. * Senior politicalfigures at this level. * Staff at the districU LGA level. Describe the present situation: Management team of the FLHF are taking charge of CDTI activities, by involving a Oncho Task force comprising individuals form the communities and their kebeles and some of the FLHF staff. However, initiation of CDTI activity for this level heavily relies on the Woreda level. lf the FLHF management team is not taking full responsibility for CDTI: . Why is this? . Are any steps being taken to improve the situation? . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overalljudgement is this indicator for sustainability being achieved? Sustainability evaluation instrument no.3 - FLHF level - September 2004 version 4.'l Check whether routine and necessary data concerning CDTI activities at this Ievel are transmitted entirclv wlthin the This indicator assesses whether the programme is becoming more integrated into the national health system. Gharacteristics of the indicator Sources of information a. The reporting process should be integrated within the government system, not using other resources. Data being transmitted includes: coverage reports; distribution reports; Mectizan statistics; training reports. . Examination of reports and report forms. . lnterviews with: * Managers and staff at this level. * Staff at the districU LGA level. Describe the present situation: Data from community registers are extracted by FLHF staff as FLHF summaries ( comprising of coverage reports, distribution report, training reports ) These are transmitted onto the Woreda level within the government system. . Why is this? . Are any steps being taken to improve the situation? . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall iudoement: is this indicator for sustainabilitv beinq achieved? Sustainability evaluation instrument no.3 - FLHF level - September 2004 version 4.2 Check whether health service staff at thls level Is routlnely and efficiently supervising CDT! at the communltles on slte in an lnteoated manner. This indicator assesses whether the CDTI programme is being implemented G-t"tt"d, Brll"tr -d Nrrb, ,, Characteristlcs of the indicator Sources of lnformatlon a. Afthough one routine supervision visit per community per year is necessary, supervision visits should focus more on communities where there are proven problems - each supervision visit must be justified. b. During visits to communities FLHF staff should turn their attention to as many health related programmes and oroblems as oossible. . Examination of: * Supervisory checklists, plans, itineraries and reports.t Log books. . lnterviews with: * Staff at this level. * Villaoe heads and CDDs. Describe the present situation: Monitoring and supervision is routinely carried out more than once during mectizan distribution per community, in some communities supervisory visits were sparingly carried out because of the issue of perdiem. lntegrated M & S visits are carried out at almost all FLHF staff. Supervisory visits did not pay attention to proven problems. Supervisory check lists were not available in some communities nor were there supervisory reports or itineraies. lf health service staff members are not routinely and efficiently supervising CDTI: . Why is this? Some FLHF were disgruntled by their no receiving per diems. . Are any steps being taken to improve the situation? Analvsis a When writing the report you have to summarise:t The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overalljudgement is this indicator for sustainability being achieved? Fully I f I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version tl I .- 4.3 Check whether there ls a routine proceas of management of problems and successes, which are Indicated by the monitoring system (coverage data, visits and This indicator assesses whetherthe programme is being implemented efiicienily and effectively, and whether management is beginning to accept ownerchip of the programme. Characteristics of the indicator Sources of information a. As soon as problems are identified through supervisory visits, coverage data etc. (e.9. communities with low coverage) health staff at this level should deal with them. b. Where relevant such problems should be passed on to the relevant communig to deal with, with the necessary support - thus empowering communities to make decisions on CDTI and cope with problems. c. Successes should be noted and reported, and appropriate feedback given to communities. d. There should be evidence of action taken based on recommendations in orevious monilorino exercises. . Examination of the following documents: * Year plans and annual reports. * Minutes of staff/ planning meetings. * Reports of previous monitoring exercises. * Letters of commendation. . lnterviews with: * Staff at this level: in-charge and others. * Community leaders and CDDs. Describe the present situation: Problems identified by CDDs are often referred to the FLHF to handle. While in some communities problems such as refusals due to fear of side effect, replacement of CDDs, support of CDDs were handled by FLHF staff in conjuction with the local Oncho task force. Most of the problems are referred to the appriopriate level to handle in most cases. Successes are commended by certification but documentation were not seen. lf there is no routine process of managing problems and successes: . Why is this? . Are any steps being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. . Your overalljudgement: is this indicator for sustainability being achieved? Fully I I I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2OO4 version 8air,g 5.1 Check whether sufflcient Mectizan is being ordered annually, and ln good tlme. This indicator assesses whether the programme is functioning effectively. Gharacterlstics of the Indicator Sources of infomatlon a. The order forms for the FLHF area exist, and orders should be based on the requests from the community. b. The Mectizan should be available at this level in time for distribution at the time requested by the communities. c. There should be no reports of shortages and/or late supply. lf there have been shortages, there should be specific plans to remedv them. . Examination of all Mectizan ordering and stock control documentation at this level. . lnterviews with: * Staff at this level (managers and pharmacist). * Villaoe leaders and CDDs. What happened at: . The last round of treatment? There were mectizan summary and request booklets with inventory of mectizan received, disbursed and returned at all the FLHF visited Mectizan supply is sufficient but timing is not in accordance with community request. There were no shortages. . The round ofthe year before? Same as above. . The round the year before that? Same as above. . Why is this? . Are any steps being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (lf any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . What is the trend in Mectizan ordering and supply? . Your overalljudgement: is this indicator for sustainability being achieved? Fully I I I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version rr 5.2 Check whether Mectizan is being collected, stored and effectively delivered within the at this level. This indicator assesses whether the programme is functioning efficien{y, ifs processes are simple, and it is becoming more integrated into the govemment system. Characteristics of the indicator Sources of information a. The Mectizan should be controlled within a government system. This does ry! have to be the system routinely used for the supply of other drugs. b. The system should be effective, uncomplicated and efficient c. The FLHF level should fetch its Mectizan from the districU LGA level itsell by means of transport supplied and paid for by government at this level. d. Communities should fetch their Mectizan from the FLHF themselves. However in situations where villages are very far from health centres, or where it is easy/ practicable for FLHF staff to deliver the Mectizan. thev mav helo with the deliverv. . Examination of: * All Mectizan ordering and stock control documentation at this level. * Vehicle log books and/ or trip authority forms. . lnterviews with staff at this level (managers, pharmacist, drivers). Describe the present situation: Mectizan is transmitted within a government system similar other drugs within the FLHF. Drugs are signed for at the Woreda level by FLHF staff and stored in a drug store at the FLHF level. Government provides funds for FLHF staff to collect mectizan from the Woreda level. Communities pick their Mectizan from the FLHF. . Why is this? . Are any steps being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overall judgemenl: is this indicator for sustainability being achieved? I I Highly I Moderately I Slightly I Not at all I Not applicable \ Sustainability evaluation instrument no.3 - FLHF level - September 2004 version l0 *:'' fx*aa&ffi 6.1 Check whether traininq is and carried out in an efflcient manner. This indicator assesses whether the programme is functioning efficienily. Characteristics of the indicator Sources of infomation a. There should be an objective need for each episode of training. This means there should be evidence that CDDs to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. b. Resources for training (human, transport etc.) should be efficiently used: * Using as few staff members as possible. * Using as little time as possible (without sacrificing quality) * Choosing the most cost-effective site etc. Examination of training materials, plans/ programmes, reports at this level. lnterviews with: * Staff at this level (the trainers). * CDDs (the trainees). Fomattedt Bullets @s Describe the present situation: Routine training is carried out yearly for 2-3 days and 2 FLHF staff are involved. Training was carried out within the communities and was not focused on deficiencies. Lumped training is carried out for both new and old CDDS. Training manuals were seen in some FLHF. Describe the situation the year before: Same as above. lf training is not efficiently done in an integrated manner: . Why is this? Training is not routinely done in an integrated because the training is routinely done for CDTI aclivities only. . Are any steps being taken to improve the situation? None yet. Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustai nabilitv. . What is the trend in the way training is done - the method and content? . Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I f I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version ll 6.2 Check whether staff at this level is planning and carrying out HSAM activities in an efficient manner. This indicator assesses whether the programme is functioning efiiciently and effectively, and whether managers are taking ownerchip of the programme. Characteristics of the indicator Sources of information a. Staff members identify situations where decision makers lack information abouU commitment to CDTI, and undertake activities to inform and persuade these persons. b. HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine. c. There is evidence that these HSAM activities have been effective and have led to action. . Examination of: * HSAM plans/ programmes and reports. * Year plans and annual reports. . lnterviews with: * Staff at this level. * Civil authorities at this level. * Community leaders and CDDs. Fi Describe the present situation: HSAM is carried out during community meetings prior to distribution. HSAM is routinely carried out in some communities. lt is not targeted. While in some HSAM is carried out by the kebele task force and these units identify areas that need HSAM. There were no HSAM plans nor reports. Despite the fact that community awareness and demand for the drug has increased over the year, community support has not yet changed. Describe the situation the year before: Same as above. lf staff is not effectively engaged in HSAM: . Why is this? Approach of FLHF staff to CDTI is so because some FLHF feel their money has been diverted. . Are any steps being taken lo improve the situation? Adequate HSAM should be carried out to FLHF staff to change their set views to CDTI. Analvsis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it and how this is likely to affect sustainabilitu. . Examine the trend in HSAM activities - is it becoming more efficient? . Your overall judgement: is this indicator for sustainability being achieved? Fully Highly tf I Slightly I Notatall rNotapplicable Sustainability evaluation instrumenl no.3 - FLHF level - September 2004 version t2 ?,ti;il{ r" 7.'l Check whether the costs lnvolved for each planned CDTI activity at this level in the year are clearly spelt out ln a wlth those for other health care activltles. This indicator assesses whether the programme is functioning efficienily, and whether CDTI is becoming integnted into the budgeting process. NOTE: . ln some situations FLHFs do not make or hold budgets. Characterlstics of the indicator Sources of informatlon a. lf there is a budget at this level, the costs for each CDTI related aclivity in the year plan should be clearly spelt out in it. These activities include monitoring/supervision, training, HSAM, and arranging Mectizan supply. b. The staff should be able to justify the amount they plan to use. There should be evidence of cost containment (e.g. by targeting training, HMAS and supervision). . Examination of the budget documents (government and NGDO). . lnterviews with:t FLHF team managers (leader, treasurer). * Local government managers at this level (chairperson, administrator, treasurer). What happened at: . The last round of treatment? Budgetary plans for CDTI nor other health programmes are not made from this level therefore the issue of budget containment does not arise. . The round ofthe year before? Sam as above. lf costs involved in CDTI related activities are not clearly defined . Why is this? . Are any steps being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainability. . What is the trend in the way costing for CDTI related aclivities is done? . Your overall judgement is this indicator for sustainability being achieved? Fully r Highly r Moderately r Slightly r Not at all ' - Sustainability evaluation instrument no.3 - FLHF level - September 2004 version Ir r) l3 Folmattodr Bullets and Numbering7.2 Check whether sufficient funds to cover these costs are being disbursed from FLHF and/or districU LGA resources. indicator assesses wfeffer the programme is becoming integrated, and whether management is beginning to accept ownerchip of the programme and can mobilise the resources it needs. Characteristics of the indicator Sources of information a. Funding disbursed is enough to enable targeted, essential CDTI activities at this level to be carried out. b. The relative contributions of all sources of funding should be clearly spelt out. c. The proportion provided by the government (FLHF and/ or districU LGA levels) should be the major one by now (end of Year 5) or covering at least half of expenditure (end of Year 3). . Examination of: * Budget documents (government and NGDO) * Records of disbursement and expenditure (ledgers, orders, approvals for expendilure etc.) . lnterviews with: * DistricU LGA level managers (technical and administrative). * NGDO managers at this level. * FLHF team managers (leader, treasurer). * Local government managers at this level(chairoerson- adminislralor. treasurer). How much was provided by the government: . Last round of treatment? No funds are actually disbursed specifically for essential CDTI activity at this level. However, due to integrated activity of programmes at this level, they are always able to raise funds to carry out essential CDTI activities. . The round ofthe year before? Same as above. . The round the year before that? Same as above. lf the proportion supplied by the government is not the major one by now: . Why is this? . Are any steps being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . What is the trend in the relative proportion of resources contributed by the official health service? Your overalljudgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately I f i Not at att I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version t4 .ffi;.': iii 8.1 Check whether adequate and approprlate transport and materials are available for necessary CDTI activities at thls level. This indicator assesses whether the programme is functioning effectively, and whether it is able to mobilise the resources it needs. Characteristics of the indlcator Sources of information a. There are adequate numbers of appropriate, functional vehicles (of any type) available for neoessary CDTI activities. b. The running costs for this transport are met by a dependable source (e.9. the government). c. There are sufficient materials available for training and HSAM. . lnspection of: * Each vehicle being used: its source; its functional status. * Stocks of materials for training and HSAM. . lnterviews with the FLHF management team. Gt"t"dt fl Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I Moderately I f I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version This indicator assesses whether the programme is Characteristics of the indicator Sources of infomation a. There is a routine maintenance schedule for vehicles (where relevant), which is adhered to and recorded. b. The costs for vehicle and equipment maintenance and repair are met by dependable sources (e.9. the government). c. Repairs are rapidly and efficiently done. d. Staff have ways of coping when transport breaks down or is not available. so that CDTI activities are not disrupted. . lnspection ot * Vehicle and equipment maintenance schedules. * Vehicle and equipment maintenance records. . lnterviews with the FLHF manaqement team. Describe the present situation: No routine mainlenance schedules for transport were available at most of the FLHF, but there were breakdown repairs made, once lransport had a problem. On one of the FLHF in a plantation, its motor vehicle was regularly maintained from government funds. A pooling system within the FLHF affords availability of transport always; from other health programme can be used to carry out CDTI activities. Make particular enquiries about the ability of the government to pay for maintenance, repairs and tyre replacement: Despite the fact that there were no proposal or letters of request for repairs, nor reports of repairs available, Government was able to make some repairs in some FLHF. lf the vehicles are not being well maintained, and/ or the government is not paying: . Why is this? . Which steps are being taken to improve the situation? the report you have to summarise: * The evidence about how well this indicator is being achieved.t Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overalljudgement: is this indicator for sustainability being achieved? Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version 8.3 Check whether the transport avallable at this level ls appropriately managed and used, in an This indicator assesses whether the programme is functioning Characteristics of the indlcator Sources of information a. Trips made for CDTI purposes should be properly authorised in writing by the relevant official. Each trip undertaken should be recorded in a log book. b. Transport provided for CDTI, and that provided for other programmes, should be combined as a pool to be used for legitimate activities of all Droorammes at this level. . lnspection of vehicle control documents: * Copies of trip authorities (also noting destination and purpose) * Log books. * The supervision plan/ matrix. ' lnterviews with the FLHF management team. Describe the present situation: Authorization for CDTI trips made were not available and there were no log books for all FLHF visited Verbal authorization is often given in some FLHF for CDTI trips. Transport is used in an integrated manner at all FLHF. lf the transport is not being well managed: . Why is this? The fact that the vehicle is assigned to each individual affords them a means of managing the vehicle . Which steps are being taken to improve the situation? Analysis \A/hen writing the report you have to summarise: * The evidence about how well this indicator is being achieved.i Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affec{ sustainabilitv. Your overall judgement is this indicator for sustainability being achieved? Fully I Highly I f I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version l7 This indicator assesses whether the programme managers are taking ownerchip of the programme, and are able to find resources for it. NOTE: . ln some situations FLHFS do not have the resources or authority to undertake this kind of planning. 8.4 Check whether there are appropriate and realietlc plans for the replacement of transport and materials when APOC support comea to an end. Characteristics of the indicator Sources of information Management should know that replacements will be needed before the end of the programme, and have specific, realistic plans to meet the need at that time. It should be planned that the government will: + Provide replacements for vehicles. * Maintain existing vehicles. * Provide stationery and materials for training and HSAM. lf it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in these donor organisations (end of Year 5), or negotiations should have started (end of Year 3). . lnspection of letters of agreement. . lnterviews with: * The FLHF management team. * NGDO project managers. * High-ranking local government officials. Findi Describe the present situation: FLHF in most cases are not involved with planning nor do they have authority for making such realistic plans for replacement. However in one FLHF at the coffee plantation, management is aware that replacement should be made once the available one goes bad. lf the plans for re . Why is this? . Which steps are being taken to improve the situation? . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement: is this indicator for sustainability being achieved? Sustainability evaluation instrument no.3 - FLHF level - September 2004 version l8 W# 'W:l..i 9.1 Check whether the team at thls leve! is skllled and knowledgeable, regarding the imolementation of CDTI in its area of This indicator assesses whether the programme has been able to develop sufficient resources for itself. Characteristics of the indicator Sources of information a. Staff should have enough knowledge and skill to undertake all the key CDTI activities themselves: planning, training, HSAM, ensuring Mectizan supply, monitoring/ supervision. b. Staff at this level should remain in one post for a couple of years. c. There should be immediate training (in CDTI) of new, unskilled project staff members who have CDTI resoonsibilities. . lnspection ot * Staff files. * Monitoring reports. * Activity reports. . lnterviews with * Managers and other staff at this level. * Staff at the districU LGA level. * Village leaders and CDDs. Describe the present situation: . Particulars of cunent staff No. of persons qualified in this area No. of persons competent enough to perform the job Mectizan ordeing/ distibution . lnformation about stability: Most of the FLHF staff are stable at a post for more than 3-6years, while a few are quite recent on the job. There is a facility on ground for training new staff once they arrive. . lnformation about in-service training: None of the health staff has ever been on in-service training for any other health programme. But some of them are on studv leave. lf the staff at this level lack skills, and are often transferred: . Why is this? . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affar* suslainahilifu . Your overall iudoement is this indicator for sustainabilitv beinq achieved? Fully I I I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no.3 - FLHF level - September 2004 version l9 | 'Sid{ " :'ii u':."i'l;'i$ffi[iffi{- 1ery",*rgq!9E4 10.1 Check whether the FLHF areas/ sub-districts being evaluated have a satisfactory phic coverage rate This indicator assesses whether the programme is ellective - if the rates are poor the proiect is clearly struggling, and /ess sustainable. NOTE: . We only investigate the GCR of the FLHF areaV sub-districts that have been sampled. . ln some areas FLHFs don't keep their coverage statistics. ln such cases these will have to be looked for at the districU LGA level. . The denominator in the calculation of GCR is the total number of villages in the area under discussion, as identified by the latest REMO Fomatted3 Bulleb and FolmatEd3 Bullets and The GCR in the FLHF areal sub-district: . At the last distribution: AIIFLHF visited achieved 100% GCR . The year before: AIIFLHF achieved 100% GCR . The year before that: All FLHF achieved 100% GCR L! Analvsis . When writing the report you have to summarise the reasons for poor performance (if any); steps beinq taken to improve it; and how this is likely to affect sustainability. What is the trend in geographical coverage? . Your overall iudqement: is this indicator of sustainability being achieved?I Highly (GCR is 90-95%for the FLHF areas/sub-districts samoled) Moderately(GCR is 80-8906) Slightly (GCRisTO-79%\ Negligibly(GCR is<7rjo/^l Sustainability evaluation instrument no.3 - FLHF level - September 2004 version I f' Characteristics of the indicator Sources of information a. The GCR for the FLHF areas/ sub-districts sampled should approach 100% b. This GCR should be stable or increasing. . lnspection of: * Distribution reports and statistics at FLHF/ districU LGA level, for the past 3 years. ' REMO list of endemic communities for this districU LGA. . lnterviews with: " Staff at FLHF level. * Staff at districU LGA level. Check whether the FLHF areas/ subdistricts belng evaluated have a satisfactory therapeutlc coverage rate This indicator assesses whether the programme is struggling, and /ess sustainable. - if the rate is poor the project is cleaily NOTE: . We only investigate the TCR of the FLHF areaV sub-districts that have been sampled. . ln some areas FLHFs don't keep their coverage statistics. ln such cases these will have to be looked for at the districU LGA level. . The denominator for calculating the TCR is the number of villages Characteristics of the indicator Sources of information a. The TCR for the FLHF areas/ sub-districts sampled should be 65% or higher. b. These rates should be stable or increasing.. . lnspection of: * Distribution reports and statistics at FLHF/ districU LGA level, for the past 3 years. * REMO list of endemic communities for this districU LGA. . lnterviews with:I Staff at FLHF level.r Staff at districU LGA level. The TCR in the FLHF area/ sub-district: TCR has been quite variable for all FLHF visited because of high level of migration. . . At the last distribution(2005): average TCR for FLHF viisted 56.9%Biftu, Genja, Bebeka , Sheko, Gizmerret760/0 0 56% 71.1o/o 81.8% . The year before(2004) average TCRwas 72.4o/o Biftu Genja, Bebeka, Sheko. Gizmerret 67.70/o 43.9o/o 89.2o/o 670/o 94.1oh . The year before that(2003): Average TCR was 23.1% Biftu Genja, Bebeka, Sheko, Gizmerret Oo/o OYo O% 48.4o/o 670/0 . Why is this? . Which steps are being taken to improve the situation? Analysis . When writing the report you have to summarise the reasons for poor performance (if any); steps being taken to improve it: and how this is likelv to affec{ sustainabilitv. . What is the trend in therapeutic coverage? . Your overall iudoement: is this indicator of sustainabilitv beino achieved? - Highly (the TGR for the FLHF areas,/ sub- districts is 60{4%) Moderately (the TCR is 55-5go/") Slightly (the TCR is 50- 540/"\ Negligibly (the TCR is<50%) Sustainability evaluation instrument no.3 - FLHF level - September 2004 version :. lnstrument 4: community level NOTE: . This instrument evaluates the CDTI programme at the level of villages and communities, where the actual distribution of Mectizan takes place. We use the term 'community' to refer r 1 to both villages (in societies where there are well-defined villages) and communities where family groups are fairly isolated from each other, and do not live in a 'village' in the acceptedtJ geographical sense of the word. . ln these communities we find the following persons are involved in the CDTI programme: 'r * The community or village leadership - both traditional and elected. * The community directed distributors (CDDs) - the persons who have been selected by the community to do the distribution of Mectizan. .i * The other, 'ordinary' community members, who take the Mectizan yearly. ln this document these percons will be referred to as 'community members'.I r- . When collecting information from'ordinary'community members discussion groups should be conducted. The focus of this level's activities in CDTI The main function of this level is lo distribute the Mectizan yearly to the community members'. . Communities select CDDs, who are supported by the leadership and the other community members. . The CDDs update the community census every year; distribute the Mectizan appropriately; and send a reoort on the distribution to the FLHF level. Geooraohical name of . thiscommunitv/village:E Project: - Researcher: Date:E Abbreviations/ acronymsCDD community directed distributorCDTI community directed treatment with ivermectinFLHF first line health facility :- HSAM lffi:l3iU!ili1l;iiil:i:iT'",""?i?*,$ilif,ill3h""1ffil',HfiJ'chare programme r- NGDO non-governmentaldevelopment organisation !, Sustainability evaluation instrument no. 4 - community level - September 2004 version 2t; Ilt(ffi :l::;lii*tffi,l1sin".e' This indicator assesses whether the programme is efficient and simple. The more streamlined and time-efficient the job, the higher its sustainability. Characteristics of the lndicator Sources of infomation a. CDDs choose visiting times and routes which will make the work less burdensome. b. CDDs arrange with the community leadership for help with specific problems, such as families who are not willing to participate in the programme. c. CDDs carry out census and distribution during the same visit (using this census data for the following yea/s order). . lnspection of community treatment registers. . lnterviews with:r CDDs. + Community members (the chief not being present). * Community leaders. * FLHF staff. Describe the present situation: CDDs in conjunction with community leaders choose mode and time for distribution once the drugs are received in the community. Community leadership helps CDDs with problems such as families whom refuse to take the drugs).. Some communities( have Kebele task forces( comprising of village school directors, Development agents from Agriculture, women and youth associations.) are used for monitoring & supervision. CDDs usually update registers during distribution. lf CDDs are not working efficiently: . Why is this? . Are any steps being taken to improve the situation? . When writing the report you have to summarise:r The evidence about how well this indicator is being achieved. * Reasons for poor performanc,e (if any); steps being taken to improve it; and how this is likely to . Your overall judgement: is this indicator for sustainability being achieved? Not applicable Sustainability evaluation instrument no. 4 - community level - September 2004 version 3, 2.1 Check whether is with the distribution. This indicator assesses whether the programme is effective, and whether the community is takingL - ownerchip of it. Characteristlcs of the indicator Sources of Information a. The community leadership should be taking responsibilig for the distribution of Mectizan within the community. b. lf coverage (geographical and therapeutic) is not adequate or not being maintained, the leadership should understand the reasons for this. c. Together with lhe community at large, the leadership should identify and solve problems related to the distribution. . lnspection of minutes of community/ council meetings (where available). . lnterviews with: * CDDs. * Community members (the chief not being present). * Community leaders. * FLHF staff. Describe the present situation: ln some kebeles, the Community leadership takes up responsibility of monitoring mectizan distribution while in others they are not interested because they feel it's a paid job. Coverage has been adequate and whenever there is a drop, CDDs inform their leaders. Leadership and community member are involved with identification and problem solving eg like when CDDs drop out during distribution and when some members refuse to take the drugs. tf . Why is this? They feel it is a government paid job and they feel their CDDs are not telling the truth. . Are any steps being taken to improve the situation? No steps taken. Analvsis a When writing the report you have to summarise: * The evidence about how well this indicator is being achieved.t Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overall iudqement: is this indicator for sustainabilitv beinq achieved? Fully I f I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 4 - community level - September 2004 version 2.2 Check whether the communlty at large has been involved in taking decisions on the distribution This indicator assesses whetherthe community is taking ownerchip of the programme. Gharacteristics of the indicator Sources of information a. The community should have taken responsibility for decisions such as: * The selection/ changing of CDDs. * The timing and mode of distribution. * The kind of support they wish to give to their CDDs. b. Members of the community are involved in self-monitoring of their CDTI project. . lnspection of minutes of community/ council meetings (where available). . lnterviews with: * CDDs. * Communty members (the chief not being present). * Community leaders. * The persons supervising CDDs: FLHF staff, lay supervisors etc. Describe the present situation: Communities select their CDDs with the aid of health workers in most of the communities visited, but timing is dependent upon arrival of mectizan at the health facility in majority of the communities. Some communities sampled indicated that the time for drug distribution was preferably the dry season. Mode of distribution was mainly house to house while other communities had a central place, at over an agreed period. All communities did not support there CDDs because they felt they were paid, some also expressed the fact that they were our relatives that support for the CDDs was not necessary. Community leaders are involved in self-monitoring of CDTI activities, either by assigning some members whom move around in communities (kebeles) to verify if they are getting Mectizan during the distribution; while other community supervisors monitor distribution and facilitate the job of the CDDs( by collecting Mectizan, providing them stationary and returning data). tf . Why is this? . Are any steps being taken to improve the situation? Analvsis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overall iudoement: is this indicator for sustainabilitv beinq achieved? Fully I Highly I f I Sfightly I Not at all I Not applicable Sustainability evaluation instrument no. 4 - crmmunity level - September 2004 version 2.3 Check whether the community memberc value and accept long-term annual treatment. This indicator assesses whether the community is taking ownerchip of the programme. Characteristics of the indicator Sources of information a. Community members should be able to mention one or more advantages of taking Mectizan. b. Community members should express the need for annual lrealment with Mectizan. c. People should show understanding ofthe need for, and express interest in long ferm treatment with Mectizan. lnterviews with: . CDDs. . Communty members (the chief not being present). . Community leaders. . The persons supervising CDDs: FLHF staff. lav suoervisors etc. Describe the present situation: Communities were able to mention some advantages of taking mectizan which included expulsion of worms, reduction of itching, increased appetite, killing of body lice and a sensation of feeling good.. Communities have also expressed the desire to have the drugs taken more frequently annually. Communities have mentioned that the drugs should be taken for upwards ol 12lo 15 years. lf community members do not value and accept the treatment: . !ryhy is this? . Are any steps being taken to improve the situation? . When writing the report you have to summarise:t The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overalljudgement: is this indicator for sustainability being achieved? Not applicable Sustainability evaluation instrument no. 4 - community level - September 20(M version 6ffi *lxx&*ffi,Wffis$e 3.1 Check whether CDDg are to the FLHF leve!. This indicator assesses whether the programme is effective. lf such repofting is not taking place Mectizan supply will be compromised, which ls bad forsustainability. Characteristlcs of the indlcator Sources of information a. Reports to the FLHF level should get there on time. Reports may be summary reports, or the original community distribution record, depending on the level of skillof the CDD. b. Adequate transport should have been arranged by the community, for distribution records/ reports to be handed to the aoorooriaie oerson at the FLHF. . lnspection of community distribution reports. . lnterviews with: * CDDs.t The persons supervising CDDs: FLHF staff, lay supervisors etc. Describe the present situation: CDDs return their distribution registers to the FLHF immediately after the drug distribution while in some cases FLHF staff or Community supervisors collected the reports from the CDDs. Also for far to reach villages FLHF staff collect data from the communities. For close by communities transport was not provided but for distant communities government provided for them, while in others the community supervisors took the initiative on how best to collect their data. lf the reportrng by CDDs is poor . Why is this? . Are any steps being taken to improve the situation? . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement: is this indicator for Sustainability evaluation instrument no. 4 - community level - September 2004 version 7#t$**.l This indicator assesses whether the programme is effective. lf the ight amounts are received it will foster community ownerchip, which is good for sustainability. Characteristics of the indicator Sources of information a. All community members who were eligible for treatment got it, and some Mectizan was lefi over for absentees and those who were temporarily non-eligible. The quality of the census data in the CDDs' records must be good enough for an accurate calculation to be made, of the amount of Mectizan that needs to be ordered. . lnspection of treatment register for the community (held by CDDs; or at higher levels) . lnterview with: * CDDs. * Communty members (the chief not being present). * Community leaders. * The persons supervising CDDs: FLHF staff, lay suDervisors etc. What happened at: . The last round of treatment? All eligible populations were treated, for absentees repeat visits were made to ensure that they got their drugs. Left over were returned to FLHF after one month. ln most communities, registers seen appeared well kept and filled out in the native language( Ahmeric) while some registers missed out names of people that received drugs as well as repeating the same year over several columns.Required mectizan for the community were calcultaedd from these registers at the FLHF. . The rounds before that? Same as above lf the wrong amount of Mectizan was received: . Why is this? . Are any steps being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . What is the trend in Mectizan supply? . Your overall judgement: is this indicator for sustainability being achieved? Fully I I I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 4 - community level - September 2004 version Characteristics of the indicator Sources of information a. The CDDs or community members fetch the Mectizan they need every year, from a designated and mutually acceptable place. b. The community should arrange adequate transport for Mectizan to be collected from such a place. c. ln the case of remote communities, the districU LGA, in collaboration with FLHFS, should ensure that supplies reach such groups. lnterviews with: . CDDs. . Communty members (the chief not being present). . Community leaders. . The persons supervising CDDs: FLHF staff, lay supervisors etc. . DistricU LGA staff. 4.2 Check whether the GDDs or communlty memberc themselves fetch the yearly supply of Mectizan. This indicator assesses whether the project fosters community ownerchip. What happened at: . The last round of treatment? ln some communities CDDs, or community supervisors collect the drugs from the FLHF. While Woredas or FLHFstaff often supply mectizan to far to reach communities. Communities in the coffee plantation provide funds for CDDs to collect the drug. . The rounds before that? Same as above. lf community members or CDDs have not been collecting the Mectizan: r \A/hy is this? Distance was the major hindering factor for some far to reach and therefore the woredas and FLHF . Are any steps being taken to improve the situation? Analvsis . When writing the report you have to summarise:i The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . What is the trend in CDDs or community members fetching the Mectizan they need? . Your overall judgement: is this indicator for sustainability being achieved? Fully I I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 4 - community level - September 2004 version 9(' ft'r.i,;;: :*liiwlt: l-) 5.1 Check whether CDDs and community authorities continue to be engaged in HSAM of other members. This indicator assesses whether the project is effective, and whether the community has taken ownerchip of it. Characteristica of the indicator Sources of information a. CDDs/ community authorities identiff situations where community members require information. b. CDDs/ community authorities take necessary steps: ' To provide the information that the people need. * To encourage community members to provide resources. . To promote acceptance and ownership. This is done for example by means of meetings, sanctions, communitv bvJaws- lnterviews with: . Community leaders. . Communty members (the chief not being present). . CDDs. . The persons supervising CDDs: FLHF staff, lay supervisors etc. Describe the present situation: A meeting is set up by CDDs and community leaders prior to distribution to routinely give health education, sensitization and mobilization to community members. Community is routinely informed about the importance of taking the drug, ownership of the programme and providing support for the CDD. ln other communities there were community task forces working with CDDs who were involved with HSAM activities. No sanctions are set by the communities. lf CDDs and community leaders are not involved in HSAM: . Why is this? . Are any steps being taken to improve the situation? Analvsis When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it and how this is likely to affect sustainabilitv. Your overall judgement: is this indicator for sustainability being achieved? Fully I Highly I - | Slightly I Not at all r Not applicable Sustainability evaluation instrument no. 4 - community level - September 2004 version l0 fiarr' Hl'S&Yll.3lt::S;s$ffi1:r1L{'i,if.pl6tri 6.1 Check whether the communlty has made arrangements to support the process of distribution. This indicator assesses whether the project can mobilise the resources it needs, and fosters community ownerchip. Characteristics of the indicator Sources of information a. b. The community should support individuals who are providing CDTI services for them (moral support, support in cash or in kind). The community should make provision for the supply of recprd books, pencils, transport and other expenses incuned durino CDTI. lnterviews with: . Community leaders. . Communty members (the chief not being present). . CDDs. . The persons supervising CDDs: FLHF staff, lay supervisors etc. Describe the present situation Generally, communities do not provide support in any form (cash or in kind or transport) to CDDs. They believe they are paid by government. Members of some communities indicated that they are poor and can not contribute anything. For some communities CDDs are compensated by using them during polio campaign as drug distributed. Record books and pencils have been supplied by Government. While communities in most cases are unable to provide transport and other expenses incurred. . Why is this? . Are any steps being taken to improve the situation? When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to . Your overall judgement: is this indicator for sustainability being achieved? Not applicable Sustainability evaluation instrument no. 4 - crmmunity level - September 2004 veaion r-1 ll I!i'.+r; :',.iitt:*ffi*$4 7.'l Check whether there is a satisfactoru ratio of CDDs to households. This indicator assesses whether the project can mobilise fhe resources it needs, and whether the community has taken ownerchip of the programme. Characteristics of the indicator Sources of information a. A ratio of at least one CDD to 20 households (or 2 CDDs per 250 population) is recommended. However the decision must be taken by the community. b. The households for which CDDs are responsible should be close to their own homes. lnterviews with: . Community leaders. . Commun$ members (the chief not being present). . CDDs. . The persons supervising CDDs: FLHF staff, lay suoervisors etc. . The present ratio in the community: The ratio of CDD to households on the average was l:20or 22 for the communities. CDDs served households that were near them and did not move great distances. . The average distances that CDDs have to walk to get to homes: The average distance for most CDDs was 2.3 Km in most instances. lf the ratio of CDDs is bdor,,/r . Why is this? . Are any steps being taken to improve the situation? Analysis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affecl sustai nabilitv. . Your overall judgement: is this indicator for sustainability being achieved? I I Highly I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 4 - community level - September 2004 version t2 Characteristics of the indicator Sources of information a. CDDs should be skilled at their work: doing the census; giving the right dose; knowing who is not eligible; knowing what to do with mild side-effects b. There should be a plan in place for training CDDs to replace those who drop out, or when new ones are elected for other reasons. c. ln areas of co-endemicity with loiasis CDDs should be able to identiff cases of severe side.effects; give first aid; and refer aporopriatelv to the nearest FLHF. . lnterviews with; * Community leaders. * Communty members (the chief not being present). * CDDs. * The persons supervising CDDs: FLHF staff, lay supervisors etc. . Observing CDDs at work. 7.2 Check whether all CDDs have received . { Fomattld3 Bullets and Numbe;rg is indicator assesses wDefDe r the project is effective. Describe the present situation: Communities indicated that the CDDs receive appropriate training, they were trained by FLHF staff for 2 days within the communities. The training provided them information on how to, determine eligibles,to determine the right dose for different people, explain the side effects and refer to the FLHF in cases of severe side effect and to addrress concerns of the community in regards to the drug. There is a system in place whereby the community leader is involved with identiffing the need for replacement and training of new CDDs. tf . Why is this? . Are any steps being taken to improve the situation? Analvsis . When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustainabilitv. . Your overall iudgement is this indicator for sustainability being achieved? Fully I f I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 4 - crmmunity level - September 2004 version t3 7.3 Check whether CDDg are to contlnue theirwork in CDTI. This indicator assesses whether the programme has mobilised the resources it needs, and whether it fosfers co m mu nity ow n erc h i p. Characterlstics of the indicator Sources of information a. CDDs should express willingness to continue with distribution in the long term, given the conditions which prevail in the community. b. Few CDDs in this communig have dropped out from the distribution work. lnterviews with: ' Community leaders. . Communty members (the chief not being present). . CDDs. . The persons supervising CDDs: FLHF staff, lay supervisors etc. Describe the present situation: All CDDs expressed the willingness to continue there work irrespective of the fact that they were not supported in most instances. There have been few drop out mainly due to migration and marriages. lf some CDDs are doubtful or unwilling to continue, or have dropped out: . Why is this? . Are any steps being taken to improve the situation? Analvsis ' When writing the report you have to summarise: * The evidence about how well this indicator is being achieved. * Reasons for poor performance (if any); steps being taken to improve it; and how this is likely to affect sustai nability. . Your overall iudqement is this indicator for sustainabilitv beinq achieved? Fully I I I Moderately I Slightly I Not at all I Not applicable Sustainability evaluation instrument no. 4 - community level - September 2004 version I$$:ffiffiffi*hh1$"$*e#o- This indicator assesses whether the programme is the rate is poor the project is clearly struggling, and /ess sustainable. NOTE: . We only investigate the TCR of the communities/ villages that have been sampled. . ln some areas CDDs don't keep their registers. ln such cases these will have to be looked for at the chiefs house or the FLHF. . The denominator for calculating the TCR is the total population in the village/ community. Fo]matH: Bullets and 8.1 Check whether the communlties/ villages being evaluated have a satisfactory therepeutlc Characteristics of the indicator Sources of informatlon a. The TCR for the communities/ villages sampled should be 65% or higher. b. These rates should be stable or increasing. c. The rates provided by the register/ FLHF management must be believable, and confirmed by other observations if nossitrle . lnspection ot * CDDs'treatmentregisters. * Yearly distribution reports for that crmmunity. . lnterviews with: * Community leaders. * Communty members (the chief not being present). * CDDs. * FLHF level slaff. 2005 average TCR for all Kebeles visited was 73.2% It was difficult to collect the TCR because most of the registers were not available and for those available there were no lreatment summaries. However information on the treatment coverages for lhe communities had to be extracted from all FLHF viited.Again they were filled out in the local language and this was difficult to decipher in some cases even with the aid of interpreters. ln 2005 TCRs were Gola76.40/o, Muyakella 76.80/o,00184.6%, 003 95.3%, Berji 71.5%, Biftu 66%, Bajeka 62.8%, Shimi 53.6o/oYo, Mahalsheko 55.5%, Shyata 66.6% ,Otawa 61.3, Alenga 60.5% . The year before(2004) average TCR for Kebeles visited were 81.2% Goll(76.80lo), Muyakella (76.60/0),001(89%),003 (95.3%),Berji71.3%, Biftu 73.1%, Bajeka 0%, Shimi 0%, Mahalsheko 0%, Shyata 0%, Alenga 0%,Otawa 63.20/o. . The year before that (2003) ) TCR 2003 7O.9o/o: Gola76.40/o, Muyakella 76.80/o,00184.6%, 003 95.3%, Berji71.5o/o, Biftu 66%, Bajeka 62.8%, Shimi 53.60/o0/o, Mahalsheko 55.5%, Shyata 66.6% , Otawa 0%, Alenga 53.7%. lf the TCR is poor: . Why is this? . Which steps are being taken to improve the situation? . When writing the report you have to summarise the reasons for poor performance (if any); steps ing taken to improve it: and how this is likelv to affec{ sustainabilitv. . What is the trend in Sustainability evaluation instrument no. 4 - community level - September 2004 version t-- I l l5 . Your overall iudqement: is this indicator of sustainabilitv beino achieved? - Highly (the TCR for the villages/ communities is An-Aro/^\ Moderately (the TCR is 55- 5qo/^\ Slightly (the TCR is 5n-540/^l Negligibly (the TCR is<50%) Sustainability evaluation instrument no. 4 - community level - September 2004 version