World Health Organisation African Programme for Onchocerciasis Control Mid-term assessment of the sustainability of the Malawi 5Extension' CDTI project: Blantyr€, Chikw awa) Chiradzulu, Mulanje and Phalombe districts November 2002 Sebastian Baine Samuel Mugisha Spy Munthali Detlef Prozesky
3Index Page Abbreviations/ acronyms and acknowledgements Executive sunmary Introduction and methodology 1. Introduction 2. Methodology Findings and recommendations l. National level 2. District level 3. Health centre catchment area level 4. Village level 5. Overall sustainability grading for the project 6. A new insight Advocacy activities and feedback/ planning workshops 1. Advocacy activities 2. Feedback/planningworkshops Appendix I Appendix 2 Appendix 3 Appendix 4 Appendix 5 Report on meeting with WR The district level feedback/ planning workshop District sustainabi lity plans Suggested changes to the 'Sustainability' instrument Timetable of the evaluation visit 3 7 9 9 T2 t6 l6 23 29 33 36 37 4l 43 45 47 49 38 38 39
5Abbreviations/ acronyms African Programme for Onchocerciasis Control community development assistant community directed distributor (of ivermectin) community directed treatment with ivermectin Christian Hospital Association of Malawi Community Health Services Unit district commissioner district environmental health offi cer district health officer district implementation plan district onchocerciasis coordinator environmental health team first line health facility health centre health management information system headquarters health surveillance assistant health education/ sensitisation/ advocacy/ motivation Intemational Eye Foundation Malawian kwacha Ministry of Health and Population non-governmental development organisation national onchocerciasis ooordinator National Onchocerciasis Control Programme national onchocerciasis control team National Onchocerciasis Task Force rapid epidemiological assessment rapid epidemiological mapping of onchocerciasis Traditional Authority World Health Organisation APOC CDA CDD CDTI CHAM CHSU DC DEHO DHO DIP DOC EHT FLHF HC HMIS HQ HSA HSAM IEF K MoHP NGDO NOC NOCP NOCT NOTF REA REMO TA wHo Acknowledgements We would like to thank the following persons for their help: . The staffat the Headquarters of the African Programme for Onchocerciasis Control (APOC) in Ouagadougou: Dr S6k6t6li, Dr Amazigo, Mr Aholou. . Staff of NOCP/ MoHP in Blantyre, for undertaking all the arrangements: Mr Tambala (NOC), Mr Sitima (Deputy NOC), Mr Mkhoma (training and monitoring supervisor), Mr Kwizombe (accountant), Ms Jere (secretary), Messrs Kamwana and Kariati (drivers). . Political and traditional leaders, health workers and community members in the Blantyre, Chikwawa, Chiradzulu, Mulanje and Phalombe districts.
7Executive summary The African Programme for Onchocerciasis Control (APOC) has been supporting the Malawi 'Extension' CDTI (community directed treatment with ivermectin) project for the past three years. At a meeting of national onchocerciasis task forces (NOTFs) in Abuja in June 2002,it was decided that the sustainability of the projects which APOC supports should henceforth also be assessed midway through the funding period; the present evaluation is the first such evaluation. It was carried by a team of four evaluators - two from Uganda and one each from Malawi and South Africa. The evaluation followed on an evaluation in May 2002 of the first phase of the Malawi project (covering the Thyolo and Mwanza districts), which found that there were considerable problems with sustainability after five years of funding. This second evaluation was carried out over a period of twelve days. Information was collected by document study, interview and observation, at sampled sites at four levels of the health service: national, district, first line health facility (FLHF) and village. The newly revised set of 'Sustainability' instruments was used and found to be suitable for the Year 3 evaluation (although a few refinements are suggested). The following are the principal findings of the evaluation: . Coveraqe: The rate of implementation of the project has been slower than was originally planned. Mectizan has not yet been distributed in Chikwawa district and part of Blantyre district: a few CDDs (community directed distributors) remain to be trained, and many others await the Mectizan which finally arrived during the evaluation. Training has yet to commence in Chiradzulu district (the NOTF first wishes to clarify the endemicity situation there). Therapeutic coverage is lower than expected: several possible reasons for this state of affairs were uncovered, but more detailed research is needed to clarify them. . Plannine: Planning at national level is thorough, but the National Onchocerciasis Control Programme (NOCP) is not yet included in the yearly cycle of planning, with equivalent programmes, at the level of the Community Health Services Unit (CHSU) in Lilongwe. Similarly CDTI was only included in the district implementation plan (DIP) in one of the five districts. There is a strong and successful tradition of integrated yearly and monthly planning at the FLHF level, in which CDTI is present to varying degrees. The national onchocercasis co-ordinator (NOC) has clear ideas about obtaining resources for CDTI after APOC support is withdrawn, but these have not yet been written down in a systematic plan. . Supervision and monitorine: Data from the village level move effectively to the national level, but in some cases the district health management team (DHMT) is bypassed. There is an excellent system of review meetings after each distribution. Supervision as it is being conducted at present is not as efficient as it could be (too many visits involving too many staffmembers) nor as effective (NOCP team members are not really aware of the reasons for the low coverage rates, and there are no checklists). There is almost certainly a problem with the way in which rates are at being calculated at present, which needs to be rectified. There are at present no indicators for CDTI in the national health management information system (HMIS), but the relevant manager at Headquarters (HQ) level at the Ministry of Health and Population (MoHP) in Lilongwe would welcome suggestions. . Trainine and HSAM (health educatior/ sensitisatior/ advocacv/ motivation): Training targets are being exceeded in terms of numbers - this is due to an under-estimation of the numbers of separate villages (and therefore CDDs) in the original project submission. In general training is conducted efficiently, but there are no specific courses available for staffat the national level. Excellent HSAM materials are available but are in short supply, 8largely due to their expense. Much advocacy work has been done, but some traditional authorities (TAs) and district commissioners (DCs) have not yet been approached. Mectizan supply: This is working well, and takes place entirely within the government system. Inteeration of support activities: These are highly integrated at the district and FLHF levels - this is one of the great strengths of the Malawi programme. Available transport and other resources are routinely shared between progralnmes, and workers perceive themselves to be a team of multi-skilled professionals. Finances/ fundine: transport and equipment: The government's contribution is at present limited almost entirely to salaries - everything else is funded by APOC (transport, equipment, stationery, allowances, HSAM material). There is a small budget for onchocerciasis control in the WHO country representative's (WR's) biennial budget, but the methods and extent of government funding post-APOC remain to be spelt out. There is no doubt though that some funding will be available - MoHP has created posts for the programme, and in the local system this implies a commitment to providing the wherewithal for the progftrmme to function. The use of transport is in general well controlled. Human resources: One of the great strengths of the Malawi programme is the fact that there are MoHP employees at village level - the health surveillance assistants (HSAs) - charged with implementing approved preventive and promotive programmes. These persons were found to knowledgeable, reasonably committed and stable. They are supported by specific staffat FLHF and district level. In some instances however too few CDDs are being trained. The overall judgement of the team is that the Malawi 'Extension' CDTI project is not far from being sustainable. The health system in Malawi is functioning at a level which is strong enough to carry CDTI; it is highly integrated; and it reaches effectively right to village level. There is still time to achieve maximal integration of the project into the MoHP system at all levels, and there is a reasonable assurance that MoHP will be able to supply the resources the programme needs to function when APOC withdraws. Detailed recommendations were drawn up, based on the findings of the evaluation. The recommendations were prioritised, and indicators and deadlines were suggested for each. The most important recommendations concem: . Entering the Government planning and HMIS systems more fully, and drawing up a detailed 'Sustainability plan' based on current ideas. . Bringing Chiradzulu district into the project immediately, and completing the necessary training in Blantyre district. . Researching the reasons for low therapeutic coverage, and correcting mistakes in calculating coverage rates. . Planning supervisory visits more efficiently. Advocacy activities were carried out, mostly at the district and TA levels. Two feedback/ planning workshops were held - one for the national onchocerciasis control team (NOCT) and one for the DHMTs of the five districts where the 'Extension' project is operating. In each case the evaluation team gave feedback on its findings, which were discussed in depth. In the case of the district level workshop the evaluators then guided the participants to draw up realistic 'Plans for sustainability' for their areas of operation. I l a 9Introduction and methodology 1. Introduction 1.1 Onchocerciasis control in Malawi It is estimated that 750 000 persons in Malawi are infected with the parasite Onchocerca volvulus, and altogether 1,5 million persons are at risk of contracting the disease. Efforts to control onchocerciasis in the country had started well before APOC began to offer its support. Between 1984 and 1987 a mission hospital determined the prevalence of the scourge in the Mwanza and Thyolo districts, by means of skin snip surveys. This was followed by the distribution of ivermectin on a trial basis. The Intemational Eye Foundation (IEF), a non- govemmental development organisation (NGDO) based in the United States, joined this progftlrnme in 1991, and mass distribution of Mectizan to affected communities began in earnest. The activity progamme was however limited to two endemic districts, Thyolo and Mwanza. With the advent of APOC and its strategy of CDTI the decision was made to focus all control efforts around the new strategy, and to expand the programme in due course to cover all hyper-endemic and meso-endemic communities in the seven most affected districts in the southern part of the country. These areas were determined after a rapid epidemiological survey for onchocerciasis (REMO) has been conducted for the whole country, in 1997. ln the Malawi CDTI prograrnme the partners were to be MoHP, IEF and the Christian Hospitals Association of Malawi (CHAM). The first CDTI project approved by APOC for funding covered the districts of Thyolo and Mwanza, and funding began in 1997. The sustainability of this initial project was evaluated in May 2002, after five years of APOC funding. The overall finding was that the project was 'potentially sustainable, but will require re-thinking and mobilisation of high-level support to get it on the road to sustainability'. The report cited the following obstacles to sustainability: . CDTI did not appear in the national health plan, and there were no budgetary allocations from MoHP for the programme. Key directors in MoHP were unaware of the CDTI programme, and there were no explicit plans to imsure the sustainability of the prograrnme after the withdrawal of APOC funds. ' At district level funds were also not being budgeted for the CDTI programme, which was consequently dependent on outside funding for its existence. . At FLHF level key managers had insufficient knowledge of CDTI activities; transport was inadequate; CDTI activities were not budgeted for; and HSAs (the actual trainers of CDDs) were themselves inadequately trained. . At village level both political and traditional leadership were not part of the mainstream CDTI prograrnme; villagers were not organised to take final responsibility for the prograrnme; and some CDDs were unable to perform key tasks. It should be noted that the present evaluation team disagrees with some of these findings, considering them not to be valid in the light of the nature of the health service in Malawi. As a country Malawi is at present facing a number of serious national problems. The HIV/ AIDS problem is very great, and is having a profound effect on society. A series of poor harvests and mismanagement of the national grain reserve has led to an acute food shortage, with widespread reports of starvation. ln addition a number of donor countries are withholding substantial amounts of overseas development aid, due to alleged misuse of funds l0 - and such funds normally make up to one third of the national budget. All of these factors may affect the NOCP. 1.2 The present evaluation At a meeting of representatives of the NOTFs of APOC member countries in Abuja in June 2002, it was decided that it was not enough to evaluate the sustainability of projects funded by APOC after five years of funding. Since this gave them very little time to undertake remedial action, an additional evaluation after only three years was decided upon. The present evaluation in Malawi is the first such evaluation. It also follows on a revision of the instrument used to evaluate project sustainability, and therefore presents an opportunity to test the instrument and assess its suitability for such a mid-term evaluation of sustainability. 1.3 Background to the Malawi 6Extension' project The scientific for the Drol ect Only the 1997 REMO report was available, to plan the 'Extension' areas that need treatment. This provided the following information, in relation to the districts to be covered: Note that no hyper-endemic communities were found in these districts; the sample for Chiradzulu was extremely small; and only hypo-endemic communities were found in Chikwawa. Not surprisingly the 1999 project application planned to conduct a rapid epidemiological assessment (REA) to complement this sparse information. The proiect application: tareets The 1999 application to APOC for the 'Extension' set the following targets (a) For treatment; Nodule prevalence District No. of communities examined 240o/o thypert 20-39Y" tmesot l0-l9o/o 'hypo' <lOYo Blantyre l0 0 1 4 3 Chiradzulu 4 0 I 2 1 Mulanje/ Phalombe 15 0 7 2 I Chikwawa t4 0 0 5 I Communities to be under treatment by: District Endemicity No. of communities 1999 2000 2001 2002 2003 Mulanje Hyper 98 60 98 98 98 98 Meso 133 80 133 133 133 Phalombe Hyper 54 54 54 54 54 54 Meso 80 40 80 80 80 Blantyre Meso 63 33 63 63 63 Chikwawa Meso 33 33 33 33 33 Chiradzulu Meso 90 45 90 90 90 11 Note that this table assumes that hyper-endemic communities exist. The REMO however only found meso- and hypo-endemic communities (see first table in this section). (b) For CDD training: CDDs to be trained per year: Total no. of CDDs to be trainedDistrict 1999 2000 2001 2002 2003 Mulanje/ Phalombe 399 553 326 r278 Blantyre 116 105 221 Chikwawa ll6 116 Chiradzulu 158 158 Proeress in achievine the tareets Training actually proceeded as follows: Targets therefore appear to have been met or exceeded. However progress in implementing CDTI in the five districts of the 'Extension' has been considerably slower than originally planned. Actual distribution of Mectizan has only been taking place in Mulanje (all villages indicatd by REMO), Phalombe (all villages) and Blantyre (about half of the villages). In Chikwawa CDDs were only recently trained for all villages, but no Mectizan has yet been distributed; and in Charadzulu no training has taken place at all, at any level. The reasons for apparently exceeding the targets, and for the delay in implementation, are discussed in the body of the report. A brief word needs to be said about the control programme in the tea estates in Mulanje. These farms operate as independent entities, operating their own health services. In this situation NOCP trains HSAs (who are employed by the estates) in CDTI; and these HSAs, assisted by volunteers, conduct the Mectizan distribution. Year Total achieved Overall targetDistrict 2000 2001 2002 *Mulanje/ Phalombe Villages covered 67 t7t 57 295 231 CDDs trained 3t7 593 267 1277 t278 Blantyre Villages covered 113 113 63CDDs trained 565 565 221 Chikwawa Villages covered 70 70 33CDDs trained 263 263 116 Chiradzulu Villages covered No training has yet taken place 90CDDs trained 158 t2 2. Methodology 2.1 Sampling Four of the five districts could be examined down to village level, but in the case of Chiradzulu only the district level would be examined. In the other four districts traditional authority (TA) areas were sampled as follows: . Blantyre: two TA areas with their trvo health centres (HCs) (the only ones included in the programme). . Mulanje: two TA areas with their two HCs (the only ones included in the programme). . Phalombe: one TA area, with its two HCs (the only ones included in the prograrnme). One of the HCs is run by MoHP and the other by CHAM. . Chikwawa: two HCs out of four were randomly selected (no coverage data available yet). These two centres serve three TA areas. In view of the fact that the evaluation team of four persons had to cover five districts, the decision was taken to sample only two villages per FLHF catchment area. In the 2% districts where coverage rates were available, the villages were selected according to the criterion of 'therapeutic coverage' - selecting one village with good and one with poor coverage. In the other 1% districts the criterion of geographical spread/ distance from the HC was used to select the two villages. This resulted in the following sample: * No treatments given yet. District Traditional authority HC/ FLHF Community Blantyre Kapeni *Limbe *Chiwembe (near FLHF) *Jumbe (far from FLHF) Somba Mpemba Chunga (good coverage) Mpingo (poor coverage) Chikwawa Maseya/ Katunga *Maperera *John (far, hill country) *Muyaya (near, hill country) Makhuwira *Makhuwira *Gangu (near, plain) *Singano (far, plain) Chiradzulu ..'i-+"L,;:;i"r 't".fu,.r !. rr_t-:- -nBtrr*,@ h,*hfl/siif it-ffi is.i.*#ft fi tffi!##'{*{*s"Ti;rl.. Mulanje Mabuka Mimosa Mbyelioma (high coverage) Katute (low coverage) Njema Muloza Monda (high coverage)Mkhumba (low coverage) Phalombe Nazombe Nkhulambe (MoHP) Nanyalo (high coverage)Likhura (low coverage) Sukasanje (CHAM) Naliya (high coverage)Siyankhuni (low coverage) 5 districts 7 TAs 8 HCs 18 communities tI I t3 2.2 Protocol Research question: How sustainable is the Malawi 'Extension' CDTI project? D es i gn: Cross-sectional, descriptive. Population: The Malawi 'Extension' project, including:the national team in Blantyre; its NGDO partner (IEF); its five district, with all staff involved in onchocerciasis control in them; the project villages, with their leaders and CDDs. Instrument: 't { record sheet, structured as a series of indicators of sustainability. The indicators are grouped into nine categories/ groups. These groups represent critical areas of functioning of the prograrnme. * The instrument assesses sustainability at four levels of operation. * The instrument guides the researcher to collect relevant information about each indicator, from a variety of relevant sources. Source of information; * Verbal reports from persons interviewed. * Documentary evidence and observations. Analysis; * Data from all sources is aggregated, according to level and indicator. * A qualitative sunmary of the situation regarding each indicator at each level is made. This is aggregated and summarised for each category of indicator, for each level. * Based on the information collected, each indicator is graded on a scale of 0-4, in terms of its contribution to sustainability. * The average 'sustainability score' for each group of indicators is calculated, for each level. Recommendations; * These are strictly based on the findings of each area of research. 2.3 Team composition The core team members were the following 1. Sebastian Baine * Institute of Public Health, Makerere University, PO Box 7072, Kampala, Uganda r. T: *25671925 861;F: *25641 531807 * sobainefn)vahoo.co.za or sbaine(Oiph.ac.us 2. Samuel Mugisha * PO Box 33613, Kampala, Uganda * T: * 25677 582 505 * kmueisha200l @yahoo.com or prcueandafg)hotmail.com 3. Spy Munthali * Economics Department, University of Malawi, Box 280,Zomba, Malawi * T: * 265 8 829 599 * smbiriyawaka@yahoo.com or spym@chanco.uninra.nrw 4. Detlef Prozesky (team leader) * Faculty of Health Sciences, University of Pretoria, PO Box 667, Pretoria 0001, South Africa * T: * 27 123541147:'F:* 27 123541758 I * prozesky(@medi c.up.ac.za ot prozesk co.za t4 Each team member became the head of a small sub-team, which was made up of one core team member and two community development assistants (CDAs) drawn from the four districts where field work was to be done. These persons were used to collect village level data, and were trained on the job. Each sub-team was also accompanied by a member of NOCT, to act as guide and facilitator. On the first two days of the exercise the core team members met to familiarise themselves with the revised instrument, and to complete planning of the evaluation process. 2.4 Advocacy visits and 'Feedback/ planning' meetings Advocacy visits were to be paid to relevant persons at each level, as many as possible, and oflicials were to be debriefed at the end of the field visits - again if possible. Finally, meetings were to be conducted for relevant offrcials at the national and district levels. During these planning meetings the evaluation team would give feedback on its findings, and the national and district teams would be asked to develop sustainability plans, based on the findings. 2.5 Limitations The evaluation team was not suffrciently aware of the unique nature of the tea estates, as a separate entity with a different modus operondi. They were therefore conceptually lumped together with Mulanje district, and never actually visited. Future evaluations should see them as an additional district, to be sampled and evaluated separately. 2.6 The performance of the revised 'sustainability' instrument The revised instrument proved easy to use, and relatively unambiguous. The following should however be noted: ' The instrument is designed for use in projects in 19 countries, which differ considerably in the way in which their health services are structured. Some of the indicators and sub- indicators may therefore appear irrelevant for a particular situation (for example, in Malawi financial planning at the FLHF level doesn't take place). Evaluators should be aware of this, and the core team needs to include comments about this issue in the 'Guidelines'. . There was a tendency to focus more on the present situation in each indicator, with a relative neglect of enquiry into the reasons for it. This is not a defect of the instrument, but rather of the way in which it was used. . There were a few indicators were the wording was still found to be unclear. Suggested changes are given in Appendix 4. . There were a few cases where the wording was not entirely appropriate use of the 'third year' evaluation. Suggested changes are also given in Appendix 4. . The instrument as it is structured at present leaves quite a lot of room for data to be written down on it: * This makes it possible to use one copy to record data from several sources - e.g. from a few interviews, from documentation etc. * Even if the allocated space is full, the blank page to the left can be used for additional data recording - and this space could even be formatted for this purpose: *15 lf you need ryDre space for rnforrlalon related to the rndrcator on the faqng page. Mte rt here. Make sure you rndrcate lhe souroe of rnlorTr€tron dearly. Having all the information collected in one place makes the analysis easier, since there are not so many different pieces of paper to work from. t6 Findings and recommendations 1. National level 1.1 Overall grading (on a scale of 0-4) 1.2 Main lindings and recommendations 1.2.1 Plannine Yearly planning is thorough and detailed, and is compiled as part of the yearly application to APOC for renewed funding. The plan varies from year to year, since the programme is still expanding. All NOTF members are given the opportunity to take part in drawing up the plan (which not all do). Roles of the different partners are clearly spelt out in the plan. The area of HSAM is relatively under-emphasised though. There is a yearly planning exercise at the level of CHSU at MoHP HQ in Lilongwe. The sections which make up the Unit are all supposed to take part in this exercise, but to date NOCP has not been involved. It does however forward its annual plan to CHSU. The NOCP leadership has clear ideas about moving towards sustainability when APOC funding ends. These have however not yet been written down, nor have they started being implemented. Malawi 'extension' project: sustainability at national level .,€" ,eS ,"... ,.C o*d "./ "/\$' ^.""'t .u'"oc ^"c a""" u- group of indicators 4 3.5 3 2.5 2 ,5 1 0.5 t .9o; o EDG o o 1 a*o'"" ^"".'-* ..o. lr9 ""t" n TI I7 Reasons for not being part of the HQ planning process, and not writing down the 'sustainability plan' : * For years the programme has been officially sited within Thyolo district, for the purposes of administration. There is no tradition of planning together with the other CHSU prograrrmes. * The programme is physically located far from HQ in Lilongwe, which makes its inclusion into routine meetings there more difficult. * The whole question of 'sustainability planning' is entirely new. 1.2.2 Supervision and monitorine Monitoring data is generally received in time from the districts, and is collected into district reports. Data related to onchocerciasis control is however not included in the national HMIS. The director of CHSU is of the opinion that this should be done. Staff at this level have developed routines for responding effectively to problems shown up by supervision and monitoring. The review meetings held after each distribution effectively identify problems, which are immediately handed to those present to solve. The written reports sent to districts after each distribution clearly present problems identified by the data, for districts to attend to. Staff in the periphery report that NOCP staff appreciates their efforts - a positive reinforcement for them. In general the supervision conducted by staff from this level is targeted at the district level. There are indications however that supervision takes place more frequently than is operationally necessary; that visits last too long and that supervising teams are too large. ln addition, supervision frequently continues up to the HC level. The visits are therefore not as efficient as they might be. There is at present no supervisory checklist, which NOCT members can use when they conduct supervision at district level. District level staff have also expressed the need for a supervisory checklist, for their visits to the HSA supervisor level. There are at present no indicators of CDTI in the national HMIS. The CHSU director is however willing to consider their inclusion. I Recommendations :'Planning' Implementation l. An explicit plan for onchocerciasis control/ CDTI should be integrated into the yearly National Health Plan, together with those for the other prograrnmes falling under CHSU. 2. A written plan to achieve a state of financial sustainability needs to prepared. This plan should include specific steps to: * Increase likely income and resource allocation for the programme. * Contain costs, to a level which equals that of expected income (see also 1.2.6 below). 3. The detailed year plan should contain specific targeted HSAM activities, to remedy shortages noted at present. This includes plans for printing HSAM flipcharts in larger numbers, for distribution to the different levels of the prograrnme. Priority: I,2,3.. HIGH Indicators ofsuccess: 1. 'Onchocerciasis control' appears in the next CHSU year plan. 2. The written sustainability plan exists. 3. The next year plan (the one submitted to APOC) includes more specific, targeted HSAM activities. Who to take action: 1. NOC, NOTF, Director CHSU 2. NOC and NOTF. 3. NOC and NOTF. Deadline for completion:l. February 2003 (or when CHSU planning takes place). 2. June 2003. 3. As soon as possible. I 18 Reasons for 'over-supervising' and not preparing checklists: * It is difficult to move from a situation at the beginning of a programme, where supervision has to be more intense, to one where lower level staff members are helped to take on that responsibility, in a more mature situation. * Since MoHP salaries are very low, field allowances form an important salary supplement. * NOCP staff members are highly experienced, and may not feel a need for checklists. 1.2.3 Mectizan supplv Mectizan supply is managed entirely by MoHP employees, using a unique system. The system is simple and effective and supplies sufficient quantities for the districts' needs, usually at the right time. 1.2.4 Trainine and HSAM t NOCP staff train members of the district EHT, and supervise these members of the EHT when they in tum train HSAs. There are however a few reports of NOCP staff actively taking part as tutors in HC level training, which is not appropriate. Training is properly targeted, but there are indications that the number of trainers is at times more than is operationally necessary. NOCP has in the past planned and carried out necessary and effective HSAM activities at district level, but is now delegating those to district staff. Recommendations :' Supervision and monitoring' Implementation 1. Supervision checklists should be prepared for the following situations: * NOCP staff supervising districts. * District environmental health team (EHT) staff supervising HSAs. The relevant staff should then be briefly trained in the use of these checklists during supervision. 2. Supervision by NOCP needs to be carefully planned and controlled: the number of visits undertaken; the number of staff involved per visit; the duration of the visits; and the level at which supervision is conducted. This is in view of the need to empower staff at district and sub-district level to do their own supervision, and also the need for cost containment. 3. Additional training in data management should be offered to all those involved in programme implementation, to enable them to analyse census and distribution data correctly. 4. NOTF should study the present national HMIS; decide on an indicator that will help CHSU to monitor CDTI nationally (probably'Therapeutic coverage'); and make a submission to Director CHSU to include it in the HMIS. Priority: I,2,3.. HIGH 4: MEDIUM Indicators ofsuccess; 1. The checklists exist and are used. 2. The 'Sustainability plan' indicates steps that will make supervision more efficient. Annual reports of distributions show that these steps have been implemented. 3. Reports of training in data management exist. 4. A CDTI indicator is included in the national HMIS. Wo to take action; 1,2,3: NOC, NOCP. 4. NOC, NOTF. Deadline for completion: 1. By the next distribution. 2. With immediate effect. 3. At the next round of review meetings. 4. June 2003. I r9 Although HQ level staff at Lilongwe have a high level of awareness of the programme, they do not as yet see the need to involve Programme leadership in routine planning and budgeting exercises. Reasons why training is not as eficient as it mtght be, and why Lilongwe HQ staff is not involving NOCP as much as other sections: * These are discussed under'Planning' and 'Supervision and monitoring' above. L2.5 lnteeration of support activities There are numerous examples of integration of support activities: the review meetings, combining supervision/ Mectizan supply, supervision/ HSAM etc. 1.2.6 Finances and fundine The costing of CDTI activities is routinely done in detail. The NOC has a good perception of funds that will be available to the prograrnme the following year, and budgets within that. There is however no sign yet of deliberate cost containment, in preparation for the leaner years ahead. The relative budgetary contributions of the partners are clearly spelt out in the budget. MoHP funds salaries, but has not yet begun to provide funding for other NOCP activities: APOC still funds these entirely. Plans to bridge this shortfall eventually are still embryonic, and no commitment in this direction has yet been obtained. Although other national programmes have budget lines within CHSU, this is not yet the case for NOCP. Funds available to the programme are carefully and efficiently managed. Reasons why MoHP is not contributingfunding; why cost containment has yet to begin: * MoHP probably perceives that, since the programme is at present adequately funded by APOC, it does not need State funding. * The 'distance factor' (from Lilongwe) also operates here. * Since the programme is still in its growth phase, it is difficult to start thinking of ways of diminishing expenditure for specific activities. t ! I Recommendations:'Training and HSAM' Implementation l. The number of NOCP trainers taking part in training needs to be carefully controlled, as well as the level at which they conduct training. This is in view of the need to empower staff at district and sub-district level to undertake training themselves, and also the need for cost containment. 2. In view of the relatively low profile that 'Onchocerciasis control' has at CHSU in Lilongwe, NOTF should seriously debate the advantages and disadvantages of the programme having an office at CHSU (in addition to the one in Blantyre). Priority: I. HIGH 2. MEDIUM Indicators ofsuccess: . The 'Sustainability plan' and the annual plans reflect the need for efficient, targeted training. . NOTF has made a decision about having an additional office in Lilongwe. Who to take action; . NOC, NOCP . NOTF. Deadline for completion; . June 2003 - or before the next training round. . The next NOTF meeting - early in 2003. 20 Recommendations:'Finances and funding' Implementation 1. The'Sustainability plan' mentioned in 1.2.1 above should indicate clearly how costs are going to be contained, in such a way that sustainable routines are established before APOC funding comes to an end. 2. NOCP should request to be present at CHSU's annual planning and budgeting meeting: * During this exercise NOCP should make a submission, reflecting the amount of counterpart funding agreed to by MoHP at the time of the original project agreement. * The Onchocerciasis Control Programme is to be allocated a line in the CHSU budget submission, similar to that of the other CHSU programmes. Priority: 1. HIGH 2. HIGH Indicators ofsuccess: l. The 'Sustainability plan' indicates how costs are to be contained. 2. NOC has attended the 2003 CHSU planning meeting, and NOCP appears in the CHSU budget for 2003-2004. Who to take action; r. Noc. 2. NOC and Director CHSU. Deadline for completion'. 1. June 2003. 2. January-May2003. 1.2.7 Transport and material resources . Sufficient transport is available but it is dependent on APOC for its running costs (i.e. not sustainable in the long run). Since the vehicles should seldom be required to go beyond the district level (and in fact seldom do), they may be heavier and mechanically more complex than is required - and will consequently be more expensive to run. . Office equipment is sufficient and in good working order. . Logbooks are scrupulously kept for all vehicles. All trips have to be authorized, but the entries are not regularly checked. Vehicle maintenance is recorded in the logbooks, but is not strictly planned according to a schedule. Office equipment is repaired rather than maintained. At present APOC bears the cost of maintenance and repair of vehicles and equipment entirely (again not sustainable in the long run). . NOC has a clear plan for the replacement of vehicles and equipment, when this becomes necessary. There are procedures to be followed within MoHP, with which he is fully familiar, and which in his experience have proved effective. . Reasons why MoHP is not contributing to running/ maintaining vehicles and equipment: * These are discussed under'Finances and funding' above. Recommendations: 'Transport and material resources' Implementation 1. The issue of achieving sustainable transport and equipment for all levels of the programme needs to be fully addressed in the 'Sustainability plan'described in 1.2.1 above. 2. A senior NOCP staff member should reconcile logbooks monthly. Priority: 1. HIGH 2. MEDIUM Indicators ofsuccess; 1. These items are included in the 'Sustainability plan' 2. The task of reconciling logbooks monthly is written in the relevant officer's iob description. Wo to take action: 1. NOC and NOTF. 2. NOC. Deadline for completion: 1. June 2003. 2. Immediately. 2l 1.2.8 Human resources The number of staff members at this level is appropriate, and they have appropriate knowledge and skills. They are highly stable and appear committed and hard working. In the nature of the APOC/ CDTI system they have had no formal training for the work in which they are engaged. They also report needing more general management and data management skills. L2.9 Coverase Geographical coverage: * ln the districts of Mulanje, Phalombe and Chikwawa all communities have now been covered (although in some villages in Chikwawa treatment has still to follow on the recently completed training). About 20%o of the target area in Blantyre remains to be covered. * Project implementation has been behind schedule. Training for all five districts was supposed to have been completed by the end of 2001. * The progmrnme has not yet been launched in Chiradzulu district - this after three years of funding of the 'Extension'. Reasons for the delay in implementation: * The slow pace of completing training may be due to: - The fact that NOCP has to work through the districts and their EHTs, and is unable to force their pace. - NOCP's modus operandi, which has been to let districts start training in one area, and to make sure that that works before going on to other areas. * The reasons for not tackling Chiradzulu appear to be: - NOTF has always been uncertain about the intensity and scope of the infection in Chiradzulu, for a variety of reasons. Only four district villages were sampled during the REMO. The actual boundary (between areas to be treated/ not to be treated) was never finally demarcated following the REMO. The known spread of Recommendations:'Human resources' Implementation 1. Selected members of NOCP should undergo formal training in the following areas: * Data management. This course needs to be carefully selected, to include both common epidemiological indices and their mathematical aspects, and appropriate software (e.g. Epilnfo). * Training methodology. Again a carefully selected, very practical course, dealing with improved lectures; preparing and using visual aids; conducting practicals; and assessment of learning. * General management: A very practical course, aimed at mid-level managers. 2. An advanced course in 'CDTI management' or'Managing onchocerciasis control' needs to be developed by a suitable institute of higher learning, to enable existing and especially new managers of CDTI programmes to leam the ropes. Priority: 1. LOW 2. MEDIUM Indicators ofsuccess: 1. Staff members have successfully completed the short courses. 2. The advanced course exists. Who to take action: 1. NOC (to find courses), APOC (tunding). 2, APOC. Deadline for completton:l. June 2003. 2. End of2003. l I22 the disease in neighbouring districts, combined with certain local geographical features, cast doubt on the way the demarcation line started being drawn. - It appears to NOTF that there is considerable danger that many hlpo-endemic villages might be treated, unless the present situation is first clarified by a REA. The project apparently erred previously in Mwanza district, by starting treatment in some hypo-endemic villages. - The reply to an application for funding for the REA was late in coming from APOC. The application was turned down. - Pending clarification of the situation by a REA (for which funds have yet to be raised) NOTF decided to concentrate on the four other districts, which appeared to present a much higher treatment priority. Therapeutic coverage: the village level rates vary considerably, but tend to be lower than in comparable projects elsewhere. Only 27% of villages achieve rates of 650lo or more. In the Mulanje tea estates the rate of coverage is higher than in the districts - up to 81.8% in 2001. This may well be because distribution is carried out by HSAs employed by the estates. Possible reasons for the low therapeutic coverage: * Seasonal movement of people. * Fear and uncertainty: the measuring stick as a sign of death, fear of 'family planning'. * Religious objections to taking medicines. * Some members of the EHT not working in a systematic way, or not being committed to the programme. * Incorrect calculation of coverage rates by HSAs. * Relative unconcern since the disease is not intense ... and so on. Recommendations:' Coverage' Implementation 1. A rapid epidemiological assessment exercise must be carried out, to clarify the endemicity situation in Chiradzulu district. Funds need to be raised for this exercise, as soon as possible. 2. Research must be conducted into the reasons for low therapeutic coverage in the villages of Mulanje and Phalombe. This could be coupled with a comparison between these villages, and the villages in Blantyre and the Mulanje tea estates, where the coverage is higher. 3. The remaining villages in the endemic area in Blantyre district must be fully enrolled in the programme. Priority: 1,2,3" HIGH Indicators ofsuccess; 1. The REA report for Chiradzulu is ready. 2. The research report on low therapeutic coverage in Phalombe and Mulanje is ready. 3. All villages in the endemic area in Blantyre are fully enrolled in the programme. Wo to take action; 1. NOC, NOCP. 2. NOC, APOC. 3. NOC, NOCP. Deadline for completton; 1. 6 months after funding has been obtained 2. April2003. 3. June 2003. 23 2. District level 2.1 Overall grading (on a scale of 0-4) 2.2 Main findings and recommendations 2.2.1 Plannine Although all districts made fairly detailed year plans for CDTI (usually in the form of a Gantt chart) such plans were only included in the DIP in one case. In some cases several members of the health team participated in drawing up plans. Such planning is not however taught in the training NOCP provides. Reasons for failing to integrate the CDTI plan into the overall year plan are: * The prograrnme is still new. * The DHMT sees it as externally funded, and it therefore does not need funding from the Ministry of Health and Population (MoHP). * The capacity of the district to fund new programmes is low. Malawi'extension' project: sustainability at district level 4 3.5 E3 E 2.5 .9 o =2oED,.(E r.J o (El 0.5 0 -""" ".C .".C ..u$ "os$'" ^-.".4 qs- ".a.- a@' dr*."" .ry "/ a""" group of indicators 24 Recommendations :' Planning' Implementation l. The district health management teams (DHMTs) should henceforth incorporate CDTI activities in the DIP, in the same format that is used for the other control prograrnmes. 2. Where this has not yet been done, NOCP should specifically train district environmental health officers (DEHOs) and DHMTs to draw up a plan for onchocerciasis control, that will fit into their DIPs. Priority: 1, 2: HIGH Indicators ofsuccess; 1. The DIPs for 2003-2004 include CDTI activities. 2. Training reports exist, which show that DHMTs have been trained to draw up suitable plans. Wo to take action; 1. DHMTs, DEHOs and district EHTs. 2. NOC and NOCT. Deadline for completion: 1. April2003. 2. March 2003. 2.2.2 Leadership In each district a district onchocerciasis coordinator (DOC) has been appointed. The yearly Mectizan distribution process in the districts is being initiated by messages from the NOCP; thereafter the DOC and her/ his colleagues in the EHT proceed more or less unaided with the arrangements preceding the distribution. The other members of the DHMT are relatively uninformed about the programme - so too the district commissioners (DCs). Reasons for the relative lack of involvement of the DHMT: * It has delegated responsibility for the programme to the DEHO. * Since the programme is new it is normal for NOCP to take the lead at this stage. 2.2.3 Monitorine and supervision Data and requests are being channeled from civil servant (DOC/ DEHO) to civil servant (NOCP). The district health officer (DHO) is however being blpassed, which is not ideal. Supervision is effrcient, with the EHT routinely supervising their HSAs (with occasional spot checks in the actual villages). Like all this team's work the supervision is also integrated - at each visit a supervisor will check several of the HSAs' many tasks. Problems uncovered from monitoring data and supervision visits are dealt with, and at the appropriate level: HSAs are always involved. The routine use of review meetings after distributions is excellent. [n one case only success was specifically noted and praised. Recommendations :' Leadership' Implementation 1. Whenever members of NOCT visit the districts they should make a point of seeing and informing members of the DHMTs. This should be done in the company of the DEHOs and DOCs. 2. NOCT members should visit DCs at least once ayear, to discuss progress and possible problems. Priority: 1,2: HIGH Indicators ofsuccess; l, 2: Reports of next supervision visits by NOCT members to the districts. Who to take action; 1, 2: NOC and NOCT, with DEHOs and DOCs.. Deadline for completion; 1,2: At next routine supervision visit. tr n n I25 2.2.4 Mectizan supplv Orders for Mectizan are usually made on the prescribed forms. In half of the districts there were reports of the Mectizan arriving late, which caused some irritation. The supply process is entirely managed by civil servants; however in some cases NOCT delivered the drug to districts, rather than asking them to fetch it. The DOCs usually store/ control the Mectizan without involving the district pharmacy: this new system may be more efficient. Reasons for the drug aruiving late, and NOCT taking Mectizan to the districts: * DOC waited for all the district census results to be handed in, before doing the ordering. This consumed a lot of time. * A consignment was held up at the local airport, due to a misunderstanding about tax exemption. * If NOCT is visiting a district anyway it makes sense to deliver the Mectizan as well. 2.2.5 Trainine and HSAM The EHT trains HSAs, and also supervises the training of CDDs by HSAs (one member of the EHT per training - the HSAs do the bulk of the work). This is reasonable since the programme is still relatively new in these districts. The team is able to give in-service training to newly arrived team members, but still depends on training organised by NOTF. The training of HSAs is efficient - both the initial training, and the refresher/ review meeting after subsequent distributions. New HSAs are trained according to need. The Recommendations:'Monitoring and supervision' Implementation l. Reports of CDTI activities at district level should routinely be passed on to the DEHO and DHMT, for them to forward to NOCP. This can be either of the following: * The full reports required by APOC. * The more limited information that will go into the HMIS (see Recommendation 4 in 1.2.2 above). 2. The district CDTI team should ensure quality data from lower levels, before compiling and submitting reports to NOCP. This means that the accuracy with which CDDs and HSAs work with data needs to be checked, and further practical training done if necessary. Priority: 1,2: HIGH Indicators of success: 1. NOCP receives reports from DHO. 2. Coverage reports are irm accurate reflection of what happens on the ground. ll'ho to take action: 1,2: NOCT, DOC, DEHO, DHO Deadline for completion: 1. Immediately after the next distribution. 2. Before the 2003 distribution. Recommendations:'Mectizan supply' Implementation l. NOCT should set a deadline for the districts to hand in census forms. If these are very late orders should be made based on the previous year's figure, plus 10%. Priority: 1. MEDIUM Indtcators ofsuccess: 1. NOCT makes orders in time for distribution. Who to take action; 1. NOC and NOCT Deadline for completion; 1. Immediately, for the 2003 order l TI 26 I training is not integrated with that for other programmes, but integration of the initial and review training is not desirable. HSAM activities are only partially successful. Both planning and implementation is lacking in important respects - e.g. some DCs and traditional leaders have not been approached. Drama groups for IEC have been trained but have no transport. Reasons for failure to implement HSAM : * The district teams have been focusing on implementation, and on the levels below them - so this has been an oversight. 2.2.6 Finances and fundine The DOCs/ DEHOs do not produce costed annual plans - rather they submit budgets for specific CDTI activities to NOCP. There is only partial evidence of efforts to limit expenditure. The DEHO and DOC have only a vague idea of the amount of funding that might be available to them from MoHP, but they do know what NOCP/ APOC is likelyto provide, and work within that - but sometimes not all of it is received during a given year. The MoHP contribution at this level is in terms of salaries, and this is clearly spelt out (but the document is at national level). There is no evidence of MoHP allocation to CDTI at this level. Because of the highly integrated nature of the EHT's work, some MoHP funding is however supporting supervision where CDTI is also covered. Disbursed funds (both MoHP and NOTF/ APOC) are tightly and efficiently managed. Reasons for inadequate budgeting practice and MoHP budgets: * The MoHP budgeting mechanisms are not programme specific, and the outcome in terms of delivering funds is uncertain. * Since NOCP/ APOC is going to supply their needs, there is not much incentive for DEHO/ DOC to prepare budgets for the MoHP system - which may not provide anything at all. They have in any case not been trained to do so. * DHMT knows that NOCP/ APOC is supporting this programme for the time being, so will tend to allocate scarce resources to programmes without outside donor funding. * The DHMT is responsible for many programmes. It is not going to make special efforts to raise funds from other sources, to control a disease which is not so severe. Recommendations:'Training and HSAM' lmplementation 1. DOCs and their teams should keep all civic leaders and decision makers (e.g. DCs and traditional authorities) informed about progress in CDTI. Priority: I. MEDIUM Indicators ofsuccess: l. Civic leaders have up-to-date information about CDTI in their areas. Wo to take action: 1. NOCT, DOCs and their teams (especially CDTI supervisors). Deadline for completion;l. January 2003 and yearly thereafter. 27 2.2.7 Transport and material resources Transport at present is adequate for the prograrnme's needs. MoHP is meeting all of the maintenance costs and provides some of the fuel, but NOTF/ APOC provides fuel for specific activities. Training materials (the two Malawi manuals) are sufficient - photocopies are made locally - but HSAM materials are in short supply in some cases. Some maintenance on vehicles is carried out but it is not routine. MoHP meets all maintenance and repair costs, but repair is very often delayed. The EHT copes with broken down vehicles, partly because of its habit of using transport in an integrated way. Transport is used efficiently, as an integrated pool (although motorcycles are allocated to persons) and at the right level. Logbooks are universally available and completed, but their content is not checked. Management is aware that the present motorcycles will have to be replaced sooner or later. They will apply to MoHP when the time comes, and (on past experience) have a reasonable hope that something will materialise. There are no plans at present to approach other donors. Reasons for unsystematic maintenance, delays in repair, poor logbook control, lack of HSAM materials: * Routine servicing is not possible due to lack of MoHP funds - this is also the reason why repairs are delayed. * The probable reason why logbooks are not checked is that managers are too busy. * They depend on NOCP to provide HSAM materials. Recommendations:'Finances/ funding' Implementation 1. The onchocerciasis control plans in the DIPs must be fully costed - limiting expenditure as much as possible. This is so that districts will begin to attract attention for budget allocations from MoHP. 2. Where this has not yet been done, NOCP should specifically train DEHOs and DHMTs to prepare careful, targeted, minimal budgets for their onchocerciasis control plans, to fit into their DIPs. Priority: 1,2: HIGH Indicators ofsuccess; 1. The DIPs for 2003-2004 include budgets for CDTI activities. 2. Training reports exist, which show that DHMTs have been trained to budget for CDTI. Who to take action; l. DHMTs, DEHOs and district EHTs. 2. NOC and NOCT. Deadline for completion: 1. April2003. 2. March 2003. 28 2.2.8 Human resources The EHT is knowledgeable and skilled. Staff is generally stable. Unskilled persons joining the team are trained by NOCP, but the EHT judges that it would be capable of doing that itself. Salaries are regularly paid but they are very low. Staff satisfaction is generally high, as far as we could ascertain (this positive attitude may however be influenced by the fact that they have been receiving allowances on several occasions, due to their involvement in programme activities). There is no real incentive system operating. 2.2.9 Coveraqe Geographical coverage: in three out of the four districts under discussion all villages indicated by the 1997 REMO survey have now been involved (although in some of them the process is not yet complete - census and/ or distribution remains to be done). Therapeutic coverage: this is discussed at the 'national' level above. Recommendations:' Transport and material resources' Implementation Plannine: 1. The issue of achieving sustainable transport and equipment for this level of the programme needs to be fully addressed in the 'Sustainability plan' described in 1.2.1 above. 2. The district transport control officer should reconcile logbooks monthly. Priority: I. HIGH 2. MEDIUM Indicators ofsuccess; l. These items are included in the 'Sustainability plan' 2. The task of reconciling logbooks monthly is written in the relevant officer's job description. Who to take action; 1. NOC and NOTF, in consultation with Director CHSU and DHMTs. 2. DHO, district transport officer. Deadline for completion; 1. June 2003. 2. Immediately. Recommendations :'Human resources' Implementation 1. The DHMT should assume full responsibility of training incoming/ unskilled CDTI staff. Priority: 1. MEDIUM Indicators ofsuccess; 1. Incoming staff members are trained on the job by other members of district EHTs. Who to take action: l. DEHOs and DOCs. Deadline for completion 1. Ongoing. I 29 3. Health centre catchment area level This area is taken to mean: . Primarily, the EHT working in the HC catchment area: village level HSAs and their supervisor, who is based at the HC. . Less importantly, the medical assistant and nurse/ midwife working in the HC. These persons are only peripherally involved in CDTI. 3.1 Overall grading (on a scale of 0-4) 3.2 Main findings and recommendations ', 3.2.1 Plannine HSAs and their supervisor draw up monthly work plans, in which CDTI activities appear at the appropriate time. In one district ayear plan (in the form of a Gantt chart) was seen, in which the activities relating to all EHT programmes at this level were set out - including CDTI. The HSAs (supervisor and village) are not however specifically taught to plan the yearly CDTI events in a sensible sequence. No written 'recommended package' for this level wir seen - although HSAs have a set of activities that they routinely carry out. Malawi 'extension' project: sustainability at health centre level .€s ^-". group of indicators 4 3.5 3 2.5 2 .5 1 0.5 0 rf .9o! o E"(E o G 1 ^"o "ro" ,y ,..-({c' ^.uo .ogd "@" i" .*o'& """C --"2" {.d" ,,. 30 Recommendations :' Planning' Implementation l. HSA supervisors need to be trained practically to produce written year plans for their areas of operation. 2. NOTF should motivate to CHSU to have CDTI officially included in the 'recommended package of activities' for HSAs, for the six districts involved in the progfirrnme. Priority; 1,2: MEDIUM Indicators ofsuccess; 1. The written year plans are produced. 2. The written official 'recommended package' includes CDTI. Wo to take action; 1. NOCT, DOCs, EHTs, HSA supervisors. 2. NOTF, Director CHSU. Deadline for completion; 1. EndofJanuary2003. 2. June 2003. I 3.2.2 Leadership The EHT in the HC catchment areas have taken the programme on board, as simply another programme for which they are responsible. They start CDTI activities when the district level EHT asks them to do so, every year. The medical assistants in charge of the HCs all support the programme, but the understanding is (rightly) that it falls in the domain of the EHT. Leadership is therefore undertaken by the HSA supervisor. In two districts traditional authorities had been oriented about the programme, but not in the other two. Reasons why the traditional authority has not been oriented: * EHTs are focused on programme implementation, and the need to approach the TA has not been a priority for them. 3.2.3 Supervision and monitorine The HSAs submit their census and coverage data to their supervisor routinely and effrciently. S/he in turn has no problem in submitting reports and data to the district level. All of this takes place entirely within the MoHP system. CDTI activities are effectively and efficiently supervised. In addition to primary visits to get CDTI launched for the year, other visits may follow if problems arise. Visits by HSAs to villages (and visits to the HSAs by their supervisor) are generally integrated, in the sense that more than one programme gets attended to per visit. Problems arising in the CDTI process are rapidly identified and dealt with, by involving village level HSAs, CDDs and the village leadership appropriately in solving the problems. The use of positive reinforcement as a management tool is limited to occasional verbal appreciation. Reasons why positive feedback is seldom used: * The mindset of managers of this new programme is on identifying problems to solve them, in order to achieve good results. I I I 31 3.2.4 Mectizan procurement and distribution Quantities ordered were correctly calculated, and there were no reports of shortages. In three cases out of four correct order forms were used; ordering was timed to fit in with the preferred community timing for distribution; and the drugs arrived on time. The Mectizan is collected, stored and administered entirely within the MoHP - using a parallel system to that used for other drugs. The Mectizan is sometimes fetched from the district by the HSA supervisor, and is sometimes delivered to the HC by the DOC. The drug is stored in the HC, where the control system is simple and effective. HSAs may fetch it from the HC, or the HSA supervisor may deliver it to them. In all these cases transport is readily available. 3.2.5 Trainine and HSAM All training can be justified. The initial training of CDDs is clearly needed, and subsequent 'training' at the review meetings deals with problems identified during the preceding distribution. Training is economical: using small numbers of tutors of the appropriate levels; choosing sites close to where CDDs come from; limiting the time spent. The village leadership is actively co-opted by the excellent practice of asking them to attend part of the CDD training: the effect this has on their participation is clear. Otherwise HSAM activities are carried out when the need arises. Traditional authorities have however been neglected in some districts. 3.2.6 Financine and fundine HCs have no financial capacity and do not make budgets - their finances are managed at the district level, where information regarding the role of finances at the HC level may be sought. I I Recommendations:' Supervision and monitoring' Implementation 1. Steps must be taken to ensure that HSAs and their supervisors handle data and reporting 100% correctly. ln particular the denominator that is used must be correct. Priority: 1. HIGH Indicators ofsuccess: 1. All reports Wo to take action: 1. NOCT, DEHOs, DOCs, EHTs, HSAs. Deadline for completion: 1. June 2003, then ongoing Recommendations:'Training and HSAM' Implementation 1. Targeted, practical training is provided for HSAs, on data management and reporting. Priority: 1. HIGH Indicators ofsuccess; 1. All reports are accurate Who to take action; 1. NOCT, DOCs, EHTs, HSAs. Deadline for comp letion:l. Before the next round of distributions. l 32 3.2.7 Transport and materials . Motorcycles (for HSA supervisors) and pushbikes (for HSAs) are provided - but some motorcycles have broken down. The district provides fuel but not enough - HSA supervisors report that they sometimes use their own money for fuel. Training materials are easily obtainable, but HSAM materials are often in short supply. . There is no scheduled maintenance for the motorcycles. MoHP pays for all maintenance and repairs, but repair is very often delayed. HSAs generally have to fund pushbike repairs themselves. Having said that CDTI is not really held back by these problems - the EHT copes. . The use of the motorcycles is controlled by logbooks, which are faithfully kept but never really checked. The use of pushbikes is controlled by a system of personal attachment - in one district HSAs gradually purchase the bikes they use by means of a monthly payment of K 300. All transport at this level is used in an integrated way, each vehicle serving the needs of all programmes. . The replacement of vehicles is a district level issue. . Reasons why maintenance is not done, repairs are delayed, HSAM materials arefew: * Due to lack of MoHP funds routine servicing cannot be afforded. Repairs similarly have to wait until funds become available. ' Providing HSAM materials is seen to be the responsibility of NOCP. 3.2.8 Human resources Staffis generally knowledgeable and skilled regarding CDTI - with one notable exception: the management of the data supplied by the CDDs. New staff members who have not yet been trained will either join a formal training event at district level or higher, or be shown the ropes by colleagues locally. EHT staffmembers at this level are fairly stable. t I 3.2.9 Coverase Geographical coverage: in those HC catchment areas that are on stream all villages are covered. Therapeutic coverage: this is discussed at the 'national' level above.a 4. 4.1 33 Village/ community level Overall grading (on a scale of 0-4) 4.2 Main findings and recommendations 4.2.1 Plannine The CDDs plan their work carefully: allocating areas to each other so that these are close to their homes; selecting times of work convenient to the community etc. They involve community leaders in dealing with problems which arise. In one case they also combine the census and distribution, thus lessening their workload. Reasons why the census and distribution are not combined: * This option is not offered to communities when the programme is explained to them. Malawi 'extension' project: sustainability at community level .*s'"" .".'t ".."" *.d .t$ ..p ""' Group of indicators 4 3.5 3 2.5 2 1.5 1 0.5 0 !i .9oi o E"g o + Recommendations:'Planning' Implementation 1. All communities should be offered the option of doing census and distribution at the same time, to reduce the CDDs' workload. For villages which accept this option, Mectizan orders will be calculated from the previous year's census. Priority: 1. HIGH Indicators ofsuccess: l. Village meetings have been held, where this option is offered. Who to take action: 1. NOCT, DOCs, EHTs, HSAs and their supervisors. Deadline for completion:1. Before the next round of distributions. f] tr 34 I 4.2.2 Leadership and ownership The community leadership is taking responsibility for the programme in the villages. They are aware of the reasons why people are not taking the Mectizan, and help to solve these and other problems related to the distribution: calling meetings, discussing benefits, accompanying CDDs, gathering the villagers for centralised distribution and so on. The community selects the CDDs at a mass meeting. On that occasion they usually also decide on the mode of distribution, and sometimes also on the time of the year that they would like the distribution to take place. Community members are aware of some of the benefits of taking Mectizan (although some tend to exaggerate these). They generally know that the treatment is to be annual, for about 15 years. 4.2.3 Reportine Reports (of the census and distribution) are sometimes made by CDDs unaided, and sometimes together with their HSAs. Transport arrangements vary - sometimes the CDDs bring the reports/ registers, and at others the HSAs fetch them from the CDDs. Transporting the reports presents no problems, and reports generally arrive in good time. 4.2.4 Mectizan and distribution CDDs receive enough Mectizan for their communities' needs - there is usually a little surplus, which is returned to the HSAs. Interestingly CDDs do not know how to calculate the number of tablets they need. Sometimes the CDDs fetch the Mectizan from the HCs, and sometimes the HSAs fetch it and deliver it to them. The transport for either process presents no problems. Reasons why CDDs don't know how to calculate their needs: * They are simply not taught this simple calculation. 4.2.5 HSAM CDDs and community leaders identify situations where community members need more information, and take steps to promote acceptance and ownership of the programme - e.g. by calling meetings to answer questions, asking those who have taken the drug to speak to others etc. There is generally a shortage of HSAM materials at this level though. Recommendations:'Mectizan procuremenV distribution' Implementation 1. CDDs should be taught how to calculate the number of tablest of Mectizan that they need for their villages. Priority: 1. MEDIUM Indicators ofsuccess; 1. All CDDs are able to make the calculation. Who to take action: 1. NOTF, DOCs. EHTs, HSAs Deadline for completion: 1. At the next round of review meetings. a 35 4.2.6 Fundine The materials involved cost negligible amounts; transport is catered for; and the volunteer spirit of the prograrnme is still intact. There is at present no place for funding at this level of the programme, and the issue is not a threat to sustainability. 4.2.7 Human resources In most villages the ratio of CDDs to the population is adequate - but there are some notable exceptions, where CDDs have a heavy workload. Similarly most CDDs do not have to walk very far to do the distribution; a few though have to walk up to an hour, in communities with scattered homesteads. CDDs are generally well trained, although some experience problems with completing their registers, or are unsure about exactly what to do in case of side-effects. HSAs, CDDs and the village leadership generally know what to do to replace CDDs who fall away (some form of formal or on-the-job training). ln general CDDs appeil willing to continue their work - annual attrition rates are within acceptable limits (1.8% - 7.0%). The rate of attrition is however increasing yearly, and it remains to be seen where it will peak. Some of the reasons for attrition are migration and wanting to be paid. Reasons why there are toofew CDDs in some communities: * HSAs appear not to know what the ratio should be, at the time of electing CDDs. 4.2.8 Coveraqe Geographical coverage within villages in Malawi is unlikely to be other than 100% - the phenomena of hamlets or multi-cultural villages does not exist, and villages are generally small. Therapeutic coverage: this is discussed at the 'national' level above. It is important to note that the CDDs and community leaders of villages sampled for their low coverage claimed that their coverage rates were in fact excellent. I T Recommendations:'Human resources' Implementation 1. HSAs should ensure that the recommended ratio of CDDs to community households is adhered to. 2. CDDs should be re-trained in drawing up and maintaining their registers correctly, and also in the management of the side-effects of Mectizan. To save money HSAs should do this during their routine visits to CDDs, at the beginning of the next round of distributions. Priority: 1,2: HIGH Indicators ofsuccess: 1. There is at least one CDD to 20 households (or per 125 population). 2. The reports of CDD re-training exist. Who to take action: 1,2: NOCT, DOCs and EHTs, HSAs. Deadline for completion; 1, 2: Before the next distribution. 36 5. Overall sustainability grading for the Malawi 6extension' project Following the instructions laid down in the manual, the team analysed the situation in the project as a whole under the following rubrics: . Mone! - It is going to be essential to formalise as much as possible the ways in which MoHP will fund the running of the programme, at all levels, within the next year or two. The prospects of meaningful funding becoming available appear reasonable. . Transport -The future availability at national level will depend on NOCP's ability to gain access to vehicles provided from time to time by MoHP. The prospects here again are reasonable. At district and HC level the integrated nature of transport use by EHTs augurs well for the future. . Supervision - At NOCP level this will depend above all on the availability of transport, which it is reasonable to assume will still be available. At district and FLHF levels again the integration of EHT activities bodes well. . Mecttzan supply - No problems are foreseen in this area. . Political commitment - This seems to be adequate at all levels. Following on the analysis above, the level of sustainability of the Malawi 'extension' project is judged to be as follows: As is the case in all CDTI projects that are funded by APOC in its member countries, the most important factor determining future sustainability is probably the strength and degree of development of the district health service. In the case of Malawi this system is well developed (although under-funded). Each district has a well-developed EHT; there are HSAs in each village; there is a well established system of communication and supervision within the different levels of the EHT - all of this augurs well for the future sustainability of CDTI in the country. Level of sustainability Description Hish This project is not far from being sustainable. With feedback from the team before departure, the project staffshould be able to undertake the required remedial action. 37 I A new insight In the course of the evaluation it became clear that there is a potential threat to sustainability, the effect of which will only become evident after APOC funding comes to an end. The present instrument fails to draw attention to it. It operates as follows: . Health workers at almost all levels of the health services in APOC countries receive salaries which do not meet the basic needs of their families, such as housing, food, clothing and education. It therefore becomes a matter of critical importance for them to find ways of supplementing their basic income. This they do in a number of ways, such as doing private practice after hours, running a farm or a small business, and so on. r I useful source of additional income is the allowances that are paid to health workers for their involvement in particular activities or prograrnmes. In situations where workers are responsible for several programmes it is natural therefore that they will pay particular attention to programmes which are more lucrative. At present CDTI in Malawi is such a programme. At national level it offers salary top-ups, as well as per diem allowances for any trip to the field. At district level it offers EHT staff members allowances for activities such as training and supervision - so too for the HSAs at FLHF level. Not surprisingly therefore staff members are highly motivated to undertake these activities. ' When however the financial benefits of involvement in CDTI come to an end, staff members will be obliged to look for other programmes and activities which will supplement their incomes. Workers in the State Onchocerciasis Control Teams (SOCTs) in Nigeria were quite open about this, perceiving an opportunity for themselves in the upcoming lymphatic filariasis control progftrrnme, and planning enthusiastically to limit their future involvement in CDTI. Even if managers were aware of the extent to which this phenomenon poses a threat to sustainability, their options for countering it appear limited. Since it is not possible to maintain financial incentives indefinitely, the only alternative appears tobe thorough and timely integration: to have the CDTI programme so firmly embedded in routines and structures at each level, that it cannot be removed or neglected. It is suggested therefore that the instruments be adapted, to highlight the extent to which this factor is operating in a given situation. This can be done by creating an indicator in the 'human resources' category, which investigates the adequacy of workers' salaries, and the measures they are taking to supplement them - particularly the role played by CDTI prograrnme allowances. The data collected under such a heading will alert evaluators to the extent to which this threat is present. Measurement of the degree of integration of CDTI is already well catered for in the instruments. Its importance however needs to be stressed anew. It is suggested that 'integration' should become the sixth key rubric against which the final assessment of sustainability of a project is made - the others at present being money, transport, supervision, Mectizan supply and political commitment. 38 Advocacy activities and feedback/ planning workshops By now it is routine for project evaluation teams to undertake two additional groups of activities, in addition to the evaluation itself: . Advocacy for the CDTI projects, with civic and government leaders at different levels. . Planning meetings with local stakeholders, at different levels. This section describes how the evaluation tearn went about fulfilling these two additional tasks. 1. Advocacy The team undertook the following advocacy activities: 1.1 National level Involvine the National Onchocerciasis Co-ordinator Throughout the evaluation there was constant communication with the NOC. The team drew him in regularly during the planning phase and while data were being analysed - for instance to clarify the puzzling finding of low coverage rates, and the apparent 'neglect' of Chiradzulu. His deputy was also closely involved. The NOC expressed his appreciation of this involvement. with oflicials in Only one day was available for a visit to Lilongwe, and appointments were made for meetings with the following persons: * Dr Somanje (Director of Disease Control, and chairman of NOTF). * Dr Salaniponi (Director of CHSU) * Dr W Aldis, WHO country representative. Due to a misunderstanding the evaluation team member was not fetched in time at the airport in Lilongwe, and the first two appointments were missed. An in-depth conversation was however held with the WR - a recent appointment - at the end of which he had a very clear understanding of the CDTI prograrnme and its future needs (see Appendix 1 for a fuller report of the meeting). A telephone call to Dr Salaniponi at a later stage also provided useful information. 1.2 District and Traditional Authority level In each of the four districts where field work was done, efforts were made to inform and motivate the authorities. This was done in the following ways: . Courtesy visits upon a:rival in the district, to the DHO, district administrator and DEHO. During the visit information was also collected from these persons. They were also invited to the coming feedback and planning meeting at Lutchenza. . Visits to the District Commissioners and some of the relevant traditional authorities, to inform them about the CDTI programme, the reason for the current evaluation, and the evaluation team's provisional findings. l 39 2. Feedback/ planning workshops 2.1 District level This workshop was held on Tuesday 19th November, at a conference centre in Lutchenza, in Thyolo district. Attendance was excellent, from all five districts involved in the Extension project (including Chiradzulu). The workshop process developed as follows: . lntroductions - the aim of the evaluation. . Feedback on findings at the community, FLHF and district levels, followed by in-depth discussion in plenary. . Each district developing its own plan for achieving CDTI programme sustainability. Since Chiradzulu has no programme yet, persons from this district were distributed amongst the others, to leam from their experience. These documents constitute the first draft of a'Sustainabilityplan' for each district. It is the responsibility of NOCP to follow up the formalisation and implementation of these plans. Further details of the workshop are given below: the full programme and attendance list (Appendix 2), and the plans developed by four districts (Appendix 3). It is important to note that the high level of attendance may have to do with the substantial allowances that were paid to all those who attended the workshop. In situations where salaries are very low employees are obliged to use every possible means of augmenting their income. 2.2 National level This meeting was held on Wednesday 20th November, at the NOCP office in Blantyre. The entire national and IEF teams were present. It should be noted that the same persons had already attended the meeting at Lutchenza on the previous day, and were therefore up-to-date concerning the evaluation team's findings at the district and lower levels. The meeting lasted about five hours, and consisted ofl . Feedback from the evaluation team, on findings at national level. . Discussion around each finding, concerning its accuracy and its implications for the NOCP. The steps that the NOCP now needs to take were discussed in detail - for example concerning significant problems areas such as Chiradzulu and the low therapeutic coverage rates.
4t Appendix I Report on meeting with WR We discussed how the country office could support the programme, once APOC funding ceases. The following emerged: WHO is strongly promoting a sector-wide approach among donors, in providing funding. Powerful donors like DFID are doing the same. This means that donors coordinate what they give and how they give it, to fit in with a corrmon plan. Vertical programmes are to be harmonised. Donations are then pooled and used in an integrated way. The WR's ffice will be donors to such WHO Malawi has recently prioritised its areas of concem: controlling HIV/ AIDS and TB; disease surveillance and outbreak management; rolling back malaria; child survival; and safe motherhood. All programmes aimed at these priority problems require a well functioning district health service as their vehicle. The WR's offce will therefore be working hard to promote such health service developmenr. I was able to point out to him that the district EHTs will be key players in the implementation of activities aimed at achieving these objectives. The EHTs require transport, primarily motorcycles and pushbikes, as a prerequisite for any project they try to implement. I also pointed out that the integrated use of available transport at this level is already a reality - which fits in with the 'sector-wide approach' insistence on resources between WHO has a tradition of supporting and husbanding the smaller technical programmes, such as trypanosomiasis, schistosomiasis and onchocerciasis. The WR's office in Malawi will continue to such both and liticall Finally, WHO Malawi has a biennially determined budget. In the 2002-3 version onchocerciasis control was allocated a sum of $ 10 000 (some of which has already been used). A feature of this budget is that it is almost entirely'projectised' - funding is allocated to carefully defined activities, and may not be used for anything else. However the present WR hopes to change that, and also to obtain extra-budgetary funding which is more flexible. The WR is clearly expecting onchocerciasis control to take its place among the recipients of his sources Detlef Prozesky 2002-1 1-18 Implication: Such integrated use of donations is exactly what the CDTI programme needs, e. in the area of Implication: Such strengthening of EHTs' work is exactly what the CDTI prograrnme needs - it uses the same infrastructure. : NOCP is assured of both technical and advocac in the circles. Implication: NOCP is assured of a small sum, which can be used to fund operationally critical activities.
Appendix 2 The district level feedback/ planning workshop I Timetable for feedback meeting - Lutchenza township Date: Venue: Master of ceremonies: Rapporteurs: 43 Tuesday 19 November 2002 Nali Motel L Sitima FNkhoma&EJere Acdvltv Time Frcllitetors Registration of participants 09.30 - 10.30 a.m. E Jere Opening prayer 10.30 - 10.35 a.m. F Nkhoma Introductions - participants 10.35 - 10.45 a.m. P Tambala/ L Sitima Introduction to the workshop programme 10.45 - 10.50 a.m. L Sitima/ S Munthali Introduction to CDTI 10.50 - 11.20 a.m. P Tambala/ L Sitima Evaluation obiectives - what is sustainability? 11.20 - 11.50 a.m. D Prozesky Methodology and limitations 1 1.50 - 12.00 noon S Baine Summary of findings at community level 12.00 - 12.20 p.m. S Munthali Lunch 12.20 - 01.20 p.m. C Kwizombe Summary of findings at health centre level 01.20 - 01.40 p.m. S Mugisha Summary of findings at district level 01.40 - 02.00 p.m. D Prozesky Tea break 02.00 - 02.15 p.m. E Jere Discussion: the way forward (with group work following) 02.15 - 05.00 p.m. S Mugisha/ S Baine Closing prayer 05.00 - 05.05 p.m. F Nkhoma 44 2. Workshop attendance Name District Title Address Kankhuni BNZ Hausi H Chinula PO Nyirongo NGC Juma IPM Mkolombwe RC Namukutiama Mphepo CE Kandiero J Katunga JL Nyundo PJ Yali S Kabudula Boko PEN Raviwa FF Zganbo A Mhone M ZaingaFLD Theu MMK Nguwo RZ Ndhlovu A Khumbo C Makonde B Banda L Chunga E Moyo MP Mbadzo PJF Chisale TL Chunga Blantyre Blantyre Blantyre Blantyre Mulanje Mulanje Mulanje Blantyre DCs Phalombe Phalombe Phalombe Phalombe Chiradzulu Chiradzulu Chiradzulu Chiradzulu Chiradzulu Chiradzulu Mulanje Mulanje Phalombe Phalombe Mulanje Mulanje Chikwawa Chikwawa Chikwawa Chikwawa Chikwawa DOC DEHO Pharmacy Tech. Accountant DOC Director of Admin. H/A IEC Human Resources DOC DEHO CDTI Supervisor DHO HRMO Pharmacy Tech. Ag. DEHO PDHSA Deputy DHO DEHO AEHO EDO DC PHSA IEC Officer Ag. IEC Oflicer Ag. Pharmacist CO PHSA DEHO PIBag66, Blantyre PIBag66, Blantyre PtBag66, Blantyre P/Bag 66, Blantyre Box227, Mulanje P/Bag 9, Mulanje Box 50, Thuchila P/Bag97, Blantyre Box 79, Phalombe Box 79, Phalombe Box 79, Phalombe Box 79, Phalombe Box 21, Chiradzulu PlBag l, Chiradzulu Box 21, Chiradzulu Box 21, Chiradzulu Box 21, Chiradzulu Box2l, Chiradzulu Box227, Mulanje Box227, Mulanje PIBag32, Phalombe PIBag32, Phalombe Box227, Mulanje Box227, Mulanje Box 32, Chikwawa Box 32, Chikwawa P/Bag l, Chikwawa Box 32, Chikwawa Box 32, Chikwawa 45 Appendix 3 District sustainability plans Blantyre Phalombe Shortcomings What can we do to improve? Who must do it? By when must we have done it? Priority No ONCHO activities in DIP Include ONCO in DIP. DHMT March 2003 I Uncosted annual work plans I Costing of annual programme. Realignment to the end of financial year. I DOC DOC ltt week Dec 2002 Itt week Dec 2002 3 Other members of the DHMT are not aware of Oncho plans Sharing the plans with them. DEMO Ongoing 2 Mectizan bypasses the District Pharmacy Mectizan to pass throueh pharmacy NOTF Immediately 4 Shortcomings What can we do to improve? Who must do it? By when must we have done it? Priority Leadership To orient the TA DOC January 2003 All Mectizan supply To store the drug at Pharmacy for safety and proper storage. Pharmacist January 2003 Training and HSAM Refresher courses for CDDs be done for 2 days: . More materials. , More sessions of H/E in communities and FVC. DOC HSAs February 2003 Dec.-April2003 Finances/ funding To incorporate the prograrnme - HMIS. NOTF By 2003 Human resources To increase training of HSAs to 4 days and include data management. NOTF By 2003 Coverage Intensification of IEC in the impact area. Promotion materials including: * T-shirts for CDDs; if possible for CDTI health workers too. * Carrier bags. IEC Officer Ongoing 46 Chilcwawa Mulanje Shortcomings What can we do to improve? Who must do it? By when must we have done it? Priority DHMT lacks of knowledge on oncho Briefing/orientation DHMT DOC Dec.2002 1 Lack of knowledge of oncho. by the DC and local leaders (DDC). Orient DDC DOC Dec.2002 2 Inadequate HSAM materials. Procure and distribute adequate HSA materials. DHO Dec.2002 - Jan. 2003 5 CDDs lack of knowledge on oncho drugs' side-effects. Training/refresh them on drugs side effects. H.S.A By early Dec. 2002 3 Lack of information on the costed budget for oncho. Conduct integrated planning (DHMT and NOCP team). DHO March 2003 4 Supervision (due to lack of reliable transport at district level) Maintenance of broken down motorcycles. DHO/ NOTF Dec.2002 - ongoing 6 National level Conduct quarterly supportive national supervision. NOC Dec.2002 - ongoing 7 Shortcomings What can we do to improve? Who must do it? By when must we have done it? Priority DHMT is not tully supportive of oncho. DHMT should tully support Oncho. DEHO/ DOC Dec.2002 Medium Broken down motorbikes (5). Maintain. NOT/ DOC Jan.2003 High Low motivation of CDDs. Provide them with incentives i.e. umbrella, T-shirts and frequent visits. NOTF/ DOC supervisors Ongoing Medium Poor calculation capabilities of HSAs/ CDDs. Train H.S.A and CDDs. NOTF/ DOC April2003 Hish Inadequate information on oncho. by communities Introduce multi-layer dissemination of oncho. messages. DOC/ NOTF On \going High 47 I Appendix 4 Suggested changes to the o Sustainability' instrument The following changes are suggested, for the sake of clarity, and also to make the instrument suitable for use at the end of both Year 3 and Year 5 of APOC funding: Instrument I . Indicator 2,3, bullet 'd': 'There should be evidence of action taken, based on recommendations in the reports of previous monitoring exercises.' . Indicator 4.1, bullet 'a': 'Staff at this level should routinely only train staff at the level immediately below it.' . Indicator 5.1, last bullet: this bullet should be split into two. . Indicator 6.2, bullet 'b': 'The amount that the goverrrment has budgeted in one or more specific onchocerciasis control budget lines (e.g. current and capital) should be increasing yearly, as a proportion of total expenses. By the end of Year 5 of APOC funding the bulk of onchocerciasis control expenses at this level should be met from govemment funds; I the end of Year 3 at least half of it.' Indicator 6.3, bullet 'a': 'Project management at this level should be aware of the shortfall, if one exists, and of its size.' Indicator 7.4, bullet 'c': 'If it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in these donor organisations (end of Year 5). or neeotiations should have started (end of Year 3).' Indicator 9.1: 'Check whether the projects in the country (or districts/ LGAs in the proiect) have a satisfactory geographical coverage rate.' Indicator 9.2: 'Check whether the projects in the country (or districts/ LGAs in the proiect) have a satisfactory therapeutic coverage rate.' Indicator 9.2, bullet 'a': 'All projects in the country (or districts/ LGAs in a project) should have a therapeutic coverage rate of 65oh or higher.' Instrument 2 Indicator 6.2, bullet 'b': The amount that the govemment has budgeted in one or more specific CDTI budget lines should be increasing yearly. By the end of Year 5 of APOC funding the bulk of CDTI expenses at this level should be met from local govemment funds; by the end of Year 3 at least half of it.' Instrument 3 Indicator 3.2: Check whether health service staff at this level is routinely and efficiently supervising CDTI activity at the communities on site in an integrated manner.' Indicator 6.2, bullet 'c': 'The proportion provided by the government (FLHF and./ or district/ LGA levels) should be the major one by now (end of Year 5) or coverine at least half of expenditure (end of Year 3).' Indicator 7.4, bullet 'c': 'If it is planned that replacement will be from non-government sources, written commitment for this should have been obtained at the highest level in I I I tr 48 these donor organisations (end of Year 5). or nesotiations should have started (end of Year 3).' Instntment 4 lndicator l.l: bullets re-written as follows: a. CDDs choose visiting times and routes which will make the work less burdensome. b. CDDs arrange with the community leadership for help with specific problems, such as families who are not willing to participate in the prograrnme. c. CDDs carry out census and distribution during the same visit (using this census data for the following year's order). 49 Appendix 5 Timetable of the evaluation visit Date Activity Friday 8 November 'John the Baptist' prepares sample, logistics.I Saturday-Sunday 9-10 November . Final planning of methodology and research plan. . Joint discussion of instruments. Monday 11 November Final preparations: transport; money; stationery; communications. Data collection: national level.(Blantyre). Tuesday 12 November Data collection: district level (4 districts).I Wednesday-Friday 13-15 November . Data collection: 2 HCs, 4 villages (in each of 4 districts). . Friday afternoon: visit to Chiradzulu district Saturday-Sunday 16-17 November . Data analysis: district, FLHF, community levels . Report writing for these levels. Monday 18 November , Data collection: national level (Lilongwe and Blantyre) . Drawing up recommendations . Preparing for following day's workshop Tuesday 19 November t District level report back workshop at Lutchenza. Wednesday 20 November Data analysis: national level. Report writing for this level. Report back meeting: national level Thursday 2l November Working on reportI Thursday-Friday 2l-22 November Evaluation team departs for home.a