Mid-term assessment of the sustainability of the Southwest II CDTI project, Southwest Province: April-May 2003

World Health Organisation African Programme for Onchocerciasis Control Mid-term Assessment of the Sustainability of the Southwest II CDTI project, Southwest Province, Cameroon April-May 2003 Eleuther Tarimo (Team Leader) Uche Amazigo Edward Banoba Albert Eyamba Musa Obadiah Joseph Okeibunor RECU 0 7 JUIL. 2003 APOC/DIR ITABLE OF CONTENT TABLE OF CONTENT.... Abbreviations/ acronyms... Acknowledgements............ Executive summary 1.0 INTRODUCTION 2. I SAMPLING........... 2.2 SOURCE OF INFORMATION 2.3 ANALYSIS ........... 3. O EVALUATION FINDINGS 3.1 SUSTAINABILTY AT PROVINCIAL LEVEL:.... 3.2 SUSTAINABILITY AT DISTRICT LEVEL............. 3.3 SUSTAINABILITY AT THE HEALTH AREA LEVEL 4 .2 J A .8 2.0 METHODOLOGY. ,9 9 ...10 ...10 ..1 I ,11 .t6 22 3.4 SUSTANABILITY AT THE COMMUNITY LEVEL................... ,,,,27 0 CONCLUSION ..._........31 4.I GRADING THE OVERALL SUSTAINABILITY OF SW II CDTI PROJECT.3I 4.2TWO FEEDBACK/PLANNING MEETINGS:....... ........35 4.3 THE WAY FORWARD. ..............36 5.0 ANNEXES 40 2Abbreviations/ acronyms APOC African Programme for Onchocerciasis ControlCBCH Chief of Bureau for Community HealthCBFA Chief of Bureau of AdministraUon and FinanceCBH Chief of Bureau of HealthCDD Community Directed Distributor (of ivermectin)CDTI Community Directed Treatment with IvermectinDHMT District Health Management Team DMO District Medical Officer DTS District Temporary StaffHA Health Area HIPC Highly Indebted and Poor Countries Initiative HSAM Health Education Sensitisation Advocacy and MobilizationKm Kilometres MEDP Manager, Essential Drug Programme MOH Ministry of Health NGDO Non-Governmental Development Organisation NOTF National Onchocerciasis Task Force PCSAFA Provincial Chief of Service for Administration and Financial Affairs PCSCH Provincial Chief of Service for Community Health PCSHM Provincial Chief of Seruice for Hospital MedicinePCSP Provincial Chief of Seruice for PharmacyPDPH Provincial Delegation of Public HealthPHC Primary Health CareOPC Onchocerciasis Programme Co-ordinatorSAE Serious Adverse EventSDO Senior District OfficerWHO World Health Organisation JAcknowledgements We would like to thank the following persons and organisations for their contributions and assistance towards the successful execuUon of this assignment: . The Director, Dr. A. Seketeli and staff at APOC Headquarters in Ouagadougou for making available the necessary financial and logistic requirements for the success of this assignment. . The WR, Yaounde and her staff contributed to the smooth functioning of the evaluation team. . Dr Ntep and her team facilitated the smooth take off of the exercise in Cameroon . Staff of the PDPH Buea: They provided information and prompt support. . Mr Akoa, of the Airpoft health Department. He provided assistance in the smooth clearance of the external evaluation team members at the Airpoft . Country Representative, Sight Savers' International, Dr. Rosa Befidi, and her staff, Health workers and community members in the Mundemba, Fontem and Mamfe Health Districts provided information and contributed to the success of the mission. . Dr Andrew Atabe, Sight Savers International Programme Officer, pafticipated as more than a guide, in the collection and analysis of data as well as writing of this report 4Executive summary Introduction The SWII CDTI project was launched in March 2000 and is therefore in its mid-term year of agreed funding from APOC. A team of external evaluators from Nigeria, Tanzania, Uganda, Cameroon and APOC management carried out an evaluation of the sustainability of the project between April 21 and 7 May 2003. The evaluators were charged with three tasks: . To evaluate the sustainability of the project. . To present and discuss results with the PDPH authorities and NGDO partner . Support the Provincial level, endemic Districts and Health Area personnel in developing post APOC sustainability plans, using guidelines for sustainability planning recently developed by APOC (and pre-test the guidelines). Information was gathered from the review of relevant documents and repofts. Interyiews were held with the Provincial Delegation team, District Health management teams, and chiefs of post in the health areas, community leaders and members as well as CDDs. The evaluation team held community meetings in all the communities sampled. Findings The project operates in a very difficult and harsh environment. Accessibility to many communiUes in three of the five Districts is by trekking, which could be as long as 10 to 50 km. Some communities are inaccessible for almost nine months in a year. The evaluation team notes these are major constraints in project implementation. All communities that have been identified as needing treatment are receiving it except for a few difficult-to-reach communities in Mamfe Health District. Geographical coverage is good. Therapeutic coverage for the project was low and decreased from 44.760/o in the first year to 33.260/o for the second year. More than 70o/o of communities had therapeutic coverage of less than 650/o. Provisional data for the third year show improved coverage but only two of the five districts have coverage of 650lo and above. Communities expressed willingness to continue with treatment for as long as it is necessary. The recent abolition of cost recovery and the payment of compensation to CDDs by government as well as intensification of HSAM to address rumors and fears of side effects (which had adversely affected project implementation in the first two years) are the main factors leading to the increase in coverage for the third year. Communities have not been empowered to play a leading role in CDTI. For example, communiUes were not aware of their roles and responsibilities in supporting CDDs; most respondents indicated that they had not been informed of these responsibilities. Apaft from the selection of CDDs, communities'choices (for example on method and period of treatment) were not respected by the project. Community ownership of the program is lacking. It was obvious, from the meetings evaluators held with communities, that no practical steps have been put in place to ensure sustainability of CDTI at the community level. The evaluation Team concludes that CDTI at the community level is not 5sustainable, To reverse this situation HSAM and related activities should be intensified immediately to promote community ownership of CDTI' The recent devolution of authority and CDTI fund management to Health Districts is a positive initiative. However, this devolution needs to be extended further to Health Area levels. Both the District and Health Areas should be empowered to initiate activities according to the dictates of their unique environments. Strong centralized direction of the project from the NGDO office in Yaounde was justified in the initial stages of the project but its continuation is not good for sustainability. Recommendations of the evaluation team are for Districts and Health Areas to take leadership and ownership of CDTI and develop realistic work plans and budgets. Funding for CDTI by government (at the Provincial and District levels) has been minimal and goes mainly for payment of salaries, provisions of stationeries, equipment maintenance and recently compensation for CDDs, following the abolition of cost- recovery. According to the Project proposal, agreed funding by partners for the first three years (respectively) in USA$ was as follows: APOC 235,857- LL7,l82 and 92,499; SSI 45,857- 32,350 and 41,288 and MOH L06,723- 47,543 and 47,387 ' MOH suppott has not been forthcoming as agreed. Over dependence on external sources of funding does not enhance project sustainability. The project finance office is located in the NGDO office in Yaounde, about 320 Kilometers from the project. The process of fund management including transfer of funds is very complex. Though there is high-level political commitment as evidenced in the involvement of the Provincial Delegate for Public Health and other poliry makers in CDTI activities, the leadership role of the government is weak and unclear. Following advocacy by the evaluation team decisions to correct these deficiencies were reached by partners. Respondents during interuiews (at all levels) and community meetings perceived the project as that of SSL The role of the other partners (health system and community) was seen as limited to the execution of activities other than leadership. There is a structural dependence of the lower levels on the higher levels and the PDPH on the NGDO. The evaluation team was intrigued by the weakness of leadership and role of the Provincial level, given that the CDTI project is in the third year and that information from different sources, particularly that contained in the letter of Agreement between APOC and the Government indicates clearly responsibility of different partners. The staff attitude and level of superuision are satisfactory. There are enough financial resources to carry out activities but these are mainly APOC and SSI funds. This would have been considered to be seriously blocking sustainability if it were a flfth year project, but steps should be taken to secure counterpaft funding from government for the foufth year budget. Regarding transport and Mectizan supply, there are no plans for the replacement of vehicle and motorcycles in the future, and the current Mectizan supply system does not enhance sustainability of the project. Treatment coverage is low and unstable. 6In line with guidelines for grading the whole project, as less than half of the seven aspects are fulfllled, and less than half of the six critical elements satisfied, the evaluation team concludes that the SW II CDTI project is NOT MAKING SAflSFACTORY PROGRESS TOWARDS SUSTAINABIUW. The project has serious barriers to sustainability and requires, rethinking on the roles of paftners, appropriate and carefully targeted HSAM to inject the right sense of ownership of the program at different levels (particularly at the community level) and mobilization of sufficient support for the sustainability of CDTI post APOC. However, since this is a mid-term evaluation of the project (third year project) the criteria for judgment were modified and lowered to take care of the age of the project. Thus it was examined to see if at least two of the aspects and two of the critical elements of sustainability were fulfilled and satisfied respectively. The results show that the project is still not making satisfactory progress towards sustainability. Finally since the grading tools are new and being used for the flrst time, the evaluation team alsojudged the sustainability of the project through assessment of the nature and seriousness of deficiencies for each of the nine groups of indicators. The judgement reached was also that the project is not making satisfactory progress towards sustainability. Two FeedbaclVPlanning meetings, one for PDPH and the other for District Health Teams were successfully organized. The evaluation team facilitated the development of sustainability plans (for the 4h year) for the Provincial, District and Health Area levels using APOC guidelines recently developed. Selected pafticipants in the meeting were requested to remain behind for two days to complete the district plans. The provincial delegation indicated that both the provincial and district plans will be completed quickly and immediately forwarded to APOC. During the PDPH feedback session it was also agreed that the present situation whereby the Provincial Delegation is not in the 'driving seat' for the CDTI project is not acceptable and corrective action should be taken as a matter of urgency. All parties concerned should respect the leading role of government on the lines of the project proposal and the annual project agreements between WHO/APOC and Government. Finally, on financial contributions by the government for CDTI, the finance officer of NOCP/Cameroon informed the feedbac(planning meeting that the government has in the last two years provided financial contributions. These were used for REMO suryeys, production of IEC materials and equipment (including a computer and printer through NGDO to SWII project). These equipments are retained by the NGDO. The Provincial Delegation will be following up with NOCP to enhance support for the province from these and other sources mentioned by the finance officer including HIPC. Way Fonruard: The aim of advocacy, a preoccupation of the whole evaluation process was to go well beyond the identiflcation of problems, to facilitating decisions by partners for remedial action. Examples of such decisions include: Setting up of a simpler and more efficient financial management system of APOC funds; Transfer of project finance officer from the NGDO's office in Yaounde to Buea (project site); Enlargement of the provincial team 7with the addition of Provincial Chief of Service for Pharmacy and Provincial Chief of Service for Administration and Financial Affairs and enhancing ownership of CDTI at all levels. The opinion of the evaluation team is that empowerment of communities; the cornerstone of CDTI remains the biggest challenge. While it is true that communities may not be willing to express their true feelings and wishes to outsiders pafticularly those on short visits, the evaluation team is convinced that potentials of communities to play a leading role in CDTI have not been exploited. Resources for CDTI should give priority to action at the FLHF and community levels. PDPH should ensure that sustainability plans developed during the briefing and planning meetings are finalized, fonrrarded to APOC and implemented as a matter of urgency. PDPH should monitor closely implementation of decisions and recommendations agreed in the planning meeting to address deficiencies identified by the evaluation team and to take prompt corrective action. SSI has played a leading role in the development of the project and experiences gained to date will be of great importance to the implementation of the sustainability plan. APOC should follow closely the implementation of decisions and activities for the sustainability plan. Special attention should go to decentralized fund and program management, pattern and trend of budget allocation. While large investments at the prounclal level were justified during the development phase of the project in the first two years, the fourth year budget should see visibility and much more funds (greater o/o of the budget) going to activities at the community and health areas. 81. INTRODUCTION Southwest (SW) is one of the ten Provinces in Cameroon. Onchocerciasis is hyper endemic in the Province. In 1997 APOC approved support for Mectizan distribution in paft of the Province referred to as South West One (SWI), which covers nine Health Districts. In December 1999, APOC funding support was approved for the remaining five Health Districts in the Southwest province and this is implemented under the South West II (SWII) CDTI project. SWII project area has a population of 405,320 (1987-population census). Each village is headed by a Chief and a traditional council, which usually makes decisions. Farming, trading, fishing and hunting are the main occupations. AdministraUve Headquafters of the Province is at Buea. The province shares borders with Taraba and Cross River States in Nigeria. The SWII CDTI project covers 5 health districts, namely Akwaya and Mamfe in Manyu Division, Fontem in Lebialem Division and Ekondo Titi, and Mundemba in Ndian Division. Prolonged rain season, from mid-February to mid-December characterize the area. Most of the SWII project area is also characterized by dense and luxuriant equatorial forest in Ndian, Lebialem and most of Manyu. However, savannah vegetation characterizes the Akwaya region bordering with the Northwest and the Federal Republic of Nigeria. The area has a very harsh topography with many hills and valleys. This renders the terrain very rough making accessibility difficult. In most of the valleys run fast flowing streams, which provide good breeding grounds for the black fly, Simulium. According to the Project Proposal, which was launched in March 2000, there is an Onchocerciasis Coordinator at the Provincial level as focal point for CDn. DMOs act as coordinators at the District level. In the first treatment period, L85,874 persons were registered while 176,389 were registered for the second round. There are 319 hyper endemic and t47 meso endemic villages. The project has completed its third year of funding from APOC and the mid-term evaluation is being carried out as agreed by NOTF representatives and APOC management during the meeting of the representatives of NOTFs at Abuja Nigeria (in June 2002). Furthermore, after the evaluaUon two briefing/planning meetings were organized, one for PDPH and the other for District Health Teams. 92.O METHODOLOGY . Evaluation question: How sustainable is the South West II CDTI project? . Design: Cross-sectional, descriptive. . Population: The SWII project, including: PDPH; its NGDO paftner; its Districts with their District Health Teams; the Health Areas health management teams; the project communities/villages, their CDDs and the projecYs finance officer. . Sampling. Details of the sampled Districts, health areas and communities/villages are contained in Table 1 below. 2.1 SAMPLING A multi stage sampling approach was adopted in selecting the sample for the evaluation. First the average treatment coverage for two years was computed for each of the five health districts. Third year treatment was in the final stages when the evaluation started, provisional results for the five districts were available to the evaluation team during the briefi ng/planing meetings. Next, the average coverage rates of the five health districts were softed in an ascending order from lowest to highest. These were put into three strata. From each stratum, one health district was selected. The selection took into consideration treatment coverage as well as geographical distribution. In the third stratum, which had only one health district, the health district was selective purposively. Through random sampling Mamfe, Fontem and Mundemba Health District were selected for the evaluation. From each of the three sampled health districts, two health areas were randomly sampled and based on their treatment coverage rate one community/village with low coverage and another with high coverage were selected, Table 1: Distribution of Samples in Health Districts/Division, Health Areas and Communities: S/N Health District (Division) R* (Coverage Rate) Health Area Community/Villages (R* Coverage Rate) 1 Mamfe (Manyu Division) Low (34.1olo) Bachuo Akagbe 1. Mbio (50,4o/o) 2. B'Akagbe (13.8olo) Eyumojock 1. Eyumojock (46.70/o) 2. Mbakem (29.3o/o) 2. Fontem (Lebialem Division) Medium (45.2o/o) Fonjumentaw 1. Melockmelia (60.9olo) 2. Letia/Njila p (27 .2o/o) Menji 1. Fossong (87.0olo) 2. Forka/Nyieh (26.60lo) 3 Mundemba (Ndian Division) High (47.0olo) Mundemba 1. Bisingi Village (50.50/o) 2. Meangwe II (19.10lo) Pamol Ndian 1. Centre C-amp (74.0o/o) 2. Mana Camp (41.7olo) l0 2.2 SOURCE OF INFORMATION Information was collected from interuiews, verbal reports and documents. Various categories of people were interyiewed at the Provincial level (Provincial Delegation Team members, NGDO and project finance officer), District (DMO, CBH, CBFA), FLHF/health area level (Chiefs of post) and the community. In the communities the evaluation team interuiewed community chiefs, members, CDDs and held community meeUngs. Information was recorded on the evaluation instrument and discussed extensively before grading the level of peformance on the indicator. 2,3 ANALYSE Based on the information collected, each indicator is graded on a scale of 0-4, in terms of its contribuUon to sustainability. The average 'sustainability score' for each group of indicators is calculated, for each level. A graph was plotted for the level being assessed. The evaluators discussed qualitaUve description of problems, and likely suggestions for solving the problem deliberated upon. 2.4 ADVOCACY Advocacy was a crosscutting concern for the evaluation team. Interviews and discussions at each of the four levels (Provincial, district, FLHF and community) focussed on identifying issues; possible solutions and enhancing appropriate decisions. ll 3. EVALUATION FINDINGS 3.1 SUSTAINABILTY AT PROVINCIAL LEVEL: Fig. 1: SWll CDTI Project: Sustainability at the Provincial Level 4 3.5 3 2.5 2 1.5 1 0.5 0 3.5 3 2.5 2 .9 o =o ED(E o .*tt"$"".""""*"p .{*.nc o** ^-oC ."C"..C Groups of lndicators Planning (Moderate 2.5) There is an overall written plan for health seruices at the provincial level. There are also yearly plans that contain key elements for onchocerciasis control (e.g. training, superuision, Mectizan distribution, appraisal meetings etc.). Key paftners (Community members, NGDO and MOH staff) are involved in the planning of activities at different levels of the health systems. However, the roles of different partners are neither clear nor known to some partners. For example no one in the Province seemed to be fully informed as to why there is delay in availability of Mectizan to enable treatment to be carried out as planned (see also under Mectizan below). The implications are that the spirit of program ownership and sustainability is weak and core program activities may be weakened with trickle down effects on the levels below. The evaluation team feels strongly that available information from different sources particularly that contained in the lefter of Agreement between APOC and the government was adequate to enable the leadership at this level to play a more active role in CDn. There is urgent need for MOH to strengthen her national and provincial level teams and mechanisms to perform their required leadership role in CDTI implementation. 12 There was also lack of clarity on the roles of NOTF and the Provincial Delegation in CDTI. For example an NOTF finance officer informed the briefing/planning meeUng of the government's decision that CDTI will be one of the beneficiaries of funding under HIPC. He also outlined flnancial contribution made by the government to CDTI in the last two years. The Provincial Delegation was not aware of these developments and decisions! Monitoring and Superuision (Highly; 3.0) Program Coordinator, Provincial Chief Seruices of Community Health and the Delegate carry out monitoring and superuision of activities at the Districts level. Occasionally spot checks down to the health areas and communities are carried out with the aim of improving the peformance of activities. Sight Savers International (SSI) staff are also involved in the monitoring and supervision along with the provincial officers and occasionally undertake supervisory visits alone. Provincial level team has empowered staff at district level through training and joint field visits to supervise staff at the lower level (Health areas). Findings from the fleld indicate that the quality of superuision by this and other levels need to be improved upon. Mectizan Ordering, Procurement and Distribution (Moderatelyl, 2.0) Adequate amounts of Mectizan were available for treatment in the first and second years. The third year saw considerable drug shoftage, which has been attributed to the increase in Mectizan demands and uptake, following the abolition of user charges by the Government and intensified mobilization of communities. Immediate arrangements were made and the problem was solved in many communities. The evaluators were informed that quantities of Mectizan ordered were based on census data from the communities. However the evaluation team found out that many CDDs and staff did not know how to calculate Mectizan requirements correctly and paid little attention to this task. Apparently because of delays in receiving data on Mectizan needs from the field, the responsible agency for ordering Mectizan uses treatment result of the previous year plus 100/o to make the orders. Most communities preferred treatment is in the month of November, this timing has not been possible due to late availability of Mexican. There has been negligible use of skills and facilities under the manager of drug procurement and distributions program for the province. Training and HSAM (Moderate; 2.7) A cascading training approach is used. At the beginning of the project, SSI trained provincial and district teams together. District teams trained Health Areas (HAs) staff who in turn trained CDDs. Staff at the provincial level were concerned with the quality of training of CDDs by FLHF staff which include nursing aids with only nine months of professional training. Findings from the field showed that the District staff not been fully empowered by the Provincial level team (including the facilitating NGDO) to conduct effective training of the level below it. OPC should discuss this issue with the Districts and Health Areas staff concerned and undeftake appropriate actions, which will further empower the District level personnel and others as necessary. l3 Integration of CDTI Activities (Slightly; 1.0) Plans made available to the evaluation team indicate that programs under the Delegation are well integrated. But observations and discussions indicated that CDTI activities carried out by OPC are not implemented in an integrated manner; this is attributed to the large volume of work falling on one staff. However for sustainability of the project closer collaboration needs to be established with other programs like essential drugs, health education and community development. Integration of CDTI activities with other programs is a key indicator for program sustainability. Enhancement of integration by word and example has to come from this level of the project. Time should be made for dialogue and coordination with other programs particularly for fleld activities. Following advocary and discussions at the Provincial level, the Delegation decided to include PCSP and PCSAFA in the Provincial team. The two staff who had not been involved much in CDTI before played an active role in the briefing/planning meetings particularly in the development of sustainability plans. Financing/Funding: (Slightly, 1.3) Funds provided by APOC and SSI are generally adequate for the project. Funds released are also spent as planned and are well controlled. However the system of approval and release of funds is complex (involving extensive movements of documents between the project and SSI in Yaounde). Following discussions with partners agreement was reached to put in place a process that is sustainable, more realistic and less complicated for fund management as is the situation in many other countries. A primary account should be opened in the name of SW2 APOC/NOTF account as reflected in the agreement. Government contribution is in terms of the costs of staff involved in CDTI, no contribution has been made to finance operational costs outlined and agreed in the Project proposal. High-level government decision on this matter needs to be taken as a matter of urgenry. NOCP should advocate to the Minister of Health for the government to make financial contribution to CDTI on the lines of the agreement with APOC. Abolition of user charges is one of the two leading factors contributing to the high coverage in the third year. A portion of user charges, 32o/o was retained by CDDs and the rest went to CDTI fund and local initiatives. The government has made an undertaking to provide support to CDDs and anangements for implementation of the decision are at hand. There should be a deliberate effort, funding and mechanism to address gaps that exist following the abolition of costs recovery, e.g. the Oncho-CDTI reserve funds, funds for management of minor side effects, contribution to supervision at the Districts level etc. which account for 680/o of the previous costs-recovery funds. Transport and other Material Resources (Moderate; 2.3) Available transport is generally adequate and well maintained but there are no plans for replacement when APOC support ends. This deficiency needs to be addressed by the government. In view of the bad terrain of the project areas, vehicles provided by APOC should be replaced after five years of effective usage. The Government and APOC t4 should make arrangements for replacement of these vehicles before APOC fund ends to avoid disruption of CDTI implementation activities. Human Resources (Highly; 3.5) Human resources at this level are adequate and with appropriate skills. The staff at the Provincial level is stable and committed. Unfoftunately, supportive staff such as drivers do not have permanent engagement with the project and may occasionally affect program implementation. The evaluaUon team recommends engagement of this important supportive segment of human resources by the appropriate level of government. Secondly the evaluation team suggests that OPC should work closely with coordinators for the control of other diseases, Community Development and EDP. The evaluation team had the impression that the OPC was trying to do too much almost single-handed. Coverage: (Negligible, 0.0) Coverage was very low in the flrst two years 44.760/o and 33.260/o respectively. The low coverage has been attributed to: Poor HSAM (particularly inability to counter rumors on SAEs); User charges; Unavailability of drugs in times planed and difficult terrain (Oguran health area in Mamfe is a good example. Only one health staff, responsible for leprosy at the district level goes to organize and provide treatment, no project staff has been there). An addiUonal factor leading to low coverage in the second year was the adopUon of centralized treatment approach (which is unpopular with communities). Centralized treatment begins with two days for treatment followed by two days for mop-up; adequate time for treatment of those who had traveled out or living in their farmstead is not provided. Preliminary data seen at the briefing and planning meeting show that therapeutic coverage for the third year is expected to be comparatively high, but only two of the flve districts had coverage of 650/o and over. Key factors attributed for the achievement are the abolition of user charges and intensified HSAM. Recommendations for the provincial level Recommendation Implementation Planning The Province should draw the attention of all CDTI partners (communities; NGDOs; APOC and MOH), to their roles based on the proposal document and lessons learned to date in the implementation of CDTI. Prioritr HIGH Indicators of success, A directive issued to all paftners on their roles and responsibilities. Who to take action: PCSCH Dea dli n e fo r com pl etion: By end of June 2003. Monitoring and superuision There should be an element of quality assurance in place as a mechanism for assessing the skills and quality of peformance of the Districts level staff following training activity. Priority HIGH Indiators of success Improvement in performance of their task within the next treatment cycle. Who to take action Provincial team Deadlin e for completion: By the end of May 2004. 15 Mectizan Procurement and Distribution. It is recommended that the NOCP should ensure that the procurement, storage and timely delivery of Mectizan tablets be done within the existing health systems from the national level to the entire project areas. Secondly, a standardised method of Mectizan requirements estimation should be strictly followed. PrioriV HIGH Indicators of success Correct estimation and timely availability of Mectizan of mectizan. Who to take action: NOCP and the Provincial team. Deadline for completion: By December 2004. Financing/Funding: The evaluators are concerned that the Government has not fulfilled her obligations as highlighted in the project proposal and the endorsed letters of agreement for the project implementation. It is recommended that government take action to comply with the obligation to ensure sustainability of CDTI whose success is of great importance to the people of Cameroon. Prioritlt HIGH Indicatorc of success Who to take action: NOCP/Provincial Team Deadline for completion: Transport and other material resources To ensure sustainability of CDTI, government and APOC should ensure replacement of transport before APOC support ends. Prioittt HIGH Indicatorc of success Who to take action: NOCP/Provincial Team Deadline for completion: Coverage: Coverage has been generally low since the project inception, although the third year has seen improvement over the first two years. The level of coverage should be raised and maintained to ensure maximum benefit to the eligible population. Prioittt HIGH Indiatorc of success The geographic and therapeutic coverage of the project. Who to take action: NOCP/Provincial Team Deadline for completion: Bv Mav 2004. t6 3.2 SUSTAINABILITY AT DISTRICT LEVEL Fig.2= SWI! CDTI Project Sustainability at District Level 4 3.5 !t .9 o =oE"(5 o 1 3 E 2 .5 1 .5 0 2 0 .-$o"'t".-""t""ce{.*"f*e\"$."d Groups of lndicators Planning (Highly; 3.0) All health districb have CDTI plans of action with key activiUes for each of the three years that treatment has been carried out. These plans were drawn up with information from the health area staff. This information included the timing of Mectizan@ distribution preferred by communiUes. Comprehensive plans of all the health programs were supposed to be available but in one district these could not be produced for the evaluation team to assess among other things the level of integration of CDTI activiUes with other district health programs. Leadership (Moderate; 2.0) The District Health Management Teams accept CDTI as their responsibility. DMOs are in- charge of CDTI activities just like any other health programmes within their districts. However, many decisions such as timing for availability of drugs and of their distribution are made at higher levels, the NGDO and the province. To begin CDTI activiUes health districts have to obtain directives from the province. They then plan their activiUes accordingly and this also depends on the availability of funds from APOC and SSL Such dependence on the directives from the NGDO and Provincial levels makes it difflcult for Districts to take on their leadership roles and weakens the ownership and sustainability of CDTI at this level. Monitoring/Superuision (Moderate; 2.7) Repofts on CDTI activiUes are channelled to the provincial level entirely within the existing government system. These reports are transmitted together with reports of other district health programmes. In Fontem, all health repofts, including CDTI reports l 3.5 3 2 1.5 1 17 are transmitted to the provincial level through a private travel agency but paid for with APOC funds District Temporary Staff (DTS) are employed in some areas and paid by SSI to transmit CDTI reports from communities and health areas to the district. Superuision checklists were not found at this level making it impossible to verify the level of integration of CDTI with other district health activities in the supervision checklist. It was also found in some health districts that health district staff members go beyond the health area level and carry out superuision in the communities. Mectizan@ Ordering, Procurement and Distribution (Slightly; 1.5) Mectizan@ is managed in an uncomplicated system. It is received or collected from the Province by the DMO. But in either case transport for collection is paid for by APOC funds. The DMO then informs health centre chiefs of post to collect their allocations. Mectizan@ is not collected and controlled through the government-established system as other drugs. Mectizan is stored and managed in the office of the Chief of Bureau Health with vaccines, separate from essential drugs in District Hospital Pharmacy. While the current method may be effective and uncomplicated for the personnel at this level, it weakens integration of the management of this aspect of CDTI in the overall health programmes in the Districts and is not good for sustainability. The quantity of Mectlzan@ requested was not received in the districts in one instalment. There was late arrival of drugs and in insufficient quantities in some districts. Ordering forms exist but were not used. Quantity of Mectizan supplied was not based on census records from the health areas rather this was determined by the higher levels. Training and HSAM (Moderate; 2.7) District teams routinely (every year) train Chiefs of Post, who in turn train CDDs. In some cases district staff use appraisal meetings after treatment to determine training needs. Resources for training are efficiently used. Advocary is done for all health programmes at the level of the District. Nurses are taught on how to carry out health education and mobilisation at their own level. Intensification of health education and mobilisation, during the last distribution, led to improved coverage in two districts. Advocacy and sensitisation for district administrative and political leaders is weak. District (municipal) councils do not plan for or release funds for CDTI activities. Finally, DHMTs have not empowered FLHF staff adequately to identify training needs of CDDs. IEC materials for training, health education, sensitisation and mobilisation exist in health districts but not in adequate quantities. Financial (Slightly; 1.3) All CDTI activities were undertaken with whatever funds were made available by APOC and SSI. Approval for expenditure is given based on the work plan drawn up during appraisal meetings. In some districts DMOs examine the financial records for balances. There is no evidence of cost reduction and containment at this level and the district health staff lack knowledge of how much funds are likely to be available for CDTI activities in the coming year. r8 There is no government budget line for CDTI. Government contribution is mainly for paying salaries of health personnel and provision for running credits to the health facilities. These credits are allocated to all health districts irrespective of their particular needs and are for all health programmes of the health district. Funding in cash for CDTI implementation only comes from APOC and SSI. Staff at this level is not aware of budgetary deficits since they just make use of whatever is provided. Transport and other Material Resources (Slightly; 1.8) There are adequate functional motorcycles to carry out CDTI activities in all districts. Health districts usually services project motorcycles with district 'credif funds. Motorcycles at FLHF level are used for all health programmes. Some health centres are also allocated government credit funds, part of which is used for maintenance and repairs of these bikes. The users in some districts meet maintenance costs themselves. APOC and SSI provide spare pafts yearly. In all the health districts logbooks are not in use. There are no control mechanisms for management of CDTI project motorcycles. There is no routine maintenance schedule and there are no plans for replacement of these motorcycles when they get out of use. Human Resources (Highly; 3.5) Health staff at this level is adequate and stable; most of them have stayed in their present position for 3 to 10 years. Staff is committed to CDTI work. Their salaries are paid regularly. However there is litfle organised in-service training (for health seruice) available to health personnel at this level. Coverage (Slightly; 1.5) All the 40 Health Areas of this p@ect area are covered but not all communities identified by REMO are currently under treatment. This is as a result of Loa loa co- endemicity and difficulty of access to some communities making monitoring and management of side effects very difficult. Therapeutic coverage for the first two years was generally low. Less than 70o/o of the communities had therapeutic coverage of 650/o and above. Only 13.60/o and 11.20lo of the communiUes had more than 650lo coverage in the first and second year treatment respectively. This was attributed to a number of factors, namely fear of side effects, cost recovery and poor sensitization. According to one of the SDOs, "the problem with CDTI is the level of health education and sensitization. There are so many rumors about side effects and nothing is done to give correct information" Preliminary results from the communities visited showed that treatment coverage for the third year has improved considerably. Key factors leading to increase in treatment coverage are shown in the figure 1 below. l9 1 KEY FACTORS LEADING TO INCREASE IN TREATIUENT COVERAGE DURING THE THIRD YEAR Fr ,,,$.i, , '." ';i,'rr.v-'r . Empoweffnent has been limited to District levels o The evaluation team strongly recommends further devolution of authorities to health areas and communities to enhance community ownership and sustained high treatment coverage tk .Ifid €w 6frtf df b W 20 Recommendations at District level Recommendation Implementation Planning All health districts should have comprehensive work plans showing that CDTI is integrated into the other health programs of the district Priority, Hlqh Indiatorc of success, Availability of integrated work plans in all health districts Who to take action: DMO Deadline for completion Durinq the provincial appraisal meetinq Monitoring / Supervision All transmission of reports should be paid for with funds from the government partner. Priority High Indicatorc of success Cost of transmission of reports not charged from APOC/SSI funds Who to take action: District Medical Officer Dea dli n e for com p I etion: By next round of CDTI activities Monitoring / Superuision Integrated supervision check list should be established and utilized in all health districts Mectizan@ requests and supply should be based on FLHF and community requests, which in turn should be based of census records. Prioritu Medrum Indiatorc of success, Availabi I ity of i ntegrated superuision checkl ist in all districts Who to take action: DMO and PCSCH Deadline for completion: By onset of next year's CDTI activiUes Mectizan@ Procurement and Distribution Mectizan@ supplies should be collected and controlled through the government established system of drug procurement and supplies. Priority Medium Indicators of success Mectizan@ managed by the existing qovernment druq procurement system Who to take action: Provincial Deleqate of Public Health Deadline for completion: By next round of CDTI activities Training and HSAM HSAM should be intensified at district level. The district management teams should empower the FLHF to identify training needs Prioritr Medrum Indicators of success, Availability of funding from councils and other stakeholders other than APOC and SSI. Identification of traininq needs by FLHF Who to take action: DMO Deadline for completion: By onset of next year's CDTI activities 21 Financial Government funding of CDTI activities should increase. A budget line for CDTI should be created at this level PrioriU HIGH Indicatorc of success Increase of government funding and creation of budget line for CDTI at district level Who to take action: NOTF, Provincial Deleqate of Public Health Deadline for completion: 2004 fiscal year Transport and other Material Resources Log books should be used routinely in all districts and they should contain routine maintenance schedule of vehicles Necessary health education materials should be made available in all districts in adequate quantities PrioiM HIGH Indicatorc of success Availability of used log book in all health districts Who to take action: DMO Deadline for completion: Bv end of Mav 2003 Human Resources In-service training should be organized for health staff of this level Prioittt MEDIUM Indicators of success Existence of schedule / reports of in-seruice traininq of district staff. Who to take action: Provincial Deleqate of Public Health Deadline for completion: Bv Januarv 2004 Coverage Therapeutic coverage rate should increase to at least 55o/o in all communities of the health district. Geographic coverage should increase to 100o/o in all the health districts Prioittt HIGH Indicatorc of success, Hiqh coveraqe rates in treatment repofts Who to take action: District Medical Office, Chief of Bureau for Health Deadline for completion: 4h Year of CDTI 22 3.3 SUSTAINABILITY AT THE HEALTH AREA LEVEL Fig. 3: SWll CDT! Project: Sustainability at the Health Area (FHLF) Level t .9 o =oE"(E o 4 3.5 3 2.5 2 1.5 1 0.5 0 .*o'foot\u.up es o.""""^**t& ."d"".C Groups of lndicators Planning (Fully; a.0) There were detailed written yearly plans for CDTI implementaUon. CDTI is paft of the routine activities of the health area personnel. CDTI activities were also integrated into the comprehensive health plan for the health area. The plan for CDTI is drawn at the District level and the health center staff adapts it for their specific usage. Leadership (Highly; 3.0) Health staff considers the CDTI as community program and theirs. Plans were drawn together with the DMOs at this level. An appraisal meeting is organized and facilitated by the District Health Management Team (DHMT) for the health areas every year. During the appraisal meeting, health area staff develop action plan for specific CDTI activities on the basis of a framework of CDTI already determined by the DHMT. The specific activities in the work plan of health area staff included: training of CDDs, mobilization and sensitization of communities, superuision during distribuUon, follow-up on cases of side effects, repofting etc. The action plan for CDTI activiUes therefore are not necessarily based on the prioriUes that are determined by the chief of post rather on the prioriUes set up by the upper levels. The District gives the directives for action. The Health Area Management Team does not take iniUative to act except when directed by the DMO. The directives from the upper levels therefore undermine the leadership role of this level l I 23 Monitoring and Supervision (Highly; 3.0) Reports are submitted to the DHMTs by the Health Area staff (nurses). When it is possible repofts on all health programs, including CDTI are submitted jointly to the DHMT office. The reporting process is within the government repofting system but the costs are covered with APOC funds. This is contrary to the signed Memorandum between APOC and the Government of Cameroon. Health workers superuise treatment activities throughout the distribution period and after to monitor side effects. Supervision is carried out routinely. Nurses, CDDs and community leaders carry out sensitization of communities. While the focus of supervision is on CDTI other health problems are addressed when they arise. Nurses manage side effects. Implementation problems are reported to village chiefs. Assistance is given to weak and unskilled CDDs. In each district, a District Temporary Staff (DTS) paid with NGDO funds delivers reports from health centres to the District Health Management team office. Mectizan Procurement and Distribution (Highly; 3.0) The quantity of Mectizan requested was received in batches in many health areas except in Fontem (Health Area) where sufficient Mectizan was received on time for the past three distributions. Forms for ordering Mectizan were available at this level but not used. In some Districts, front line health facilities (health areas) take Mectizan to the communities, especially remote ones. Other communities collect drugs from health areas. Health areas were required by higher levels to return Mectizan immediately after distribution. This most unfoftunate decision made it difficult for absentees to be treated Mectizan is not stored and managed within an established functional system (health area pharmacy) provided by the government. The current system though effective may not be sustainable. Chief of Post collects Mectizan from District but with APOC funds Training and HSAM (Moderate, 2.5) Every year before treatment nurses are retrained routinely by the DHMTs and CDDs by nurses and other health area supervisors on impoftant CDTI topics. In few Districts training emphasize areas of weakness identified during the appraisal meeting. The evaluation team recommends that in the fourth year of project implementation it will be crucial to adopt a targeted approach in retraining of chief of posts. Training needs assessments should be conducted by DMOs to identify areas of weakness prior to retraining session. The education background of the chief of posts should be considered a component of the assessment. For example, for a two-day tralning, half a day may be used to discuss different aspects of CDTI. The rest of the training sessions should be devoted to retraining pafticipants on specific weak areas such as record- keeping, accurate census and correct dosage. Similarly, chief of posts should be taught to retrain CDDs using the targeted approach. In a few and exceptional cases HSAM is based on findings from the appraisals after distribution. After the second distribution and in preparation for the third, some Health 24 Areas organized intensified HSAM to increase coverage and this among other things led to an increase in coverage rate in the third year. Chiefs of post in some cases identify situations where communities lack information and informed them during communities general meetings. HSAM activities are conducted routinely every year because of conUnued reports on refusals but the evaluation team thinks that these (activities) could be organised in a more efficient and targeted manner. Communication strategies and HSAM materials were found grossly inadequate and insufficient. The evaluation team is of the opinion that these critical elements and indicators of sustainability deserue immediate attenUon of program management at national and provincial levels. Financial (Negligible; 0.5) Government's contribution to CDTI activities at this level is insignificant in relation to the expected and defined role of government in the program. All funds used for the different activities (e.9. training/retraining of CDDs, collection of drugs and superuision of distribution) were from APOC Trust Fund and SSI. Chief of posts had no idea about the cost of CDTI activities and in some cases did not participate in budgeting of activiUes at this level. Funds were utilized as received from the DMO. Chiefs of posts were not aware of the sources of funds. Costs for activiUes were not clearly defined at this level. Transpoft and Other Material Resources (SlighUy; 1.8) There are 40 health areas with 52 motorcycles, one motorcycle per health area provided by APOC (20), SSI (15), MoH (6) and 11 for other programs (leprosy, G-fZ, SOWEDA). Transpoftation is adequate at this level. However, because of the very difficult terrain in the Fontem and Mamfe Health Districts, it may be necessary to provide replacements earlier than in other districts (five years). Motor rycles, are used for all health programs (but priority is for CDTI). Government financial contribuUon is negligible and grossly inadequate to cover the running costs for the motorrycles, expect in few health areas where the Motorcycles are maintained from health centre funds. APOC and SSI provide new tyres and tubes for replacements every year. When motorrycles break down health area staff in some cases repair them with private funds. Training and HSAM materials are not sufficient. It is hoped that government will provide stationeries for HSAM.The evaluation team found an average of one poster per village as grossly inadequate to have an impact on community perception and ownership of the program. Transport is not well managed. In all sites, log sheets though available (and were used in the past) are not being used. Management at health area is aware of the need for replacement of motorcycles, but there are no plans in place for replacement. Human Resources (Moderate; 2.0) Staff at the health area level is'stable'; they have been at the facilitiesfor 4-L2 years. Staff in general, have good knowledge on CDTI, and possess the necessary skills in 25 monitoring of side effects and superuision of CDDs. However, staff at this level is not adequately skilled in HSAM and training. In particular, they lack adequate knowledge for community mobilization and sensitization strategies. In some health areas voluntary health staff were used as superuisors and paid with APOC funds. Coverage (Fully; a.0) All the communities visited, which were identified by REMO to be hyper/meso endemic were under treatment for the past three years Recommendations at First Line Health Facility level ImplementationRecommendation Prioitvt MEDIUM Indiatorc of success Health Area staff initiate action plans based on identified needs of their communities. Who to take action: District Medical Officer (DMO) Deadline for completion: Before the next distribution Leadership: District Health Management Teams should immediately retrain health area staff and empower them to prepare and implement their own action plans according to the needs of the communities in their catchment areas. Priority MEDIUM Indicatorc of success Government and other partner besides the NGDO and APOC pay DTS. Who to take action: DMOs Deadline for completion: By the end of the next distribution Monitoring & Superuision. The project should devolve the payment of the DTS to the government partner to ensure efficiency and sustainability ofthe process. From the foufth year of program implementation, health area staff should be trained to carry out supervision of CDTI in an integrated manner. The costs of transmifting reports to the district office should be bome by the government and community. DMOs should routinely send lefters of commendation to chief of posts and communities. Prioittz HIGH Indiators of success CDDs keep Mectizan for two week following general distribution and absentees are treated. Management of Mectizan by government established system Who to take action: DMOs, PDPH Deadline for completion: End of next distribution Mectizan Procurement & Distribution. The duration of distribution should be extended in all communities. Mectizan should be left with CDDs for about two weeks after the community distribution in order to allow enough time for absentees to receive treatment. This recommendation will improve treatment coverage. Where feasible Mectizan should be stored and managed through the government established system. Prioittt HIGH Indiatorc of success HSAM is efficient Who to take action: Chief of post Training & HSAM: Health education, sensitization, mobilization and communication messages should provide communities with specific information to deal with identified problems. In the fourth vear oroiect manaqement should taroet trainino of 26 chief of posts. The education background of the chief of posts should be considered in planning targeted retraining sessions. To strengthen HSAM, community and social mobilisation officers and health educators should be co-opted to assist the project and improve the effectiveness of HSAM on the benefits of lonq-term compliance and ownership of CDTI. Deadline for completion: End of next distribution Funding/Financial NOTF should urgently sensitise the national, provincial and district governments to immediately assume their appropriate responsibiliUes. Governments should cover the costs of Mectizan procurement and delivery to health areas, supervision, advocacy and transmission of reports to districts. Priority HIGH Indicators of success Government cover substantial costs of the activities at the health area level Who to take action: NOTF Deadline for ampletion 30 September 2003 Transport & Other Material Resources Project should as a matter of urgency put in place an effective and efficient control mechanism (including the use of log book) in order to extend the life span of the motorcycles. From the fourth year of implementation, district funds should be used to support travel costs of chief of posts for the collection of Mectizan from the districts. PrioriU HIGH Indicators of success, Existence of plan for replacement of transport and other material resources Who to take action: DMO Dea dl i n e for com pl etion: 31 May 2003 Appropriate and sufficient number of training and HSAM materials should be made available to health areas before the next distribution to enhance the efforts of chief of posts, and sustain community interest in taking Mectizan that seem to have been achieved only during the last distribution Priorittt HIGH Indicators of success Sufficient training and HSAM materials are available to Health Areas. Who to take action: Chief of CommuniW Health Deadline for ampletion Human resources Health Area staff should be trained on community sensitizaUon and mobilizaUon. And this includes utilization of IEC materials. This is an essential ingredient to improve treatment coverage. Priority, HIGH Indicators of success, ALL health area staff trained on sensitisation and mobilisation of communities and evidence of appropriate use of IEC materials Who to take action DMOs Dea d I i n e for com pl etion: Mav 31, 2003 27 3.4 SUSTANABIUTY AT THE COMMUNITY LEVEL Fig. 4: SW|I CDTI Project: Sustainabllity at the Community Level 4 3,5 3 2,5 2 1,5 1 0,5 0 3 3 T,7t .tr .9 c, =oE)G o 2__ 1 """'""--""'O*""..- *..'' €$ a.-- "-" "/ Groups of lndicators Planning (Moderate; 2.0) CDDs plan their activities but not in an efficient manner. As an example, census update is carried out separately from Mectizan distribution. This decision of the upper levels increases the workload of the CDDs unnecessarily. In some communities it was found that CDDs also take measurements and record heights during census and return at a later date to distribute drugs. Only in few communities are CDDs assisted by community leaders and families to encourage people to take Mectizan. Leadership (Moderate; 2.0) In all communities visited leaders were not sensitized and mobilized to take ownership of CDT in the first and second year of program implementation. As a result they did not play an active role in the distribution of Mectizan. Treatment coverage was very low, mostly due to fear from exaggerated rumors of side effects. During the last (third year) distribution, leaders were mobilized and played an active role in dispelling rumors and convincing people to take Mectizan. As result coverage increased considerably and most people interuiewed indicated willingness to take treatment for as long as necessary. The challenge to the project is to enhance community leadership at all stages in CDTI. Meetings of all community members were organized at the beginning of the project to select CDDs. But there have not been subsequent meetings to discuss progress and emerging issues. Communities have indicated their preference for timing of distribution but in practice this is determined by higher levels, NGDO and the health system. I 2 28 The mode of distribution for the first year was house to house. The mode was changed in the second year to centralized treatment (which is unpopular with communiUes). The upper levels soon realized that the method of distribution imposed resulted in decline coverage and reverted to'house to house'treatment in the third year. (See also section provincial level under coverage). The overall conclusion is that most communiUes are not playing their expected role of decision-making in the CDTI paftnership. Monitoring (Highly; 3.0) CDDs prepare reports and forward them to Chief of posts on Ume. Some CDDs trek several kilometres to health centres. In Fontem Health Area, CDDs trek 6 -12 kilometres, without support from communities, to submit reports. In other instances, e.g, Mundemba Health Area CDDs hand the reports to supervisor, when the later is in community to monitor side effects. A few communities in Mamfe Health Area provide suppoft for transport to CDD to deliver reports to the superuisor. Obtaining and Managing Mectizan (Moderate; 2.0) Drugs were enough for the first and second years. Shortages were experienced in many areas in the third year. Surpluses were repofted in some communities. The evaluation team was surprised to find out that based on instructions from higher levels, drugs are mostly returned to the chief of post immediately after distribution; thus giving no chance for absentees to be treated later. Staff did not know how to calculate amounts of mectizan tablets required correctly and data used was often from non-updated registers. HSAM (Highly; 3.0) CDDs and community leaders were involved in health education and mobilization of the population for the last distribution (third treatment round). CDDs in some cases sensitized communities during women, youth and general meetings. Health EducaUon and sensitization materials are inadequate and CDDs lack adequate training and skills. As a result, communities have little knowledge of their roles and responsibilities in CDTI. Financing (Slighfly; 1.0) Many communities do not provide support to their CDDs. Communities did not see it as their responsibility to provide transport for CDDs to collect Mectizan from the health facility. Communities assumed that the NGDO and or MoH remunerate CDDs and have not considered suppofting their work. These sentiments were expressed by communities during meetings with evaluation teams. CDDs trek long distances to collect drugs from health centres; in Fontem district CDDs trek 6- 12 kilometers. Even in difficult-to reach areas, CDDs collect drugs where health workers fail to deliver them. CDDs also have to trek the same long distances to deliver repofts to FLHFs, repoftedly with little community support. Immediate changes are necessary, communiUes should be sensiUzed to provide suppoft to CDDs and to appreciate these tasks as community and not CDD responsibilities. In only very few communities, like the PAMOL Camps, where CDDs were exempted from work on the days they go for training. Human Resources (Moderate; 2.7) 29 In most communities the CDD population ratio is above that recommended by APOC; at least 2 CDDs per 250 persons (see APOC training manual). The ratio found by the evaluation team communities ranged between 250 - 600 persons to one CDD. CDDs distribute Mectizan by central and house-to house methods. In addition, they visit homes to follow-up on cases of side effects brought to their attention by community members. Distances between households in some villages are considerable. All these increase the workload of CDDs considerably. Because community members are not sensitized adequately to they provide no assistance to CDDs. Most CDDs are skilled some areas and there is plan to train replacements. However, there is a dire need to consider targeted training from the foutth year of the project. Emphasis should be on aspects (e.g. recording keeping, HSAM, correct dosage) in which CDDs are very weak In two villages CDD did not give correct dosage of Mectizan@ and one thought Mectizan should be given twice yearly. CDDs are willing to continue to help their people for more than 15 years. They also think the work is fine and the drug is useful. Successes of CDDs in the management of problems related to distribution were acknowledged orally through the village chiefs and health area staff. In some cases as in Pamol Health Area, Tee shifts and caps, which come in low supply, were given in recognition of good performance to motivate CDDs. Coverage (Slightly; 1.3) Though treatment is on going in all the communities visited, therapeutic coverage for the first two years were very low, decreasing from 54.5o/o in the first year to 37.80/o in the second. However, the evaluation team noticed that the coverage for some communities has improved following intensifled sensitization of community leaders and members after the very poor and decreasing coverage of the second year. The third year result is expected to be above 650lo. The evaluation team probed respondents at the four levels covered (Provincial, District, Health and Community) on key factors that have led to the impressive treatment coverage (e.g. 74.4o/o in Fontem District) during the last distribution. Our flndings summarized in Figure 1, are quite revealing and considered useful to the national onchocerciasis control program in Cameroon. Recommendations at Community level Recommendation Implementation Planning. Census taking and distribution of Mectizan by CDDs should be undertaken during the same period. Immediate action should be taken by project to improve the participation of community leadership in assisting CDDs to mobilise and educate community members. PnonV HIGH Indiatorc of sucess CDDs and Who to take action DMOs Deadline for ompletion May 6s 2oo3 Leadership. Priorittt HIGH 30 Community leadership should be urgently be visited and empowered about their roles and responsibiliUes in CDTI. They should understand the communities have the powers to select CDDs and change those not performing well, Communities should decide the timing and method of distribution. Indicatorc of success Community leaders take full responsibility of CDTI Who to take action NOTF Deadline for mmpletion May 31$ 2003 Monitoring, To sustain the interest of CDDs, proJect should sensitise communities to provide adequate suppoft to their CDDs Priority HIGH Indicators of success CDDs receive support from their communities. Who to take action Chief of posts Deadline for completion Mectizan ordering, procurement and distribution : The project should allow CDDs to keep Mectizan for at least two weeks after distribution and to treat absentees. This recommendation is especially important for very far and difficult -to-reach communities, many such communities are in the SWII project. Prioritu HIGH Indicators of success CDDs are allowed to keep Mectizan to treat absentees. Who to take action: DMOs Deadline for completion: End of the next distribution HSAM The practice of intensified health education should continue and strengthened. Health education, sensitization and mobilizaUon materials should be made available in sufficient quantity to support the work of chief of posts and CDDs in mobilizing communities. Priority MEDIUM Indicatorc of sucess, Increased coverage of >650/o and 100o/o therapeutic and geographical coverage rates respectively Who to take action: Chief of Posts Deadline for completion End of next distribution Financing Immediate changes are necessary in order for communities to appreciate the tasks of CDD as their own responsibilities. The Provincial delegate and NGDO paftner should as a matter of priority organize the health personnel at districts and health area levels and embark upon immediate sensitization of the leaders and communities on the responsibiliUes as the lead partner in CDTL ThiS is very crucial for sustainability of long-term treatrnent. Prioritlt HIGH Indicators of success Amount of support provided by communities Who to take action: Community leaders and members have good knowledge of their roles and responsibilities. Deadline for completion: PDPH, NGDO, DMO Human Resources The ratio of CDDs per population should be reviewed and ratified before the next treatment. Communities should be encouraged to select as many CDDs as possible so that one CDD will treat a maximum of 50 people and their work reduced to 2-3 days. CDDs should also work in pairs. Prioritf MEDIUM Indicators of success. Increase in the number of CDDs Who to take action Provincial delesate/NGDO Deadline for completion: May 31$ 2OO3 Coveraqe. Prioritu HIGH 3r Indicatorc of success, Distribution period is extended to 4 weeks rate nues to increase.and Who to take action Provincial Deadline for ompletion: End of next distribution The duration of distribution in communities is of concern and should immediately be reviewed upwards to the range of 4 week, to reflect APOC CDn guidelines and the wishes of the communities. 4.0 coNcLusroN 4.1 GRADING THE OVERALL SUSTAINABIUTY OF SW II CDTI PROJECT. Making a judgment of the project in terms of the seven aspects of sustainability a a (a) Make a.lutlqment of the project, in terms of each of the seven 'aspects'of sustainability: Integration: Integration of the project into the health system at the provincial level is particularly weak. CDTI activities at the District and Health Area are not integrated within the health system. It is managed at all levels almost as a vertical program. Comprehensive health plans contained CDn. Transport provided by APOC/SSI is used in an integrated manner for all health programs. Resources (Human, financial and material): Government contribution has been negligible. This has been only in the area of payment of the salaries of health personnel and occasional supply of stationeries at the Provincial and District levels. The management of funds for the project is not within the government system. Transport is available but there is no plan at any level for the replacement of transport and other material resources. Government does not meet the running costs for transport. There are enough human resources but not at the provincial and health area levels. a)L Efficienry: Program support activities are not being planned and integrated for efficient management of resources. Examples are Mectizan procurement and delivery, update of census and treatment, the flow of funds for activiUes implementation of acUvities. I aa JJ Simplicity: The management of fund is undertaken in a very complex manner. An example is the transfer of funds into project account. Fund transfer into the project account goes through a complex process shown in the box below. Attitude of Staff: At all levels, the evaluation team rated the attitude of staff towards CDTI to be very positive. FLOW OF DISBURSEMENT OF APOC FUNDS FOR FIELD ACTIVITIES Finance Officer in Yaounde checks Provincial Delegate for Public Health SSI CR in Yaounde studies and approves OPC in Buea makes request to PDPH SSI CR approves in Yaounde OPC raises cheque to PDPH PDPH signs cheque F.O. in Yaounde pays to account OPC raises cheques to be drawn from Limbe account Funds get to project account in Limbe PDPH in Buea signs OPC withdraws funds from project account in Limbe SSI Rep in Limbe signs I I 34 Community Ownership: The sense of community ownership with regards to taking responsibility for the program and supporting CDDs was very low. CommuniUes, where CDTI is implemented did not show knowledge of their roles in CDTI. EffecUveness: Though treatment has taken place in all Health Districts and health areas within the project, more than 700lo of the communities have therapeutic coverage of less than 650/0. Furthermore, some communities in Mamfe Health District have not been treated. (b) Next, the evaluation team examined the six key aspccts of the project- 'crltlcal elements' of sustainability. If these are not prcsent it is unlikely that the project will be sustainable: In line with the guideline for grading the whole project as "Half or less of the aspects are fulfilled, and half or less of the critical elementssatisfied" the evaluaUon team concludes that the SW II CDTI projedis NOT MAKING SAfiSFACTORY PROGRESS TOWARDS SUSTAINABIUW. This project has serious barriers to sustainability. It will require rethinking and mobilization of high-level support to get it on the road to sustainability. a a 35 Since the above grading tools (a and b) are new and being used for the first time, the evaluation team also judged the sustainability of the project through assessment of the nature and serious of deficiencies under each of the nine groups of indicators, Three serious deficiencies (blocking sustainability) were identified. These relate to community ownership, leadership at different levels and Mectizan. The judgement reached was that the project is not making satisfactory progress towards sustainability. Different ways of making judgment on sustainability for three-year project were examined. One was to lower the criteria for judgment. Thus it was examined to see if at least two of the aspects and two of the critical elements of sustainability were fulfilled and satisfied (instead of more than half for each). The results show that the project is still not making satisfactory progress towards sustainability. Another way was to assess whether the grading of the program would improve in the next two years if one assumes that the on going planned activities and improvements facilitated by the evaluation team would be successfully implemented. The conclusion was that more fundamental changes are needed, the grading of the project would not change. 4.2 TWO FEEDBACK/PLANNING MEETINGS: Two feedback/planning meetings, one for Provincial team and the other for District level teams were successfully organized. The agenda for both meetings are attached. Two days proved inadequate for brieflng and planning district sustainability plans. The evaluation team had two choices. First was to end the meeting and request the project to organize another one at a later date to complete the plans. The other option, which is the one preferred by the evaluation team, was to request a number of key participants to remain behind for two days to complete district sustainability plans. The Provincial Delegation confirmed that the plans (including budget and justiflcations) would be completed rapidly and fonararded to APOC. Below are a number issues that emerged after the briefing session in the provincial workshop. . The present situation whereby the Delegation (health system) does not seem to be in the 'driving seat' of for the planning and implementation of the CDTI project is not acceptable and corrective action should be taken as a matter of urgency. All parties concerned should respect the leading role of government on the lines of the project proposal and the annual project agreements between APOC/WHO and Government. With regard to reference in the report that the government has not made flnancial contributions for CDTI on the lines outlined in the project proposal the flnance officer of NOCP/Cameroon pointed out that government has made contributions to improve IEC, procure equipment including a computer and printer to SWII CDTI project. The equipment was retained in the NGDO office. a I I I a36 The present complicated financial management of APOC funds should be simplifled as follows: The project should have its primary Bank Account in the town where the project is located or the nearest convenient location where funds can be withdrawn easily. The DelegaUon team (MOH) should take full control of the day-to-day management of APOC funds, while the facilitating NGDO is expected to provide the needed technical support and financial control. The NGDO should remain a mandatory signatory to all cheques. The present Finance Officer being paid with APOC funds under technical assistance should relocate to the project office in Buea and work under the direction of the Provincial Delegation (MOH). This arrangement should continue until the delegation is in the position to absorb the Finance Officer. Regarding Mectizan procurement and distribution, deliberate effort should be made from the fourth year of the prqect to avoid shoftages of Mectizan. The project should evolve more reliable systems of Mectizan requirements computation and timely requisition to MDP and NOTF should facilitate the prompt release of Mectizan tablets to the project level. The project level should utilize the existing government established channels of drug delivery to the districts and Health Areas. 4.3 THE WAY FORWARD 4.3.1 Evaluation of SWz CDTI project found several deficiencies that block and will continue to block sustainability of the project unless rethinking, concerted effoft and action are taken to correct them. The main deficiencies include weakness related to leadership at different levels, Community ownership and distribution of Mectizan. 4.3.2 Build on lessons from the evaluation. The above deficiencies and others were discussed in detail with paftners and were the focus of rethinking and development of the Provincial and district sustainability plans. As a result of extensive advocacy which was a key component of the evaluation process a number of important decisions were reached. Examples of such decisions include: Setting up of a simpler and more efficient financial management system of APOC funds; Transfer of project finance officer from the NGDO office in Yaounde to Buea (project site); Enlargement of the provincial team with the addition of Provincial Chief of Serryice for Pharmacy and Provincial Chief of Service for Administration and Financial Affairs and enhancing ownership of CDTI at different levels (pafticularly the community). 4.3.3 Strengthen leadership role of PDPH. The PDPH (MOH, Buea) should make every effort to obtain its share of funds provided by the Minister of Health for CDTI under the HIPC initiative. The Provincial Delegation should ensure that the sustainability plans are finalised and implemented as a matter of urgency. The Delegation should monitor closely the implementation of recommendaUons 37 and activities to address deficiencies identified by the evaluation team and take prompt corrective action. SSI has played a leading role in the development of the project and experiences gained to date will be of great importance to implementation of the sustainability plan. Working relationship between PDPH with NOTF and MOH needs strengthening. For example PDPH did not have key information (related to government contribution to CDTI and other areas) to play its role effectively. 4.3.4 Learn by doing. A number of difficult issues are likely to arise in the course of implementing the sustainability plans. A number of issues emerged in the briefing/planning meeting. Examples include: How can essential community ownership in CDTI be maintained in the campaign approach? How can the project ensure that systems of remuneration of CDDs enhance community ownership of CDTI? Clarification on these and other issues were provided in the meeting. What is being suggested here is that the project should have an organized way through which emerging issues are constantly reviewed and findings used to make necessary adjustments. 4.3.5 APOC should follow closely implementation of the sustainability plan. Special attention should go to decentralized fund and program management, pattern and trend of budget allocation. While large investments at the proMncial level were justified during the development phase of the project in the flrst two yeas, the fourth year budget should see visibility and much more money (greater o/o of the budget) going to activities at the community and FLHFS. 38 Annex 1 TWO CDTI EVALUATION TEAM NAME ADDRESS Cafter Center Country RepresentaUve Cafter Center, Yaounde, Cameroon Tel.:22173326 Cell: 9923599 Email: qrbp@camnet.cm Dr. Albeft EYAMBA Box33277 Dar es Salaam, TANZANIA +0255 742 605025 eleuther@ud.co.tz Dr. Eleuther TARIMO Mr. Edward BANOBA NOCP 15 Bombo Road Box 1661 Kampala, Uganda Cell: 077595314 Dr. Musa OBADIAH Helen Keller International, la Akila Machunga Road Opposite National Library, Jos Tel.: 234 73462672 (office) 464291(office) email: hkw;@htsen.org Dr. Uche AMAZIGO wHo/APOC Oaugadougou 01, BP 549 Burkina Faso Email : amaziqouv@oncho.oms. bf Dr. Joseph Chukwudi OKEIBUNOR Depaftment of Sociology/Anthropology University of Nigeria, Nsukka Enugu State, NIGENA Tel.: 234 4277Lt69 (Home) Email : jokeibunor@yahoo.com 39 ANNEX 2 PROVINCIAL LEVEL WORKSHOP PROGRAMME Sustainability of CDTI in Cameroon SWII Project " Feedback"/Planning Meeting AGENDA 9.00 - 9.301 Reqistration of participants 9:30 - 9:35 To be appointed2 Openinq prayer 9:35 - 9:50 PCSCH3 Welcome 9:50 - 10:00 SWII CDn Project Co-ordinator 4 Introductions of participants PDPH10.00 - 10.305 Official Opening oPc10.30 - 11.006 Coffeeffea Break 11:00 - 11:20 Dr Okeibunor7 Presentation of Evaluation instruments Dr TarimoPresentation of findings from Evaluation and discussions 11:20 - 12:20I PCSCH12:20 - 1:009 Perspectives of the project and future direction and discussions 1.00 - 2.0010 LUNCH 2.00 - 3.30 Dr. Amazigo11 Guideline for sustainability plan, Discussion and Identification of issues for group work 3.30 - 5.00 PCSCH12 Group work for the development of Provincial level Sustainability 5.00 - 5.05 To be appointed13 Closinq Prayers Trme FacilrtatorActiviItem 40 Two 1 Ooeninq prayer 9:00 - 9:05 To be aooointed 2 Welcome Introduction to the day's activities 9:05 - 9:30 PCSCH 3 Continuation of qroup work 9.30 - 10:30 Group Leader 4 Coffeeffea Break 10.30 - 11.00 oPc 5 Group work continues 11.00 - 12.30 Group Leader 6 Presentation of report on group work 12.30 - 1.00 Drs. Tarimo/Amaziqo 7 LUNCH 1.00 - 2,00 B Incorporation of comments and consolidation of plan 2.00 - 4.30 Team leader 9 Presentation of final plan 4.30 - 5.00 Team 10 Next steps 5:00 - 5:15 PDPH Dr: Ntep SSI Rep Dr. Tarimo Dr. Amaziqo 11 General matters 5.15 - 5.25 SWII Project Co- ordinator/ Dr. Okeibunor 72 Closing Remarks s.25 - 5.30 PDPH 13 Closing Prayers 1s.30 - 5.35 To be appointed Time FacilitatorItem Acti 41 Annex 3 DISTRICT LEVEL WORKSHOP PROGRAMME Sustainability of CDTI in Cameroon SWII P@ect "FeedbackJPlanning Meeting AGENDA One 9.00 - 9.30Reqistration of paft icipants1 9:30 - 9:35 To be appointed2 Openinq prayer PCSCH9:35 - 9:503 Welcome SWII CDTI Project Co-ordinator 9:50 - 10:004 Introductions of participants Dr. Okeibunor10.00 - 10.305 Presentation of Evaluation instruments 10.30 - 11.00 oPcCoffee/Tea Break6 Dr Tarimo11:00 - 12:007 Presentation of findings from Evaluation and discussions 12:00 - 1:00LUNCH8 1:00 - 2:30 Dr. AmazigoGuideline for sustainability plan, Discussion and Identification of issues for group work 9 2.30 - 5.00 Group leaderGroup work for the development of District level Sustainability 10 5.00 - 5.0s To be appointed1l Closinq Prayers FacilitatorTimeActiviItem 42 Two 9:00 - 9:05 To be appointed1 Opening prayer 9:05 - 9:30 PcSCH2 Welcome Introduction to the day's activities 3 Continuation of qroup work 9.30 - 10:30 Group Leader 4 Coffeeflea Break 10.30 - 11.00 OPC 5 Group work continues 11.00 - 12.30 Group Leader 12.30 - 1.00 Drs. Tarimo/Amaziqo6 Presentation of report on group work 7 LUNCH 1.00 - 2.00 2.00 - 4.30 Team leaderB Incorporation of comments and consolidation of plan 9 Presentation of final plan 4.30 - 5.00 Team 10 Next steps 5:00 - 5:15 PDPH Dr: Ntep SSI Rep Dr. Tarimo Dr. Amaziqo 11 General matters 5.15 - 5.25 SWII Project Co- ordinator/ Dr. Okeibunor 5.25 - s.30 PDPH12 Closinq Remark 15.30 - 5.35 To be appointed13 Closing Prayers Time FacilitatorItem Activity