African Programme for Onchocerciasis Control: report of the external evaluation

r] African Programme for Onchocerciasrs Control (APOC) Report of the External Evaluation *$l* t l, Evaluators: Harald Burmeister Patrick Kayembe Kalambayi Georges Koulischer Olademeji Oladepo Bernard A Philippon Eleuther Tarimo (Chairman) October 2005 JAF11.10 Page i a Table of contents Glossary Acronyms.......... Acknowledgement Executive summary... 1. Introduction............. 2. The evaluation process...... 3. Findings and main recommendations 3.1 Implementation of the CDTI strategy 3.1.1 Community directed treatment. ..... 1.2 Drug Procurement and Delivery. Capacity building.... Monitoring una sup.*ili;; . :.. :... : :..... :.. :... Severe Adverse Effect (SAE) Programmes in confl icVpost-conflict areas Treatment Coverage... Evaluation and Sustainability Adequacy of impact assessment 3.1.10 Operational research 3.1.11 Implementation of Projects: Achievements and delays 3.1.12 The vector elimination projects in APOC... 3.2 Programmemanagement............ 3.2.1 Programmegovernance 3.2.2 Role of CSA............... 3.2.3 Role of TCC............... 3.2.4 APOC headquarters Office: adequacy and effectiveness of Programme management, Financial management, and staffing......3.2.5 APOC Financial Management.............. 3.2.6 APOC staff management 3.2.7 The Country Level 3.3 Integration of CDTI into overall health services 4. Partnership .............. 4.1 The various partners in synergy at all levels 4.2 Theinternationallevel.......... 4.3 The national level 5. Conclusions on findings and way forward after ApOC 5.1 Will APOC achieve its objectives? ............ 1 6 7 8 8 8 J 3 3 aJ 3 3 3 3 J 4 5 6 7 8 9 25 25 26 26 t lv v I I 1 1 t4 .......... l5 .......... l6 .......... 16 ,..,...... 17 .......... l9 ..........20 ,.......,.23 23 JAFI1.10 Page ii 5.2 The what after APOC? Annex I : Terms of reference for the external evaluation Annex 2 : Analysis of treatment results all APOC since inception Annex 3 : APOC Approved projects as at July 2005.......... Annex 4 : APOC organization chart 40 .......,.,42 ..'.....,,46 ..........48 ........489 a { a Glossary Geographical coverage: Therapeutic coverage: Ultimate Treatment Goal (UTG) Particioating Countries : Phase 1: Phase 2: The Proerarnme: JAFl1.10 Page iii number of communities treated in a given year over the total number of mesoAryper-endemic communities as identified by REMO in the project area (this should be expressed as a percentage). The Donor Community Bodies doriating funds for APOC and its projects and activities number of people treated in a given year over the total population (this should be expressed as a percentage). calculated as the maximum number of people to be treated annually in meso/fiyper endemic areas within the project area, ultimately to be reached when the project has reached full geographic coverage (normally the project should be expected to reach the UTG at the end ofthe 3'd year ofthe project). Countries conducting onchocerciasis control prograrunes in partnership with APOC The period of APOC's work from 1996 to 2000 The period of APOC's work from 2000 to 2007 APOC as a whole s at JAFI1.10 Page iv Acronyms I t AFRO APOC CDTI CDD CSA CSM DEC EPI FLHF FLHWs HSAM IEC JAF LGA LF MACROFIL MDP MoH NGDO NOCP NOTF OCP PHC SAE SIZ RAPLOA REMO TDR wHo African Regional Office of WHO African Programme for Onchocerciasis Control Cornmunity Directed Treatment by Ivermectin Comrnunity Directed Distributor of Ivermectin Cornmittee of Sponsoring Agencies Cornmunity Self Monitoring Diethyl Carbamazine Expanded Programme of Immunization Frontline Health Facility Frontline Health Workers Health Education, Sensitization, Advocacy and Mobilization Information, Education, Communication Joint Action Forum Local Govemment Area Lymphatic Filariasis Macrofilaricidal Drugs for Onchocerciasis and Lymphatic Filariasis Mectizan Donation Program Ministry of Health Non-Governmental Development Organization National Onchocerciasis Control Programme National Onchocerciasis Task Force Onchocerciasis Control Programme Primary Health Care Severe Adverse Effects Special lntervention Zones Rapid Assessment of Loaisis Rapid Epidemiological Mapping of Onchocerciasis UNDP/World Bank/WHO Special Programme on Research and Training World Health Organization a.t tJAFl l.l0 Page v Acknowledgement The Extemal Evaluation Team greatly appreciated the cooperation received from all partners bound to carry the Programme through to cornpletion. information received in thought-provoking exchanges with groups or individuals, supported by a wealth of documentation, have been determining factors in carrying the evaluation process. The team wishes in particular to express gratitude to: o the Participating Countries visited, notably, in each country, the Ministers of Health and the staff of national, district and local-level health services, and the members of the National Onchocerciasis Task Force (NOTF) . management staff at APOC Headquarters, in particular Dr A. Seketdli, Director of APOC, Dr U. Amazigo, Dr M. Noma, and numerous other resource persons o AFRO, the WHO representatives and country office staff of the countries visited o the members of the NGDO Coordination Group for Onchocerciasis Control, and the NGDO Coordinator at WHO Headquarters, Dr T. Ukety o the Tropical Disease Research Programme (TDR) of WHO, Geneva o at the World Bank, Dr O. Bangoura, Head of the Onchocerciasis Coordination Unit, Mr B. Benton, Dr B. Liese o the members of the Comrnittee of Sponsoring Agencies (CSA) of APOC o the members of the Technical Consultative Committee of APOC o Mr. A. Daribi, Onchocerciasis Liaison Office, Geneva ? I JAI'11.10 Page 1 Executive summary The context (i) The overall goal of the African Programme of Onchocersiasis (APOC) whicli was launched in 1995 with is to eliminate river blindness as a public health problem in 19 African countries where the disease is rampant. Specific objectives area: (a) to establish within a period of 12 to 15 years effective and self- sustainable Community Directed Treatment with ivermectin throughout the endemic areas within the geographic scope of the progralrlme, and (b) if possible, in selected isolated foci, to eliminate the vector by using environmentally safe methods. The programme became feasible in 1987 when ivermectin, the first safe drug for large-scale treatment of river blindness. Key features of the programme mobilization of communities to play a leading role in planning and overseeing their annual treatments and a robust partnership, which includes communities and govemments in affected countries, NGOs, International Organizations and the private sector. (ii) As part of its rigorous follow up of projects the Joint Action Forum (the APOC goveming Body), at its last session (Kinshasa, December 2004) established an external evaluation team of six people to assess, progress of APOC towards meeting its objectives and whether these will be achieved and to make recommendations on how best to sustain CDTI after 2010. (iii) The evaluation team consulted widely and participated in meetings of APOC governing and technical bodies. Team visited five (Cameroon, Democratic Republic of Congo, Nigeria, Sudan, and Uganda) to assess situation on the ground. The Countries and projects visited were carefully selected to represent the situation of the l9 countries as much as possible. Thus projects selected included those doing poorly, in conflict situations and older ones. (iu) The findings of the evaluation team are summarized below (section 2), under two headings: Programme implementation; Management and Partnership. While some achievements are pointed out the summary report is essentially futuristic focusing on issues and recommendations. The last part of the summary report (Section 3), contains Conclusion on findings and suggestions on way forward. Findings and main recommendations Pro gramme impl ement ation (") The APOC strategy, CDTI is based on findings from sound scientific studies and addresses an important neglected problem, affecting mostly the rural poor, which can be tackled at low cost. Thus CDTI has a sound moral appeal. The strategy also has clear objectives, plans and targets. (vi) Mobilization of communities, procurement and delivery of drugs and capacity building are areas of great success. There is extensive satisfaction of communities with ivennectine treatment. Community assertiveness and demand for drugs is high wliich is good for sustainability. Large numbers of well-trained CDDs, the backbone of CDTI has been trained and deployed. Procurement, delivery and supply of Mectizan, is increasingly through the overall MOH system. Supplies are adequate. Issues i*lud., lnadequate mobilization of communities to increase numbers of CDDs and to take their (CDDs') ownership including responsibility for providing them with incentives; lack of uniform MOH policies on provision of incentives to community volunteers; ensuring that mechanism for custom clearanci are sustainable, rapid JAFl1.10 Page2 change of staff in countries and signs of the issue of 'why should I continue to take ivermectin when I am no longer ill'? (vii) The Mid+erm evaluation (2000) was very concerned with the danger posed by SAEs and the inadequacy of response. Exemplary work done by: one MOH and NOTF in one country; APOC and MDP, on the prevention and management of SAEs. Tools and experiences gained in the country as well as APOC guidelines are now being disseminated to other countries and projects The challenge is one of implementation and continuous updating of the SAEs progrmme. (viii) Data in Annex 2 is revealing. Numbers of people treated and therapeutic coverage, have increased steadily for most projects, except for those in conflict areas. APOC launched an adapted CDTI program for countries in conflict/post-conflict situations. Issues include weak analysis of coverage data, and non use of accepted definitions of therapeutic coverage at times. The challenge in relation to countries in conflict includes adequate preparation for 'when peace comes'. Ther has been considerable delay in implementation of CDTI projects. As of March 2005 five projects were still in the plaruring stage, 24 projects had been approved but not launched. These projects will not have received basic APOC support by 2010, when APOC is planned to end. (ix) To supplement routine monitoring, APOC has introduced independent monitoring and evaluations, which have been successful in enhancing implementation of CDTI. Out of 49 projects evaluated (2002- 2005), 35 (73%) were judged as moving towards sustainability. Issues include: synthesis and dissemination of lessons from evaluations carried out to date. (x) Impact assessment, to find out the extent to which ivermectin treatment prevents and or leads to regression of manifestations and damage caused by river blindness has stated in 14 sites in 9 countries. Issues raised include: Introduction of monitoring of human infection (some countries already doing this) and expansion of impact monitoring to all participating countries. (xi) APOC provides substantive contribution to research via support to TDR, MACROFIL initiative (search for a macrofilaricide) and through the Director's and TCC's Grants to small operational research projects. A number of key APOC tools and strategies, emerging from the research have been developed and used. Issues raised include: Inadequacy of the role of TCC and APOC management in research funded by APOC; inadequacy of the current capacity of APOC to support research; Lack of standards for drawing up budgetsr and relevance ofstrategy adopted and quality ofresearch ofmacrofilaricide. (xii) Vector elimination has stated in four sites. Results from the Itwara and Mpamba-Nkusi focus (Uganda) show that elimination is an achievable objective. Results from Tukuyu (Tanzania) and Bioko (Equatorial Guinea) show that elimination of S. damnosum s.l, which is notoriously more difficult, is uncertain. (xiii) Along with other programs, APOC leadership has followed actively the evolution of the concept of "self-sustainability" (meaning communities, governments and local NGDO resources in affected countries to sustain CDTI after APOC, with no outside support) to "sustainability' in a global context, providing clarifications to partners as necessary. (xiv) Goverrunent contributions, in the form of salaries of staff and installed capacity have contributed enormously to the success of CDTI. Government contribution is particularly visible in the sustainability plans developed after the evaluation ofprojects. I I This issue was already raised by the 2000 Medium Term Evaluation fJAF1l.10 Page 3 Key recommendations NOTF and CDTI projects should intensiff sensitization and mobilization to encourage communities to select more CDDs to increase the CDD/population ratio and thus decrease workload. Community sensitization and mobilization for selection of women as CDDs should be consistently encouraged. NOTFs should organize special meetings to deliberate and reassess the present situation on the issue of incentives, the lackof which can have unfortunate effect on sustainability of CDTI. Each NOTF should formulate a plan including budget needed to sustain capacity building APOC and NOTFAIOCP should intensify their effort to ensure that all projects go to scale with Community self-monitoring as soon as possible. APOC should facilitate exchange of information and experiences between Countries/projects in conflict situations and those in post-conflict. APOC should synthesize experiences from reports of CDTI project evaluated to date, Ieading to lessons, which can be a source of inspiration to individual projects. APOC should analyze the process of developing sustainability plans and the resulting plans developed to date with a view to arriving at standards for drawing CDTI budgets. APOC should consider introducing monitoring of human infection in the impact assessment protocols, as it is already being carried out on a national basis by some APOC participating countries. APOC should consider expanding impact assessment studies to all participating countries and in countries with only one sentinel site, the number should be increased. APOC management should give more priority to operational research. A significant asset should be the creation of one Senior Staff Research post at the HQ for the coordination of APOC research activities and making the Programme to participate actively at the intemational scientific level. CSA should alrange for an Expert review of strategy and quality of research on Macrofil. ln view of the delays that a number of CDTI projects have experienced and taking into consideration the irnportance of ensuring that the project, some of which will operate under difficult post conflict environment, are brougl-rt to satisfactory conclusion, the external evaluation team recommends the extension of APOC Trust Fund support on a decreasing scale to 2015. On vector elimination, in Bioko, maintain entomological monitoring over another two-year period. Should biting fly reappear meanwhile, a protocol could be worked out for a national project. In Tukuyu, APOC should ensure that optimal technical assistance is provided to the project both for treatments and entomological evaluation. ln due course, the procedure should be similar to that in Bioko by the end of operations. In Itwara the question raised is that of the certification; TCC has approved the request and also recommended a publication on the Itwara success story. In Mpamba- Nkusi it is recommended that one more annual larvicide round be performed, and intensive entomological monitoring. Without embarking in nuisance control operations which are not within its mandate, APOC should brings technical assistance to Participating Countries intending to carry out such operations on their own. a a a Programme management (xv) CSA is taking more interest and leadership on sustainability of projects. kr the circumstances participation of resource persons in CSA meetings might provide useful infonnation for decision makrng. TCC has now been relieved from financial evaluations, and technical reviews have been standardized. TCC members felt that the situation has improved considerably. The challenge now is for TCC to exercise its power of initiative and innovation and operational research. (xvi) APOC Programme management was good. An important issue noted was weakness in a JAFI l.l0 Page 4 communication between levels, and partners particularly in the area of financial management. (xvii) On APOC Financial Management, planned improvements include: schemes to provide additional staff training at headquarters, more country visits to monitor and train personnel at country offices and projects and financial management reviews. Issues identified include, persistent backlog of financial statements to be reviewed and cleared; LoAs, a major repetitive task has to be renewed annually, a multi- year formula might be better; inadequate and accurate financial reporting along the chain upwards to the Word Bank. The evaluation team discussed the above issues and concluded that more in -depth analysis of the impact of current arrangements and optional solutions is needed. (xviii) There has been considerable staff increase,to 14 Professionals, and 52 General Service (2003). Staff now appears less overwhelmed, An important issue raised related to inadequacy of contractual arrangements for some staff. (xix) At The Country Level the NOTF mechanism coordinates the effort of CDTI partners. Two recurring issues emerged: What is the future of NOTF? Decentralization yes but where is the strategy? Similar questions were raised in the 2000 evaluation. (xx) The role of NGDOs has been critical to the success of CDTL Most NGDOs at the country level expressed willingness to continue supporting CDTI; "We were here before APOC, we will be there after". Reaffirmation of the same message emerged from the NGDO Group meeting, at its 25th session2, which referred to "its role in supporting onchocerciasis control up till and after 2010" The challenge is for NGDOs to step up their presence even further and participate fully in planning for 'when APOC is no more'. There does not seem to be much improvement on the participation of national NGDOs in CDTI activities. (xxi) The role of WHO at country level varies but includes logistical, administrative and hnancial management support. Need for increased support from WHO and other agencies like UNICEF will arise, particularly after APOC. (xxii) On funding at country level, the Memorandum for APOC, stipulates that NOTF will be responsible for 25Yo of the ivermectin distribution costs (in cash or in kind), which will not be available from the APOC Trust Fund. Data on contributions of the countries visited was scanty. (xxiii) lntegration has taken different forms including other programmes using the the CDTI infracture. APOC has developed and uses a quantitative tool to assess extent of integration. About half of twenty-nine projects evaluated, mostly in 2003, achieved a level of integration considered by APOC as adequate to enhance sustainability (a score of 2.5+ out of 5.0). The evaluations indicated that while integration is accepted as the life hne for CDTI sustainability, there were no strategies or plans. Another issue is that integration of services is beyond the mandate of CDTI. A recent APOC initiative to support meetings of high level decision makers for programmes involved in community health interventions to find ways of improving integration is promising. But to succeed the initiative requires more investment and visibility. Key recommendations (xxiv) APOC should promote and oversee the early development and implementation of policies and programs, in consultation with govemments, NGDOs, WHO and all other partners, to prepare stakeholders for the continuation of the CDTI prograrnme after APOC. 2 Ouagadougou, 10-12 March 2005 JAF11.10 Page 5 (xxv) CSA and JAF should consult at an early stage with partners: governments, international organizations, donor agencies and NGDOs on ways to support CDTI and the continued fight against onchocerciasis in a coordinated fashion after APOC comes to an end. The primary focus may be on setting up an intemational coordinating and surveillance mechanism to become effective by 2010 and to be available to governments to support their programs. (xxvi) To enhance effectiveness and quality of its deliberations on sustainability, CSA should consider inviting resource persons, possibly representatives of NOTP on a rotation basis to participate in deliberations on relevant agenda items. To alleviative the heavy load of work on Progranrme Management, CSA should rational and reduce statutory and other meetings, (xxvii) APOC should consider conducing a detailed review, possibly resorting to WHO Headquarters or outside consultants, of its financial management systems, (xxviii) APOC should conduct a detailed review, possibly resorting to WHO Headquarters or outside consultants, of the adequacy of staff and making appropriate recommendations to assist the organisation to face its considerable task and challenges from this evaluation. (xxix) TCC should play more attention to issues crucial for the future of APOC (e.g.: CDTI sustainability at various levels, integration) and include regular planned working session on these topics in its annual meeting agendas. TCC should reinforce or develop its proactive role in proposals of new research activities, evaluation of protocols coming from outside and follow-up of on-going research projects, especially those conducted with APOC partial or complete financial support. (xvx) APOC management should prepare a position paper on decentralization of selected functions from Ouaga to the country level. The paper should draw on the experiences of other programmes and agencies and what can be expected. This paper would be an important input to the participatory planning work below. (xxxi) APOC management should prepare an analytical position paper on issues related to NOTFs responsibility for covering 25%o of the ivermectin distribution project costs, for review and decisions by TCC and CSA. (xxxii) As overall integration of health services is beyond CDTI, Ministries of Heath, in collaboration with partners including donor agencies, should find ways ofenhancing integration ofhealth services and develop an appropriate strategy and plan. JAF11.10 Page 6 1. Introduction Why APOC? I .l Five interrelated developments led to the birth of APOC. First was concern of the global community in the early 1990s, that while the Onchocerciasis Control Programme (OCP) was at the verge of eliminating river blindness'(through large scale larviciding) as a public health problem in ten west African countries, the disease remained rampant in endemic areas of l9 other countries in Africa. The second development was the availability in 1987, of lvermectin, the first safe drug for large-scale treatment of Onchocerciasis, which the manufacturers, Merck & Co, lnc, decided to provide free of charge for treating the disease as long as needed. Community trials carried out in affected communities between 1987 and 1989 showed that the drug was effective and with minimal side effects. Following these studies Ivermectin was and continues to be used successfully in the former OCP countries. 1.2 About the same time a number of NGDOs started Ivermectin treatment in some of the 19 countries using different approaches. To enhance cooperation, an NGDO Coordinating Group for Ivermectin distribution was formed in 1992. Fourthly and very important, a tool for Rapid Epidemiological Mapping of Onchocerciasis (REMO) was developed by the Tropical Disease Research Programme of WHO (TDR) to identiS areas and communities most in need of treatment. Fifthly, a TDR multi-country study ori community-based treatment of onchocerciasis in eight countries, in the early 1990s established that treatment approaches in which the communify plays a major role, "self treatment", later the term was changed to "Community Directed Treatment with Ivermectin" (CDTI), achieved higher levels of treatment coverage than where the community just played a passive recipient role. 1.3 Finally by 1995 it was clear that the task of providing effective mass treatment in affected areas of the 19 countries was huge and complex, beyond NGDOs, needing more resources, more government commitment and stronger partnership and research. (The number of people treated per year at that time stagnated at around 8 million). APOC was formed at the end of 1995 to spearhead a programme whose ultimate goal was to eliminate onchocerciasis as a disease of public health importance in the 19 countries. Specific objectives of APOC 3, are (a) to establish within a period of 12 to 15 years effective and self- sustainable Community Directed Treatment with ivermectin throughout the endemic areas within the geographic scope of the progralnme, and (b) if possible, in selected isolated foci, to eliminate the vector by using environmentally safe methods 1.4 Another key feature of APOC, besides "Community Directed Treatment" is strong partnership that unites the participating countries and the global community, with five components: Joint Action Forum(JAF), the Committee of Sponsoring Agencies (CSA), Programme Management, the Technical Consultative Committee (TCC), and the NGDO Coordination Group. JAF, which includes Ministers of health from participating countries and meets annually, is the highest decision-making body a. 1.5 Other features of the APOC prograrnme include: Early introduction of independent monitoring and evaluation; Surveillance and prompt action on serious side effects; Operational research and impact assessment. On funding APOC may pay up to 75o/o of the cost of ivermectin distribution projects, while National Onchocersiasis Task Forces (NOTFs and the government defray the balance concerned. APOC funding for each project is given for a maximum of five years followed by three years of winding down. 3 As revised by the Joint Action Forum (JAF) in 2001 o African Programme for Onchocerciasis Control (APOC), Programme Document for Phase l, 1996, Ouagadougou JAFI1.10 PageT 1 .6 The cumulative number of "CDTI projects" has increased enormously over the years, from 7 (1997); to 48 (2000) and 96 (2005). There are also four additional projects in support of the complementary strategy of vector elimination in selected foci, while seven ("HQs support projects") have been established in some countries to support the Coordination of the Programme at national level. Wy the external evaluation? 1.7 External evaluations at regular intervals are part of APOC's rigorous follow-up of projects. Extemal Evaluation of Phase I of the Programme (1996-2001) was carried out in 2000. The Phase II APOC plan (2002-2010) indicates that "Further external evaluations of APOC will be undertaken in 2004, 2007 and 2010, at the end of the Programme". In this framework, the lOth session of JAF (Kinshasa, December 2004) established an external evaluation team of six people to assess progress of APOC towards meeting its objective and make recommendations on how best to sustain CDTI after 2010. Annex I contains the detailed Terms of Reference for the evaluation team. 2. The evaluation process Wide consultations, and review of documents 2.1 The Extemal Evaluation Team (consisting of six members), met with representatives of the World Bank, WHO and Director of APOC in Geneva, on 25 and 26 February 2005 to discuss and clariff TORs, agree on data that should be obtained and to draw up a work plan of activities, their organization and timing. 2.2 In March 2005 the tean-r held discussions in Ouagadougou with APOC management and other key staff, during which the evaluation team was briefed on ongoing APOC activities and emerging issues. The team reviewed relevant documents including the 2000 mid-term evaluation report, and allocated tasks to its members. The timing of the work in Ouagadougou provided opportunity for members of the team to participate in two important meetings: 25th session of the NGDO Group for Onchocerciasis Control and the 20th session of APOC's Technical Consultative Committee (TCC). The team also took advantage of the TCC to hold talks with senior staff of the WHO Regional Office for Africa (AFRO, Harare and Brazzaville). Field visits 2.3 A great deal of the team's attention focused on planning field visits to assess the situation on the ground. Selection of projects to be visited was based on information provided by APOC Management related to: respective dates of launching; implementation stage and degree of sustainability; geographical and therapeutic coverage; projects on schedule or delayed, including countries in a post-conflict situation and inclusion of one Vector Elimination project. 2.4 Based on these considerations, projects were selected in five countries: Cameroon, Democratic Republic of Congo, Nigeria, Sudan, and Uganda. Each country was visited by two members of the team over March to May. One team member attended the Mectizan Donation Programme (I!DP) meeting in Atlanta (USA) in April. Prior to country visits, documents related to countries to be visited (including JAF1l.l0 Page 8 Projects submissions, Letters of Agreement, Technical and Evaluation Reports, Sustainability Plans, as well as TCC comments and recommendations) were reviewed. Within countries, information was obtained from field observations, interviews with National Onchocerciasis Task Force (NOTF) members, decision makers and other individuals at different levels of the health system. Data analysis 2.5 Main areas considered in data analysis included: trends overtime (e.g. geographical and therapeutic coverage), the path to sustainability, response to important events (such as SAEs), capacity building, level of skills in relation to tasks to be performed, interest and motivation of staff, and innovation. Data obtained was analyzed and compiled into a report by each country team. Key components of country reports were pooled at Ouagadougou in June 2005, together with information obtained by the team from other sources. Conclusions and recommendations emerging from the process were presented in July to the Comrnittee of Sponsoring Agencies (CSA). Feedback from CSA was an important input for preparing the final report. 2.6 The main hurdle to the process of evaluation was the limitation of time, particularly in view of the extensive travel needed for field visits. Secondly two evaluations, APOC and SZ were running more or less concurrently with some members participating in both. 3. Findings and main recommendations 3.1 Implementation of the CDTI strategy To assess the extent APOC has moved towards achieving its two objectives, this section examines progress and its adequacy in eleven key implementation areas: corrununity directed treatment; drug procurement and distribution; capacity building; monitoring and supervision; severe adverse effects; programmes in conflict/post conflict situations; treatment coverage; evaluation and sustainability; irnpact assessment; operational research and vector elirnination. 3.1.1 Community directed treatment. Achievements 3.1.1.1 The hallmark of the CDTI strategy is community empowemrent. Comrnunity leaders and members showed good understanding of the strategy, burden posed by disease and most of their responsibilities in the CDTI process. Communities everywhere played active role, in planning CDTI, selecting and deploying Community Directed Distributors (CDDs), collecting and distributing Mectizan. An increasing number of communities are carrying out "self-monitoring" to assess progress as to how well they do in implementing CDTL 3.1.1.2 There was also widespread satisfaction and appreciation of CDTI benefits by community members as evidenced by expressions such as, "itching and skin rashes are gone" and "blindness is less". 3.1' 1.3 CDDs, the backbone of CDTI, were well trained. The mid-term evaluation reported a high rate of attrition among CDDs, "up to 20o/oper year in some projects". Findings of the present evaluation suggest a low attrition rate of between 2 to 5% in most projects. Anecdotal evidence suggests that communities where CDDs were selected by open election at village meetings experienced less attrition compared to those selected by village authorities and health workers. A recurring issue over the years is payrnent of CDDs. JAF11.10 Page 9 While CDDs assert their willingness to continue ivermectin distribution as long as necessary, they often showed expectation for some form of recognition and incentives. CDDs in one country indicated to the evaluation team that incentives were not necessary, since they are treating members of their own communities. CDDs in this country are based on a system of clanship. Other communities have devised Iocal incentives in the form of "token" monetary gifts, meals during distribution, provision of overnight accommodation and help in cultivation of CDD's farm. 3.1.1.4 Findings from evaluation of 3'd and 5'r'year, show that over 7l .4%oprojects evaluated scored higher than the APOC cut off point of 2.5 (out of 5) and had a mean score higher than other levels (FLHFs, District and NationaVstake) s. 3.1.1.5 The present evaluation team was pleased to note during field visits, that many First Line Health Facilities (FLHF) staff were generally skilled, well informed on CDTI and provided considerable support to CDDs and communities. This finding shows some improvement on findings and concerns of the 2000 evaluation report. 3.1.1.6 The role of the District/Council level in supporting CDTI projects was good in manyprojects. Some district managers had electronically retrievable data on CDTI including treatment coverage by villages and were conversant with the situation on the ground. 3.1.1.7 APOC has over the years provided a total of 140 vehicles, 952 motorcycles and 3397 bicycles for the programme. fssues of concern 3.1.1.8 ln spite of the positive findings noted above, instances of inadequate sensitization and motivation of community leaders and communities to take ownership of CDTI and motivation of CDDs were found in the field. For example when a community leader was asked why the community was not providing motivation to CDDs as required under CDTI, he addressed his response to the local health worker, "But you have not informed us of this matter". Some communities visited believed that CDDs were well paid by NGDOs and government. Some CDDs thought that their supervisors pocketed their money. There was also evidence of instances where unsustainable incentives were been given, which might work against CDTL 3.1.1.9 The ratio of CDDs to population served up to 1:450 in some communities, resulted in reported heavy workload of three to four weeks annually. A recurring recommendation of many CDTI monitoring and evaluation reports in the period 2002-2004 has been on the need to have adequate numbers of CDDs. Response to these recommendations has not been adequate, What was intriguing in the field was the aversion of CDDs to suggestions for training and deploying new CDDs to decrease workload! Responses were invariably "no, we can do the work provided we get appropriate incentives". But community leaders in general were for more CDDs. 3. 1 . 1 . I 0 While the CDTI policy is that communities should be mobilized to take ownership and provide sustainable motivation to CDDs, some other programs, including Vitarnin A supplementation and immunization, supported by donor agencies, pay comparatively large amounts as allowances to their community based "volunteers". The differential treatment of comrnunity volunteers is confusing and may work against CDTI. t APOC, Year 2004 Progress Report, Ouagadougou JAFI1.10 Page 10 3 . l. I . 1 1 There was no evidence of reviews and continuous rethinking on CDD/CDTI issues that may be encountered in the future and how such developments could be prevented. For example does CDTI depend too much on CDDs, who can be strong or weak? Do PHC facilities feel less concerned with CDTI since so much responsibility rests on CDDs and communities themselves? Are CDDs over-burdened or over- demanding. becoming "unionized", thus at tirnes being against increases in CDDs and holding CDTI in ransom? 3 .l.l.l2 Lack of simple transport facilities such as bicycles in many villages limits the ability of CDDs to reach communities (where the population is very scattered) to provide treatment and to follow up absentees and refusals. Inadequacyllack of transport at First Line Health Facilities (FLHS) is also an obstacle to training, sensitization and supervision of CDDs. 3.1.1.13 Independent Monitoring data shows that of 757 CDDs surveyed 52.7% did not receive any kind of incentive,26.6yo got monetary incentives and,20.6%o transport support. But "there was no association between giving monetary or in kind incentives to CDDs and treatment coverage u". But experience in the field was that the issue of incentives is complex and just does not go away. This is an area that needs continuous follox up, surveillance and rethinking. 3.1.1.14 A contribution of CDDs to overall health activities, that is often overlooked, rests in the CDTI census carried out by CDDs in communities, registering each member of the target population. Some health workers in the field indicated that such data, for example that related to under-fives and expectant mothers, when aggregated for geographical areas might be useful to other programmes. Information on the availability of the census data might be of interest to Reproductive Health and Child Health programmes and a basis for future collaboration. 3.1.1.15 On Participation of Women, religion and age-long traditions of male dominance in many cultures, as well as illiteracy, seemed to work against women involvement. Information from records and discussions with field staff indicated that selection of women CDDs is gradually gaining ground in some communities. Reports from communities where women have been selected as CDDs indicated that their performances were equal to that of their male counterparts. Another positive trend observed in some projects was that of "pairing" male and female CDDs. Recommendation 1 (a) NOTF and CDTI projects should intensify sensitization and mobilization to encourage communities to select more CDDs to increasethe CDD/population ratio and thus decrease workload. The object is for a CDD to be responsible for treating only people in the close neighborhood, if possibte not exceeding one hundred. As an added measure in support of CDDs role, and of the wider CDTI social contract, simple messages (written or pictortal according to degree of literacy of target groups) indicating clearly the role of dffirent partners should be made available to the communities and posted at appropriate places. Community sensitization and mobilization for selection of women as CDDs should be consistently encouraged during community meetings in light of the positive reports of their comparative performance with male CDDs. NOTFs should organize special meetings to deliberate and reassess the present situation on the(c) 6 Amazigo, U.V. Obone, O, N, and Dadzie,KY et al. Monitoring CDTI for sustainability: Lessons from APOC. Annals of TropicalMedicine and Parasitology, g6 (Suppl.1), S75-S92 , (b) 6) JAFl1.10 Page 1 1 issue of incentives, the lack of which can have unfortunate effect on sustainability of CDTI and take prompt remedial action. Basic data on the present sttuation in dffirent projects should be collected to facilitate discussions. (d) MOH and NOTFs should continue to intensify effirt to strengthen FLHFs to play a leading role in supporting CDTI at the community level. (e) APOC might consider supporting a study to explore the feasibility, usefulness and,cost effectiveness of using the census data collected by CDDs for other health activities. APOC, NOTFs and other agencies, should provide support with transport particularly bicycles for CDDs with emphasis on areas where populations are scattered. 3.1.2 Drug Procurement and Delivery Achievements 3.1.2.1 Drug procurement and distribution is anarea of great success in APOC. Annual orders of the drug, based on total population and treatment figures of the previous year have been efficiently used in ordering for Mectizan. Issues identified in the 2000 Mid term evaluation such as long hold ups of drugs during custom clearance have been largely solved. The role of APOC in the procurement and distribution of ivermectin is now less, which is good for sustainability. Governments are increasingly carrying out clearing of Mectizan shipments at point of entry. Down the chain, Mectizan delivery to rural communities is good, even where APOC has terminated funding in the last two years. FLHFs collect drugs a:t the districVcouncil levels and the community collects it from the FLHFs. The drug collection usually does not necessitate additional funds since it is collected when health workers travel to the upper level for other purposes. 3.1.2.2 ln general, drug orders to and delivery by the Mectizan Donation Programme (MDP) do not suffer delays. Annual drug procurement and delivery has on the whole been adequate and timely even in conflict areas. 3.1.2.3 An important finding in the field was the high interest and demand for Ivermectin by communities, "Why is Mectizan late? When will it arrive" were common questions in some of the projects visited. Issues of concern 3.1.2.4 LINICEF and WHO have continued to provide support in a number of countries for the clearance of Mectizan from the port and its storage before collection by government or partners. However, it was understood that these arrangements are provisional. 3.1.2.5 Late request of Mectizan was documented in three of the countries visited delaying the distribution and increased absenteeism. There were reports if ivermectin of known or unknown origins circulating in areas non targeted for CDTI . Treatment in areas, which are not targeted, is provided by health facilities. JAFl1.10 Page 12 Recommendation 2 (a) APOC should encourage NOTF and governments to ensure that they use sustainable arrangements for Customs clearance and storage of Mectizan where applicable (b) NOTFs should continue to be alert to and investigate promptly reports of leakages and take prompt appropriate action (c) NOTFs should intensify efforts to ensure that drugs are ordered early enough to avoid delays in arrival and distributions. 3.1.3 Capacitybuilding Training, including re-training 3.1.3. I Training is another area of APOC success. A cascading down process starting with NOTF down to communities through existing health structures at the various levels has been used to improve operational skills in many countries. This ripple effect is a source of strength for the Programme. Targets for training at the local level include health workers and CDDs. APOC records show that the number CDDs trained has increased enormously from 539 CDDs in 1998, when the training process was launched, to 242,826 (178,808 retrained and 64,018 newly trained) for 2004 in the whole programme. 3.1.3.2 APOC has developed sets of training material, which are packaged according to target audiences and purpose, used both for training and enhancing sustainability. A practical manual, "Community- Directed Treatment with Ivermectin (CDTI)". issued in 1998. has proven a valuable tool. Leaflets and visual material on different aspects of oncho are also widely used. 3.1.3.3 Training, including re-training, has covered a wide range of technical needs, including: REMO; RAPLOA and handling of SAEs; Geographic lnformation Systems (GIS); computer use, data analysis, financial management of APOC Trust Fund as well as development and use of instruments and guidelines to evaluate CDTI. APOC support has taken different forms including technical and financial support 3.1.3.4 As regards financing of the above measures, APOC contribution is made under a specific budgetary line for "Training, Worlshops, Mobilization & Advocacy'', and as separate lines within each project. t is estimated that, to date, a total of over US $ 7, 5 million has been spent or firmly committed for these items under APOC Phase IL Two other sources of funding are the governments and NGDOs. Although overall data on contributions of NGDOs is not available, findings in the field showed that considerable support is provided. 3.1.3.5 The TCC, in its Fifteenth Session (September 2002), when reviewing the issue of APOC support to projects after 5 years, recommended that support to measures for capacity building including advocacy, training and new capital equipment should be provided for years 6 to 8 so as to avoid declining in performance due following withdrawal of APOC support after 5 years, and to consolidate progress towards sustainability. Before this decision APOC was not expected to support programmatic activities including CDD training after the fifth year. I I 3.1.3.6 Extensive Health Education/Sensitization/Advocacy/Community Mobilization (HSAM) JAFl1.l0 Page 13 has taken place in communities as evidenced by the knowledge of community leaders and communities on CDTI in such areas such as: Mectizan benefits, dosage, people to be included or excluded and that the Mectizan was donated (free), fssues ofconcern 3.1.3.7 A major challenge is how to sustain resources for training (both basic and re-training). Funding by governments is inadequate. 3.1.3.8 Epidemiological and analytical culture was generally weak. For example some of the treatment coverage data seen in the field showed great disparities between sub districts and villages. But there was no evidence of the data being analyzed and used to target specific support/activities to areas with poor coverage. 3.1.3.9 A recurring issue in the field was "why Mectizan needs to be taken continuously, for additional number of years, by people who no longer experience skin and visual problems". The issue was reported to accentuate refusals or absenteeism. lncreased sensitization in relation to the issue is needed at all levels, the community, health services, and most importantly the political class. 3.1.3.10 Despite the strength of the cascading approach to training on the lines indicated above weaknesses were reported in at least one country; one level of the system may be by-passed with detrimental effect and or quality of training may drop in the process. Corrective action can be triggered when supervisors detect insufficiencies on the spot. Recommendation 3 (a) Each NOTF should formulate a plan including budget needed to sustain capacity building. The plan should include needs for training and retraining and other aspects of enhancing capacity including reviews and revisions of IEC materials. APOC and partners should ensure that adequate funds are available for projects which have completed their fifth year as needed. (c) NOTF and projects should intensify sensitisation of communities, health service workers, the public, and political leaders on the basics of CDTI including need for continuing with ivermectin treatment for many years. APOC and NOTFs should support operational studies on compliance with and continued use of ivermectin in areas with dffirent pre-treatment epidemiological patterns, for example with or wtthout blindness and with severe or mild skin disease. The findings and lessons will provide strong evidence on which to develop new IEC materials that would motivate community members to continue taking Mectizan even after symptoms have disappeared 7 (d) (e) APOC and NOTF should expand rapidly contmunity self-monitoring by supporting the training of community s el ect ed monitors /supervis ors and FLHWs. (b) ? A proposal on these lines apparently has been submitted to ApOC by TDR JAr'1r.10 Page 14 (0 NOTF should organize re-training of district and FLHF level health workers that are involved in CDTI, on data analysis and use. The aim should be development of their capacity and culture of using data as input to decision making, for example targeting special support to villages or sub districts that are outliers in therapeutic coverage. 3.1.4 Monitoring and Supervision 3.1.4.1 Monitoring and supervision are carried out by National, state/provincial and district/LGA levels in a cascading manner. The cascading approach, in practice means that the State and Provincial levels supervise and monitor CDTI at Council/District levels while in turn the district supervises and monitors CDTI at the FLHF level. FLHFs in turn supervise and monitor CDDs and community level activities. The approach provides for spot checks carried out by higher level supervisors to supplement information obtained from routine activities. Issues ofconcern 3.1.4.2 Cornmunity supervision by district and FLHWs was inadequate in some projects. Supervision checklists often were not used or were poorly completed. Furthermore, opportunities provided for supervision of other progralnmes, such as vitamin A distribution, were not exploited for CDTI supervision. Other issues encountered in some projects include: lack/inadequacy of simple transportation e.g. bicycles/motorcycles (see also under capacity building); Inability of district level authorities to pay field allowances; Community self-monitoring of drug distribution is still absent in most communities. Some district level Onchocerciasis team officers complained that the APOC CDTI reporting form was "too long," measuring "technicalities" rather than "performance". Finally Reports of supervision were often not available. 3.1.4.3 Independent monitoring which was introduced by APOC in the early stages of the programrne has been successful in enhancing implementation of CDTI . The main issue has been high cost of the exercise. The evaluation team is pleased to note that nationals from other projects are mostly carrying out the ongoing independent monitoring of CDTI. Thus independent monitoring is one of tlie functions that could be gradually devolved to NOCPA{OTF. Recommendation 4 APOC should actively support NOTFs to set up effective monitoring systems to take full responsibility of proiect monitoring before the end of APOC. While the policy should apply to all projects and countries the strategy should incorporate learntng by doing approach with intensified support to selected projects. Experience and lessons gained will be of use to other projects APOC should review and revise supervision gaidelines as necessary to enhance stmplicity of use in thefield. (c) APOC and NOTF/NOCP should intensify their effort to ensure that all projects go to scale with Community self-monitoring as soon as possible. t (a) (b) JAF11.10 Page 15 3.1.5 Severe Adverse Effect (SAE) Achievem ents and isszes 3.1.5.1 The mid{erm evaluation observed that, "in two countries, fear of SAE had been a reason for people refusing to take ivermectin. [n one of these, at the time of the visit of the present evaluation team, CDTI was suspended in 12 out 14 CDTI projects, after 62 cases including 19 deaths had occurred in previous years in two of the projects; in the other, 199 cases were recorded from 1999 to 2000 inclusively''. 3.1.5.2 The present evaluation team was pleased to note that the number of cases had reduced considerably and were almost totally confined to first year (round) of treatment. The most recent SAEs outbreak occurred in one of the two countries affected in 1999, early this year (2005) with a total of 2l cases and no deaths. However a new threat for CDTI sustainability appeared on this occasion, the launching of a misleading media campaign, relayed through the internet, giving exaggerated numbers of SAEs cases, even fatal, and calling for legal action. A quick response by the MoH prevented further harm. 3.1.5'3 As regards management of SAEs, recommendations made by the mid+erm evaluation have been implemented to the letter. APOC guidelines have been completed, adjusted and widely disseminated to countries with projects with risk areas. In one country, mallagement of SAEs benefited from the joint effort of MoH, APOC and MDP. The partners developed a programme combining prevention (RAPiOA and sensitization), patient surveillance and emergency care, as well as physio-pathological investigations, under the supervision of a senior fulltime national specialist. The experience of this expert is being used by APOC for training and building similar systems in countries with Loa loa-infected onchocerciasis zones. 3.1.5.4 The evaluation team learnt that there are sometimes strong demands for Ivermectin in hypo endemic areas. Thus it is likely that some Ivermectin is finding its way to hypo endemic areas. Recommendation 5 (a) APOC should make available the system developed by MDP/MoH/APOC rapidly to countries with high risk of SAEs. The system should be updated as soon as validated new information and advances are avstlable. (b) APOC should update the existing guidelines for SAE management and disseminate them in countries at risk of co-endemicity. (c) APOC should develop a SAE surveillance mechanism in selected hypo-endemic onchocerciasis areas, infected with Loa loa. NOTFs should organize seminars on management of CDTI in Loa loa-infected areas, witlt targeted education messages on the sAE key partners including the media. APOC and NOTFs should develop an SAE surveillance mechanism in selected hypo endemic onchocerciasis areas, especially during the post-APOC era. (d) (e) JAFl1.10 Page 16 3.1.6 Programmes in conflict/post-conflict areas Achievements 3.1.6.1 Field visits of the evaluation team included one country in conflict and two in post-conflict situations. The evaluation team commends APOC for spearheading a series of strategies and activities to get CDTI going in conflict areas, under very difficult circumstances. 3.1.6.2 ln the two countries in post conflict situations visited by the evaluation team, a number of favorable factors emerged: political support up to highest level in the Ministry of Health; availability of trained human resources in NOTFs; effective training and retraining; Mectizan distribution well organized (but still financed by APOC Programme periding incorporation into national potential drug system); functional FLHFs with satisfactory degree of integration of CDTI built in from the start; NGDO commitment; adequacy of IEC documents. All this is the result of past well thought-out efforts, which must continue unabated towards consolidation. fssues ofconcern 3.1.6.3 One great concern is non-availability of government resources. In one of the countries visited project staff received no government salaries and survived on APOC and NGDO support. Elsewhere salaries paid by State are below subsistence level and are topped up by APOC and NGDOs. Furthermore, all at risk populations living in conflict or former conflict zones can still not be reached for lack of roads, areas mined with explosives, and continuing military activities. Extensive population movements are likely to take place during and post conflict periods. Recommendation 6 (a) Regardless of the number of years a project has been in existence, APOC should consider CDTI in some conJlict areas as in its initial phases, keeptng in mind that some work cannot be done until peace comes. (b) APOC shouldfacilitate exchange of information and experiences between Countries/projects in conflict situations and those in post-conJlict. Such experience could help giving guidance on preparations for when peace comes in countries in conJlict. (c) APOC should provide adequate funds, to repeat REMO in areas where it was not completed, train and retrain CDDs and FLHWs and other priority activities. (d) APOC should docuntent experiences qnd lessons to date on CDTI in conJlict situations. Such lessons could lead to guidelines; which will be useful in future for countries that might move into conflict situations, particularly when APOC is no more. 3.1.7 TreatmentCoverage Achievements 3.1.7.1 To date 63 projects in 14 countries are operational. At the inceptioti of APOC (1995) coverage had stagnated around 8 million people treated annually. Annex 2 shows that the number of persons treated yearly in Participating Countries has increased from 14,580,000 (1997),lo 21,992,000 (2000); 28,453,000 I tJAFll.10 Page 17 (2002) and 37,307,000 (2004). There has also been a steady increase in therapeutic coverage, from the beginning of CDTI projects to over 70% (2004), in Uganda, Tanzania, Cameroon and Ethiopia. In the case of Nigeria the increase reached 70o/o in 2003 but declined to 60.7%o in 2004. In Uganda the coverage declined from72.9oh in 2003 to 7O.7oh in 20048. There has been a steady increase in Congo to 660/o in 2OO4 and to 60 %o in Malawi. DRC, a country in post conflict situation has achieved a steady increase to 45.6oh, 2004. But for Sudan, Liberia and CAR, countries still in conflict, coverage has been generally low and variable. Angola just started. fssues ofconcern 3 .1.7 .2 Analysis of data was weak, with the average data hiding a lot of inequities. The weakness in the culture of data analysis was generally widespread, see recommendations under Capacity Building. 3.1.7.3 Accepted definitions for estimating therapeutic coverage were not consistently used in the field. The mid-term evaluation recommended that steps should been taken to ensure that standard denominators and methods for measuring coverage are used, to enhance consistency and comparability of data on Programme achievements and constraints. However field findings showed persistence of the problem. 3.1.7.4 Other issues observed in the field include: Some CDDs do not record in their "log books" to indicate reasons for those who are not treated, as "absent", "refused", "pregnant" or "breast feeding". Another major area of concern relates to updating of REMO, a good assessment tool. Claims by some staff and communities in the field that some villages previously hypo endemic are now hyper endemic emphasizes the need for updating of REMO. Recommendation 7 (a) NOTF and partners should ensure that retratning programs include correct methods of recording Mectizan dis tribution. (b) Urgent steps should be taken by APOC/NOTFs to enltance the use of standard denominators and methods for estimating coverage. (c) Adequate investment should be made in REMO to catch up with updating needs. 3.1.8 Evaluation andSustainability 3.1.8.1 The 2000 mid-tenn evaluation noted that there was considerable skepticism and divergence of opinion on sustainability, let alone its measurement. The concept of sustainability has evolved over the years both in APOC and other programrnes. At the beginning of APOC, self-sustainability or self- sufficiency meant the ability of a project to continue functioning effectively, using only resources generated within the country itself. To facilitate understanding and meaningful debate, APOC has provided a clear definition on sustainability as follows: "CDTI activities in an area are sustainable when they continue to function effectively for the foreseeable future, with high treatment coverage, integrated into the available health care service, with strong community ownership, using resources mobilized by the community and the government." The evaluation team paid tribute to APOC for her leadership in this area but regretted that 8 The declines in Uganda and Nigerian lrave been attributed to the "sustainability test", stoppage of NGDO financial support to pro-iects where APOC had stopped. Details of the causes of stoppage are being studied. (NGDO Group Meeting Report, March 2005). JAI.I1.10 Page 18 the mindset of all partners may not have caught up with the present thinking. 3.1.8.2 Evaluations have been carried out in 49 projects (2002-2005) out of 53 targeted, verdict was that 35 (73%).were moving towards sustainability. The projects involved a total of 442 fifth year District/LGAs and 84 3'd year District/LGAs. 3.1.8.3 One NGDO monitored the performance of 20 projects in three countries which had received no external support for 2004 from any source, neither from APOC nor from the NGDO itself. The first year's results identified problems with treatment leading to a decline in numbers treated in two of the three countries. But in some projects, actions by local health system leadership and communities were effective in sustaining treatment. The NGDO in consultation with allthe partners will monitor the projects for a further year to know exactly where the problems are. 3.1.8.4 An important development has been the incorporation of the development of sustainabilityplans as part and parcel of the evaluation of projects. Independent monitoring of the implementation of the sustainability plans, recently initiated by APOC is proving to be a powerful tool for enhancing improvements and sustainability of projects. (The development of sustainability plans and their evaluation is a breakthrough in solving a chronic public health problem. Since June this year rnonitoring three implementation of CDTI Project sustainability has been carried out in five plans and a total of 12 will have been done by the end of next year. Lessons gained here may be of interest to CDTI devolution and integration). Issues ofconcern 3.1.8.5 The mid-term evaluation (2000) said: "There appears to be a lack of standards for drawing up budgets. APOC Headquarters staff and TCC members complain that projects over-budget considerably and staff have to spend considerable time revising budgets downwards. At country level concerns were expressed about a lack of guidelines/standards for projects to use in calculating amounts in budget -for themselves and for TCC/APOC Headquarters". The report noted "setting budget standards (with country adjustments if necessary) will make every one's life easier". Such standards are yet to be set. Experience and lessons from the monitoring of sustainability plans will make the development of the above standards easier and more realistic. 3.1.8.6 Other issues identified include inadequate synthesis and leaming from the evaluations and inadequate coordination of evaluation initiatives by partners. Recommendation I (a) APOC should synthesize experiences from reports of CDTI projects evaluated to date, leading to lessons which can be a source of inspiration to individual projects. To enhance learning by doing APOC should also convene a meeting involving projects in 3d year ofphasing out to pool experiences and draw lessons. The experiences and lessons should be widely shared with other projects. APOC and NGDO Coordinating Group should enhance thetr collaboration in planning and in carrying out evaluations of CDTI projects. t (b) (c) JJAFl1.l0 Page 19 (d) APOC should analyze the process of developing sustainabtlity plans and the resulting plans developed to date with a view to arriving at standards for drawing CDTI budgets. 3.1.9 Adequacy of impact assessment Ifypothesis and process 3.1.9.1 The APOC hypothesis for impact assessment, is that regular ivermectin treatment will: Reduce severe itching; prevent development ofonchocercal skin disease and or regress early lesions; Prevent/delay progression ofonchocercal eye lesion and blindness and may regress early stages ofocular lesions; lead to reduction in vector infectivity and to improvement in socio-economic impact status of the comrnunity. Four international assessment teams, each comprising an ophthalmologist, a dermatologist, a socio-demographer and a medical entomologist have been set up, with a total of 147 national associates. Standardized evaluation protocols have been developed and 14 sites in 9 countries selected. Achievement 3.1.9.2 Baseline survey was carried out in 2000, a second one is taking place in 2005 and the last one is planned for about 5 years from now. Although analysis of data from the remaining sites will still be ongoing over some months, preliminary results available in one project indicate significant regressions (10/l) of all skin lesions and prevalence of microfilaria in the anterior eye chamber, while posterior eye lesions stagnated at the same level, as could be expected. fssues ofconcern 3.1.9.3 Without parallel indication of the evolution of human infection, monitoring entomological parameters is losing part of its interest. Almost impossible to find ivermectin-free areas in some countries, and areas with less than 25o/o of the population treated had to be accepted; Some studies have been carried out in projects with only one treatment in a 5 year period of time. It is now clear that by year 2010 there will be projects which will not reach "the stage" for the second round of impact assessment. 3.1.9.4 Studies are based on nodule prevalence, which is known not to reflect changes of infection over short and mid-term. Measurement of microfilaria prevalence and mean microfilaria load in the communities is more accurate indicators of the evolution of human infection.. 3.1.9.5 collected. There are no economists in the impact assessment teams and socioeconomic data is not being 3.1.9.6 At least five sites will be excluded from the results of the second phase for various reasons, for example one country not selected for CDTI, another one where baseline data could not be collected because of local unrest, one annual treatments were irregular for the same reason, one where treatments started with a 2-year delay and one where analysis revealed doubts on treatment data. Consequently, the number of sites available for further survey (3rd round) has been brought down to I l. Recommendution 9 APOC should expedite the analysis of data collected during the second round paying attention to inforntation that can be obtained in sites where requirements of the protocol were only partly met. Consideration should be given to using the second round data as baseline in sttes where (a) JAFl1.l0 Page20 CDTI did not start or experienced long delay after thefirst impact assessment. (b) APOC should consider introducing monitoring of human infection in the impact assessment protocols, as it is already being carried out on a national basis by some APOC participating countries. (c) APOC should consider expanding impact assessment studies to all participating countries and in countries with only one sentinel site, the number should be increased. (d) APOC should encourage partnership with in country research institutions to collect data that can help to assess the socio-economic impact of CDTI. (e) Since lverntectin is believed to produce collateral benefits on other health problems, APOC should consider supporting initiatives to collect and analyze data on health trend, particularly for children aged 4-5, comingforfirst treatment, to observe impact of CDTI on the general health status of the population. Collection of data could be incorporated in school health examination prot ocols. APOC should, in its phasing out process, mobilize adequate funds to ensure that delayed impact as s es sment proj ects are adequately funded. 3.1.f 0 Operational research 3.1.10.1 TDR is a key partner of APOC on research. Between 1996 and 2004, APOC contributed to TDR about US $ 307,000 yearly for operational research. Research projects supported include those leading to rapid assessment of endemic areas eligible for CTDI (resulting in REMO - Rapid Epidemiological Mapping of Onchocerciasis ), rapid assessment of Loa loa in areas of co-endemicity (RAPLOA), conditions of implementation and evaluation of CDTI, sociological factors conditioning population's adhesion and participation into CDTI, CDD engagement and sustainable efficiency. 3.1.10.2 More recently, TDR has proposed to APOC a multi-country study on compliance with treatment. TDR has also obtained funds from the Melinda & Bill Gates Foundation for a study on impact of Ivermectin treatment on the survival of O. volvulus in humans (worm life expectancy and female fertile life expectancy). A study of "long-lasting" ivermectin distribution in two countries is also being undertaken: the results of such a study will be of paramount interest to better estimate the number of years CDTI has to be applied in a given onchocerciasis control programme. Developments of methods like REMO and RAPLOA have been determinant assets for APOC implementation. APOC and TDR joint support to MACROFIL 3.1.10.3 Under the MACROFIL initiative (search for a macrofilaricide) APOC and TDR each contributes US $ 700,000 annually. The approach of TDR is that of complementir-rg investment of a small pharmaceutical company in the development of a potentially macrofilaricidal drug already screened. Only one product has emerged until now, the Moxidectin, which should enter the phase of clinical study on onchocerciasis patients, if and when adverse reservations by the US Federal Drug Administration (FDA) are addressed. 3.1.10.4 MACROFIL has been instrumental in the development of ivermectin protocols and the development of the ( DEC patch-test > for early detection of low macrofilarial loads. It is now involved in Y (/) JJAF11.10 Page 2l experimentation trials of associative protocols of treatment in areas of co-infection Onchocerciasis -Loiasis- Lymphatic Filariasis, and in genomic studies of worms in patients not responding to ivermectin. fssue of concern 3.1. 10.5 The evaluation team learnt of a concern expressed by a member of CSA that there has not been tangible outcome from the Macrofil work so far. The evaluation team can re-affirm that large-scale use of a macrofilaricide would be the only altemative -and superior- means to ivermectin for onchocerciasis control. However, the team does not have technical expertise in drug development to evaluate either the relevance of the strategy adopted or the quality of the research work being performed by the company. CSA might consider obtaining the services of individual/s with appropriate skills to review the strategy and quality of the research work of Macrofil. The TCC grants and the Director's grants Achievements 3.1.10.6 Both types of the above grants are restricted, supporting relatively small operational research projects (maximum US $ 10,000 for the former and 5,000 for the latter) emanating from national investigators in participating countries. The TCC, to which 72 projects have been submitted, has accepted 24 so far, for a total budget of approximately US $200,000. Almost half the proj ects proposed and accepted came from three countries only, investigators being mostly the same. As for the Director's grants, only l0 applications have been submitted so far. 3.1.10.7 Proposals submitted are diverse and mainly deal with strategies for improving coverage, community motivation, evaluation of the impact of strike action by PHC workers on CDTI coverage, women participation in CDTI, causal relationship between epilepsy and onchocerciasis and building operational research capacities. Issues ofconcern 3.1.10'8 There are often repetitions between proposals and it appears that adequate stock of previous results and/or on-going projects is not adequately taken into account in the development ofnew proposals. There is also a big gap between on-going operational research and issues that need to be addrissid as a matter of urgency for sustainability. Examples include debate on impact of cost-sharing on sustainability, basic level of budget required to sustain CDTI, optimal reduction of number of patients per CDD in a given population, clearing local discrepancies in endemicity levels which could affect corre.age. Wayforwardfor APOC 3.1'10.9 APOC needs to act on two fronts. First APOC has over the years supported research contributing to finding fundamental knowledge to improve oncho control strategies. Strategic study areas needing future support include: longevity-fertility of adult worms after arurual rounds of ivermectin, repeated over different periods of time; Genomic changes and their significance, if any, in adult worrts submitted to annual ivemectine treatments over different periods of time; Indications of reduced ivermecin efficiency through follow-up of known cohorts of patients treated over several years; Estimation of the impact of mid-low level coverage on infection and disease in humans; In-depth physiopathological studies of SAEs; Integration studies of CDDs and other levels of health systems (building on previous studies). APOC has also facilitated meetings of operations at the country level leading to exchange of information JAFl1.10 Page22 and leaming from one another. A number of operation staff at country level have been deployed for evaluation, monitoring and impact assessment missions. The multiple contacts and exchanges with outsiders has resulted in the building of an impressive corrmon reservoir of expertise for which APOC can be proud. 3.1.10.10 The second front is that of operational research. APOC has done verywell in solving'intra' CDTI issues. CDTI now is increasingly facing difficult, "hardcore" problems, whose solution is often beyond CDTI and APOC. The best tools that APOC, countries and projects have to find solutions to these issues is learning by doing and from one another. Options and solutions should come from evidence in the field rather than from academic meetings. Take the example of the policy of devolution of management from central to lower levels, a "hardcore issue". A learning by doing approach, selecting a handful of 'suitable' candidate countries, providing intensified support, monitoring progress and pooling experiences as basis for obtaining and disseminating lessons is the way forward. Skills in operational research will be critical to countries when APOC is no more. APOC needs to work in both fronts, balancing its support. But clearly the capacity of APOC to address the issues outlined in this section needs to be considerably augmented. Recommendatiort 10 (a) APOC management should gtve more priority to operational research. A significant asset should be the creation of one Senior Staff Research post at the HQ for the coordination of APOC research activities and making the Programme to participate actively at the international scienttfic level. APOC should strengthen collaboration with TDR, in terms of joint 'reflection', review of Programme and CDTI project needs of applied/operational research to address new/emerging problems. (c) TCC should reinforce its work in reviewing existing and emerging research issues and together with APOC management provide more technical input to TDR, at the stages of protocols validation, follow-up, discussion of results and evaluation, and implementation. (d) APOC should advocate and support training of key CDTI staff particularly, district coordinators andproject managers on methodologies and other key aspects ofoperational research. (e) APOC should arrange for an Expert review of strategt and quality of research on Macrofil. APOC management and TCC should make a distinction between projects of very local interest, aimed atfinding solutions to local problems, and proposals of larger scale, possibly multi-centre, the results of which could be of wider interest in participating countries and which consequently could be eligible for somewhat higher and more attractive budgets. APOC should advertise its support to operational research widely in participating countries with a view to reaching a larger spectrum of countries and investigators to be involved in the research supported by APOC. APOC should participate actively in futuristic thinking and research aimed at finding directions for the control and eventual (eradication?) elimination of oncho. t (b) 0) G) (h) IJAFt 1.10 Page23 3.1.11 Implementation of Projects: Achievements and delays 3.1. I I .1 The cumulative number of approved "CDTI projects" has increased enormously over the years, from 7 (1997); to 48 (2000); 65 (2003) and 96 (2005) and is expected to reach the ultimate number of I 1 I by 2007. There are also four additional projects in support of the complementary strategy of vector elimination in selected foci, while seven ("HQs support projects") have been established in some countries to support the Coordination of the Programme at national level. As of march 2005 five projects (3 in DRC, one in Uganda and one Angola) were still in the planning stage, had not been approved. At that time there were also 24 projects which had been approved but not launched. Clearly most projects which had not been approved or approved but not yet launched were in conflict or post conflict situations. The basic APOC support is 5 years, followed by three years of "phasing out" period during which sustainability plans are implemented. These 29 projects will not have received the basic APOC support by 2010. 3.1.L1.2 Other reasons for delay, besides conflict and post conflict situations include, Serious Adverse Effects (SAEs) and of managerial or procedural nature. The Iatter include delays in HQ due to heavy workload on APOC which has to cope with a significantly higher number of projects than initially expected. From the field, difficulties lay in uneven quality of project proposals, or non-compliance with WHO rules in financial accounting and management. 3.1.11.3 In view of these delays, it is already foreseen tbat 49 projects will still be receiving basic support of 8 years by 2010, with numbers declining to 5 in 201 5 . The EET had the opportunity to discuss these delays with APOC management, and their implications for sustainability and possible solutions. The EET commends APOC management for its foresight and sound analysis and supports the Management proposal that APOC support be extended beyond 2010 to 2015. Recommendation 1I In view of the delays that a number of CDTI projects have experienced and taking into consideration the importance of ensuring that the projects, some of which will operate under dfficult post con/lict envtronment, are brought to satisfactory conclusion, the external evaluatton leqm recommends the extension of APOC Trust Fund support on a decreasing scale to 2015. 3.1.12 The vector elimination projects in APOC Rationale, selection of projects and achievements 3.1.12.1 The hypothesis for focal vector elimination operations in APOC is the rapid, cost effective, complete elimination of the vectors, in small, isolated foci, which, when combined with Ivermectin treatment and with a post-larviciding entomological surveillance over a few years, can put a time limit to the endless annual rounds of ivermectin treatment. A source of inspiration for the elimination was six successful operations carried out in confined foci in Kenya and Uganda in the 1940's to 1960's; the vector in five foci was S. neavei and S. damnosum in one. Four foci were selected (two in Uganda, one in Tanzania and one in Equatorial Guinea) for APOC support, following technical assessments by TDR consultants. 3.1.12.2 Thus the focal eradication strategy was mainly aimed at the control of onchocerciasis in East Africa, where the distribution of the disease is notoriously patchy ; this is especially so in areas where the vectors belong to the Simulium neavei complex, which show restricted flight range and great vulnerability to ecological changes, but such is also the case in East African areas where the vector species are members JAFl1.10 Page24 of the S. damnosum complex, more diversified and apparently less migrant than their sister species from Western and Central Africa.. The four foci selected (out of six) are indicated below together with an update on developments and achievemen s for in each. 3.1.12.3 Itwara focus (Western Uganda) was taken over by APOC from GTZ (German cooperation) in 1998 for "Completion of Vector Elimination". GTZ had successfully canied out vector control operations on the main focus, in the period 1994-97 . Adult S. neavei s.l. has not been observed since I 997 in the main focus, the vector is considered eliminated. In the sub focus, by 2002 annual rounds of selective treatments had progressively brought fly and larvae occurrence down to zero, Monitoring is now confined to three sites. 3.1.12.4 ln the Mparnba Nkusi (Uganda) focus, where no vector control had ever been attempted, the national plan for feasibility studies was accepted in August 1998, but because of delays in APOC funding, annual larvicide rounds started in 2002. The vector has not bee recorded in peripheral areas since 2003. Monitoring is still being carried out after the last larvicide campaign which ended in March (2004). TCC.20 recommended the APOC of the requested budget for the 2005 campaign. 3.1.12.5 In the Tukuyu focus, the first larvicide round took place at an inappropriate time (rainy season 2001,-2002 instead of dry season 2001) and flies reappeared a few months after cessation of larviciding. Although the second round was apparently properly performed, again biting flies reappeared shortly after the campaigrr. No appropriate action was taken either in Tukuyu or in Ouagadougou for analyzing the results and planning for the next round, with the result that no treatment was carried out in 2004 and that the 2005 campaign had to be planned in a huny following strong intervention by the TCC and APOC management. Treatments were to start by July 2005. 3 .l .12.6 After a ground larviciding campaign in 200 I covering the Northern range of the Bioko island, it was concluded that it was not possible to reach breeding sites fully. Aerial larviciding covered the whole island in March 2003. However, biting flies could not be fully controlled and recrudescence was observed by July. The 2004 campaign could not take place owing to insecticide shortage; instead larviciding was canied out from February to March 2005. Fly has not reappeared since March, despite intensive and reinforced monitoring. Other Oncho vector control activities 3.1..12.7 Reasons for delays in implementation of activities include o Ecological and environmental difficulties of black-fly elimination, especially S. damnosum s.l; o Management at the country level has not been satisfactory; technical, adrninistrative or financial levels ; o APOC heavy administrative mechanisms, when routinely applied, may be not be fully adequate for operations needing flexibility and reactivity. TCC could not always give VEP the priority it deserved. As a result, TCC somewhat under reacted to slow progress, and did not find its way to making innovative proposals. 3.1. 12.8 Results from the Itwara and Mpamba-Nkusi focus here confirm that S. neavei elimination is an achievable objective. Elimination of S. damnosum s.1,. which is may be notoriously more difficult, is uncertain. I ,JAFl1.10 Page 25 Recommendation 12 (a) In Bioko, theProgramme should maintain entomological monitoring over another two-year period. Should biting fly reappear meanwhile, a protocol could be worked out for a national project In Tukuyu, APOC should ensure that optimal technical assistance is provided to the project on a current basis for the time of the campaign, both for treatments and entomological evaluation. In due course, the procedure should be similar to that in Bioko by the end of operations. (c) In Itwara the questton raised is that of the certification ; the TCC suggested that an ad hoc meeting of experts be convened in 2006 to decide about the relevance and procedure of such action. The evaluation team endorses this recommendation. The evaluation team also recommends that an extensive publication, on the ltwara success story be undertaken. (d) In Mpamba-Nkusi it is recommended that one more annual larvicide round be pedormed and intensive entomologicql monitoring network. Considering that, at the end of 2005, the Mamba- Nkusi will be the only APOC-supported VEP showing good sign of repeating the ltwara success, and that the tvvo projects appear as pilots for future elimination of S. neavei foci in East Africa, APOC should give the projects all facilities to reach a successful end, notwithstanding the 2005 deadline for larviciding (e) Without embarking in nuisance control operations which are not within its mandate, APOC should bring technical assistqnce to Participating Countries intending to carry out such operations on their own. The highest health authorities in countries implementing Vector Elimination programmes should be invited to take stock of the entomological knowledge brought in or supported by APOC with the aim of developing of a national operational capactt not onlyfor onchocerciasis control, but also for transmission control of other vector borne diseases which jeopardize development. 3.2 Programme management 3.2.1 Programmegoyernance A chievem ents and issues 3.2.1.1 Preparation of and participation in CSA, JAI', TCC and other meetings, within or outside the APOC institutional framework, are heavily taxing for APOC staff. Numerous meetings are necessary or essential, but the full structure places a severe strain on the Programme's limited human resources, not to mention expenditures involved. Reflection must come from within: the Executing agency and Programme Management can steer the course towards steadfast rationalization. Recommendation I3 In a streamlining effort, the Programme Management with the support of CSA, and JAF as required, must seek a rational reduction of statutory and other meetings, thus alleviating the heavy task involvedfor APOC Headquarters. (b) U) JAFl1.l0 Page26 Role of CSA and the TCC in enhancing, consolidating, and ensuring sustainability of Programme operations 3.2.2 Role of CSA 3.2.2.t CSA is composed of representatives of four "sponsoring Agencies" e, while a representative of the NGDO Coordination Group and a representative of the sole donor of ivermectin (Merck & Co, Inc.) are "invited" to participate in the sessions. CSA thus brings together representatives of partners acting at the international level. Meeting at regular intervals during the year, CSA reviews plans of action and budgets and examines reports, for transmission to JAF with any observations. Among its tasks, CSA plays a prominent role in the selection of TCC members. In practice, under the strong impetus of the World Bank as Fiscal Agent of the Prograrrrne, and WHO as Executing Agency, including the Director of APOC, CSA maintains the course set by JAF. 3.2.2.2 At CSA's l09tr'session (Geneva, March 2005), sustainability of projects, a cornerstone of CDTI, featured as one important concern during the discussions. The focus on sustainability is critical as the Programme develops, and as an increasing number of projects reaches the phasing out stage. In this respect, the presence of resource persons to take an active part in debates on sustainability should be considered. They could be invited on an ad hoc basis, i.e. with no need to change the Memorandum. One example would be the NOTF representative of a Participating Country, on a rotating basis, when sustainability or closely related items are on the agenda. Thus NOTFs could bring to CSA's attention, in the light of APOC decreasing involvement, actual funding gaps in activities essential to sustainability such as training and retraining, EC, advocacy. CSA is the appropriate body to see to it, that sustainability in general and on specific projects succeeds. 3.2.2.3 Although it has been repeated by CSA that there will be no more call on donors for onchocerciasis control beyond the existing APOC Trust Fund, CSA could play a role, either in authorizing the utilization of the existing budget beyond the initially accepted APOC life span, or in initiating fund raising toward other categories of sponsors (those interested in PHC, integration or CDTI promotion, for example), or in supporting multi-country initiatives by an APOC (and ex-OCP) Participating Countries coalition directed to specific prograrnmes like those devoted to neglected diseases or poverty alleviation. Recommendatton 14 (a) CSA, in enhancing consideration of sustainabtlity as a cornerstone of CDTI, should consider calling additional resource persons, to participate in deliberations on relevant agenda items. (b) CSA, as long as it is in existence and with the post-APOC era in mind, should play a role in maintaining interest of specific sponsors, and prompting or supporting multi-country initiatives directed to programmes of relevance to onchocerciasis. 3.2.3 Role of TCC 3.2.3.1 In 1996 the APOC Programme Document for Phase I specified that < the main functions of TCC will be to review applications for funds for ivermectin distribution projects, including both their technical justification and financial feasibility, and to review the implementation of funded projects. The TCC will e The World Bank and WHO, together with UNDP and FAO, are referred to as the four "sponsoring Agencies". With time and developments, one sponsoring agency, FAO, had diminished interest compared to the role it had to play during OCP, and finally withdrew from CSA. TINDP interest was also considerably weakened a JAF11.10 Page27 also be expected to advise the Programme on technical and research matters )). At that time, TCC was composed of l0 members: 5 independent scientists, two representatives of the NGDO Group, one of The Carter Center, one of the Ivermectin Experts Committee and one of the EAC/OCP. 3.2.3.2 In 2000 the mid-term External Evaluation stated that the TCC's "overload mirrors the APOC overload at all stages before, during and after the review process". The External Evaluation recommended that TCC role should be urgently re-defined to align with its technical mandate, and it suggested that APOC Phase II could be an opportunity for formal TCC restructuring. In 2001, following the mid-term External Evaluation, the Programme Document for Phase II decided that < APOC will realign TCC focus to that of technical, implementation and operational research considerations >; the composition was enlarged to twelve < scientists/experts > appointed by the WHO/DG upon recornmendation by CSA, one of the twelve being selected by Merck & Co, and three of the remaining eleven being proposed by the NGDO Group. The members are presently eleven. They are knowledgeable persons in onchocerciasis control and CDTI in APOC countries, highly dedicated to the success of the Programme, and very careful in fulfilling their statutory tasks. fssues ofconcern 3.2.3.3 TCC has now been relieved from financial evaluations, and technical reviews have been standardized. However, the number of projects in APOC has increased considerably, which entailed multiplication of monitoring and evaluation reports. Consequently, TCC semi-annual sessions are still minoring the notorious APOC management overload. For example, at its 20th session in March 2005, TCC reviewed 29 technical reports, 8 new proposals and l0 operational research proposals. Thus very little time is left to TCC members for prospective and in-depth analysis and reflection on topics crucial for APOC objectives attainment and sustainable achievements, such as strategic and operational research. As one example, TCC could plan to include a sustainability item only in the agenda of its next 2lst session (September 2005). 3.2.3.4 To an extent, TCC's overload has impaired its power of initiative and innovation, as well as its scientific analytical activities. Hence, its guidance role for APOC strategy should be more thorough and adequate, especially in the present situation when issues like decentralization, need more and more foresight. ln the same way, as mentioned earlier, TCC's role in operational research is commenting on projects initiated and conducted by TDR. Although some TCC members actively participate in country field visits, on evaluation and impact assessment missions, TCC tend to be viewed, at project level, as parl of APOC management. Its role as TCC could generate more interest. Recommendation 15 (a) The scientific weight and capacity of TCC should be reinforced through the selection of adequate profiles at the time of contracts renewals. For example, skills in community development should be included TCC should pay more attention to issues crucial for the future of APOC (e.g.: CDTI sustainability at various levels, integration) and include regular planned working session on these topics in its annual meeting agendas. (b) (c) TCC should reinforce or develop its proactive role in proposals of new research activities, evaluation of protocols coming from outside and follow-up of on-going research projects, espectally those conducted with APOC partial or completefinancial support. (d) Given the need to have a more participatory approach in the planning and management of JAFI1.10 Page 28 activities, and the current heavy workload of TCC, NOTFs should seek supportfrom a network of expertise at the country level as needed, to enhance local ownership and prompt proviston of technical support 3.2.4 APOC headquarters Office: adequacy and effectiveness of Programme management, Financial management, and staffing 3.2.4.1 The overall functions of APOC - mainly administration of the Programme; launching, follow up and evaluation of projects; support to countries; support to and attendance of meetings of statutory bodies - are carried out efficiently. A PO C Pro gram m e m anagem ent 3.2.4.2 An important issue noted was weakness in communication between levels, and partners. For example observations made by APOC Headquarters (with TCC's concurrence in some cases), refer to cases of inadequacies of project administration, late submission and shortcomings of project proposals, technical reports, and of sustainability plans. On the financial aspect, APOC notes insufficiencies in financial management in spite of clearly written procedures, delays in submitting monthly financial reports, uneven quality of statements and attachments provided, and also unauthorized expenditures using funds left over under a Letter of Agreement (LoA) before the signature of the next one. Also, there have been cases of misuse and mismanagement of funds. 3.2.4.3 Fromthe projects side, widespread complaints were heard on APOC slowness in signature and transfer of funds, which may affgct performance: some projects have remained unfunded for up to several months; certain demands by A|OC on reports and documents are seen as overly heavy; equipment purchased is not always adapted to requirements; rationale is not always clear for cuts made by APOC on project budgets presented; mana$ement decisions by APOC without sufficient consultation with the field; lack of feedback on reports sent" especially financial ones, often leaving the actors in the field somewhat bewildered and unable to draw lessons. Some NGDOs felt that funds were often not forthcoming in sufficient amounts and might havle been earmarked for future extension of APOC 3.2.4.4 In a number of specifif cases there are acceptable clarifications, and grounds for early corrective action. Regular supervisory visit$ would resolve some of the major issues related to financial management (see below) Recommendation 16 APOC should intensify communlcation with project levels and partners particularly through field visits to enhance a shared understandin!, ofproblems encountered and possible solutions. 3.2.5 APOC Financial Mailagement 3.2.5.1 Financial managemerlt covers, inter alia, accounting for Headquarters operations; annual budget preparation and budgetary contiol; treasury, including receipts and payments; preparation of monthly and annual financial statements; firiancial analyses; disbursements of funds to projects; contribution to the preparation of annual LoAs for all projects; control of projects monthly expenditure submissions. To face these tasks, the financial staff at Ouagadougou is of excellent professional quality and liighly motivated. JAFl1.l0 Page 29 Achievements 3.2.5.2 Some improvement measures have recently been planned, including schemes to provide additional staff training at headquarters, more country visits to monitor and train personnel at country offices and projects and financial management reviews. These measures if and when implemented should help to decrease complaints noted between APOC and the field. 3.2.5.3 Intemal and external audits, lastingtwo to three weeks each, are carried out everyyear. Each audit team has some three members. Reports of internal monitoring are made available to the auditors. APOC management has to respond to all audit queries. For example, following up on earlier internal queries, APOC has developed guidelines entitled: "Management of APOC Funds by NOTFs: Financial and Administrative Procedures (2004)" which has been disseminated to projects. The emphasis of audit is on APOC headquarters. But while extemal auditors confine their visits to APOC headquarters, internal auditors occasionally also visit the field. 3.2.5.4 To reinforce its administrative and finance network, and enhance decentralization of certain functions, APOC covers the costs of six adrninistrative and finance assistants, employed by the respective WHO Country Offices, trained in APOC Programme matters and remaining in close liaison with APOC. They function in the framework of relationships between APOC headquarters, NOTFs and WHO country offices. Normally, the work of such officers include functions such as: the training/retraining of project staff and other capacity building measures in the field; following up on the processing by APOC Headquarters of expenditure statements; conducting internal operational and financial audits of NOTFs and projects; assisting in the preparation and processing of Letters of Agreement; assisting the planning and budgeting for NOTF and project activities; control/supervision of contributions by govemments and NGDOs, in the framework of the established responsibility for NOTFs to cover 25oh of the ivermectin distribution projects Issue of concern 3.2.5.5 A number of problems were identified as follow a o a Some administrative and financial assistants in the field attended to a wide range of responsibilities, others however were limited to the conduct of ad-hoc functions such as the review of project expenditure statements Effectiveness of the financial management system appears to be affected by staff shortage at Ouagadougou, as well as, in some cases, by lack of language abilities. Constant backlog of statements to be reviewed and cleared (a backlog of some 170 project expenditure statements with up to five months delay was awaiting review and clearance at the time of the team's visit). Reviews of monthly project financial reports are cursory only and on a sampling basis; feedback provided by APOC, is insufficient. LoAs, a major repetitive task has to be renewed annually due in line with WHO/APOC's budgeting procedures. Delays have sometimes resulted in late LoA approvals, with corresponding delays in the related disbursement of funds to projects, and in project implementation itself. One solution suggested was a multi-year formula - with a clause "subject to availability of funds" introduced in the LoAs. Such an arrangement should be flexible enough since many projects, even within one single year, do not develop exactly according to plan or to standard patterns. Training and supervision ofcountry and project level personnel needs to be intensified a a JAFI1.r0 Page 30 a Bottlenecks in the flow of funds, from the source (World Bank) to the ultimate beneficiaries (the communities). Completp and accurate financial reporting along the chain upwards remains problematic. Another point was that APOC should develop and implement a system to plan for its treasury operations (i.e. inflows of donor contributions, interest eamings, etc. and outflows for all expenditures) to facilitatd its financial planning of headquarters and field operations, including fund raising, over a medium term horizon. 3.2.5.6 The evaluation team discussed the above issues and the emerging solutions, within the time available. It was agreed that more in-depth analysis of the impact of current arrangements and optional solutions is needed. Work under recommendation 17 (a) should link to that under recommendation l8 (b). Recommendation 17 (a) APOC should consider conducing a detailed revtew, possibly resorting to WHO Headquarters or outside consultants, of its financial management systems, with the objective of strengthening the organization's /inancial management, aiming inter alia at improving oversight of country/project activ,ities and reducing delays in processing of expenditure statements, disbursements, and letters of agreement. In view of the close relationship between work in this area and Staff manaSement, the implementation of this recommendation should be combined with that of Recommandation I8 (b) below. (b) APOC should clearly define the responsibilities of all its financial and administrative assistants stationed n WruO country offices, to ensure they can play a role conducive to decentralizing certaih APOC functions, and to alleviate the administrative burden on APOC Headquarters in thdse matters. In consultation with WHO, APOC should also consider expanding the resppnsibililies and authority of such fficers, inter alia to screen and recommend approval or otherwise of project expenditure statements, advise on operational budgets, take part in'operational audits and reviews, and provide training. 3.2.6 APOC staff managenrlent 3.2.6.1 There were 5 staff mpmbers originallyin 1996, and 20 in 2000. Until the end of 2002,i.e.the end of OCP, APOC was relying pn OCP for information systems and for administrative support (including personnel, budget and finance). jA restructuring carried out in 2003 was aimed at improving management. Moreover in 2003, 32local staff were added: 25 taken over from OCP upon its winding down, and 7 new. As from January 2005, all :Z dtaff moved to a fully-fledged fixed-term WHO status. Conditions were superior, security of tenure enhaflced, with beneficial implications for continuity and staff morale. In total, there are 66 posts in 2005 (61 ptaff and 5 vacancies to be filled), i.e. 14 Professionals, and 52 General Service. 3.2.6.2 Staff now appears leds overwhelmed with quantity, though there is still some concern with the quality of the output. In recent years more attention of staff is going to effectiveness and consolidation of APOC itself. More attention is [eing paid to capacity building within APOC Units, for exarnple increasing ability to produce computerized data. One senior staff, summarize it all, alter egos are being established at senior levels. JAF1l.10 Page 3 I fssue of concern 3.2.6.3 Nine of the 14 professional posts (one was vacant at the time of the evaluation), are occupied by staff scientifically or technically well qualified, working under a WHO short-term status (STP), to whom more costly fixed-term contracts have not been given for budgetary reasons. Keeping staff under STP contracts year after year, aparl from being a source ofinsecurity for the staff, the practice is bound to enter into conflict with WHO policies (aft.er 4 years, STP contracts must be converted into fixed-term status or the incumbent's STP contract must not be renewed) unless some exceptional ad hoc formula is agreed upon between APOC and WHO Headquarters, which does not seem within easy reach. Therefore, there is a serious precarious element in APOC staffing situation at a time the Programrne is at its peak and fully qualified and knowledgeable staffrequired at its full strength. 3.2.6.4 The evaluation team has considered whether staff increases would contribute effrciently to face the workload, notably in the financial management field. The fact that the Programme is at an advanced stage and will soon develop plans for transfer of activities to the countries, might militate both ways: on the one hand it may appear late to absorb new staff close to a phasing out period when the size of the organisation should gradually decrease, and on the other some reinforcement may be appropriate at a time when a specific and considerable effort must be deployed to properly transfer functions to countries. The evaluation team thinks that the options need to be examined in detail. Recommendation 18 (a) APOC and l4tHO Headquarters should seek a responsive solution for maintaining those STPs whose contracts cannot be converted into fixed-term status, including ways to allow the continued employment of essential professional staff APOC should conduct a detailed review, possibly resorting to WHO Headquarters or outside consultants, of the adequacy of staff and making appropriate recommendations to assist the organtsation toface its considerable task and challenges from this evaluation. 3.2.7 The Country Level Programme Management: present and prospective 3.2.7.1 NOTFs are the cornerstone of Programme management at national level, as they bring all stakeholders together, in a fundamental partnership for establishing major policies, planning and setting the course for implementation by their secretariats, the NOCPs. NOTF consists of the Ministry of Health normally in the chair at senior level, NGDOs, WHO country offices, NOCP, some MoH heads of departments, and resource persons. 3.2.7.2 With the rapid increase in the number of projects in recent years the volume of work for the secretariats has increased hugely. Some NOCPs appear to be too tied up with routine work such as writing reports and coordinating monitoring and evaluations exercises, with little time available for develop-ent and overseeing work. APOC Headquarters has set up projects for support to NOTFs, most of whi.li hulrt decisively contributed to improvements in management. (b) JAF1l.t0 Page32 fssue of concern 3.2.7 .3 On decentralisation of functions from the headquarters to country level, there is as yet no clear plan. It was understood from APOC management that attempts to decentralise certain functions to the country level on the lines recommended by the 2000 Evaluation had not succeeded. The evaluation team would like to reiterate that reconhmendations should not be seen as everything from policy directive to strategies, plans and activities. Plans for decentralisation should be drawn out in a participatory manner. It might also be useful for APOC to prepare a review paper that will clariff some basic issues on decentralisation based on available information. For example to which mechanism/s at country level can APOC decentralize to and why?. 3.2.7 .4 Pace of decentralization will vary in different countries according to their capacities. Thus in the selection for the initial NOTFs fot intensification of decentralization, priority should be given to those that are likelyto generate practical experiences for other countries. Candidate areas for decentralization include: Evaluation, monitoring and supervision of projects under implementation; Review and approval of technical, financial management and endorsement of research proposals and subsequent monitoring and supervision. 3.2.7.5 Other issues related to the NOTF mechanism and its future, are discussed under partnership (Section 4 and recommendation 26 are relevant). Recommendation 19 (a) APOC management phould prepare a position paper on decentralization of selected functions from Ouaga to the country level. The pqper should draw on the experiences of other programmes and agbncies and what can be expected. This paper would be an important input to the participatory planning work below. (b) APOC, in liaison w)ith NOTFs, WHO country Offices, and other partners, should develop a planfor transfer of functions to countries, defining areas of activity to be transferred, ways and means, role of NOCP/NOTFs, WHO country Offices and other partners, as well as time schedules aiming at an ffictive transfer before the end of the Phasing out period. lltHO support to the process should be provided in response to individual country needs. The overall plan should include learning by doing approach, begtnning with intensified support to a few countries, whose experience can be a source of inspiration and practical use to other countries. (c) As part of the transfer exercise, APOC should advocate with governments to examine their roles at each level in the country, and identify areas that need to be strengthened. More generally, APOC should establish programs to better support NOTFs in helping sensitize, advocate with and support governments in an effort to pool experiences under the Programme, and also help determine basic levels of funding under government budgets and on the allocation of onchocerciasis funds between dffirent levels of the health system JAFl l.l0 Page 33 Funding at the country level 3.2.7.6 The Memorandum for APOC, Phase II (2002-2007) and Phasing out Period (2008-2010) stipulates that NOTF will be responsible for 25o/o of the ivermectin distribution costs (in cash or in kind), which will not be available from the APOC Trust Fund. No formal rules or agreements, other than what is spelled out in the original "Project Proposals" issued at project inception, have been set up among the various parties as to their respective obligations or the specific modalities of the contributions to be made; funds are usually provided on the basis of ad-hoc and arbitrary decisions, and it appears that only in rare cases are mechanisms in place to ensure or veriff that the 25o/o stipulation is being adhered to. 3.2.7 .7 There are no records kept at APOC or NOTFs of contributions made by governments, NGDOs and other local partners, which could facilitate a coordinated and regular follow-up on their compliance. It is also important to rethink aspects of 25o/o contribution. For example to enhance sustainability, the contribution of the government to CDTI should gradually increase as that of APOC decreases so that by the fifth year the government contribution becomes 100%. Thus by the 3'd year, the government should be contributing about 50% ofthe total project budget. The issue is 50% ofwhat level ofbudget? It should be noted that much higher levels of budget are justified during the development phase of projects. In this respect data coming from the Independent monitoring of sustainability plans is revealing. First, levels of sustainability budgets are modest. One of the four projects evaluated had received APOC funding; the other three had received none two years. All projects were found to be doing; each had received increased funding from the district level. Both geographical and therapeutic coverage remained high. There was concem in one of the three projects, which had not received APOC support, as to whether achievements could be sustained for long periods. 3.2.7.8 Data on contributions of the countries visited was scanty. It would be appropriate and highly welcome if governments could envisage covering or enhancing expenditures for training and capacity building, measures connected with CDTI including administrative functions, transportation of CDDs or NOTF staffthrough the provision of vehicles, motorcycles or bicycles. Support for CDTI is increasingly being requested and provided under different funds including those related to Highly lndebted Poor Countries (HIPC) (for eligible countries) and Sector Wide Approach (SWAP). Recommendation 20 (a) APOC management should prepare an analytical paper on issues related to NOTFs responsibility for covering 25% of the ivermectin distribution project costs, for review and decisions by TCC and CSA. Meanwhile efforts should be stepped up by APOC in keeping track of contributions by governments and NGDOs while prompting compliance with the obligations. APOC should develop and implement, together with NOTFs, a mechanisrn whereby planned and actual contributions by local partners can be better monitored. This mechanism should produce triggers for corrective action on the part of NOTFs and, if needed, APOC, such as targeted advocacy measures from within or outside the country. (b) In countries becoming eligible for participation under the Heavily Indebted Poor Countries(HIPC) debt reduction program, governments should be formally urged to include the onchocerciasis Programme in the list offunding priorities. Similarly, health lending under any Sector-Wide Approach (SWAP) project should be designed to include funding for the oncho cercias is pro gram. a JAFl1.10 Page34 3.3 Integration of CDTI into overall health services 3.3.1 Integration here refers to incorporating elements CDTI into the health system. There are two expected benefits from integration. First is efficiency through synergism and economies of scale in the use of scarce critical resources. Second is convenience to users of services who can get several services in the same place and or concurrently. ln practical terms integration means there should be as much fusion of services as possible when the user is reached. Both benefits are good sustainability of CDTI. 3.3.2 APOC has developed and uses a quantitative tool based on responses to three questions, to assess extent of CDTI integration at the National and State/Provincial levelsto. The questions are whether: "there is a written work plan and if it shows how implementation of activities in an inte$ated manner will be achieved"; "Staff combines CDTI activities with those of other progtarrrmes, where this is relevant"; and "Staff combine two or more tasks on a single trip" Findings show that the first area, availability of integration plans is the weakest. About half of twenty-nine projects evaluated, mostly in 2003, achieved a level of integration considered by APOC as adequate to enhance sustainability (a score of 2.5+ out of 5.0). 3.3.3 CDTI has been used as entry point for PHC meaning that oncho infrastructure, was gradually expanded to provide other services like treatment of malaria where there was nothing (e.g. Sudan). Iintegration in terms of combining selected components of other programmes with CDTI is another area of considerable achievements. Examples of such "add-ons" include provision of Vitamin A supplement, prevention and treatment of filariasis. Other areas being introduced or under discussion include: Malaria home-based management, prevention and treatment of schistosomiasis, intestinal worrns, HIV/AIDS; Disease surveillance, NPI, Blindness Prevention program and Reproductive Health. 3.3.4 Examples of sharing of resources in the form of pooled transport are increasingly being reported by evaluations on sustainability of CDTI. Another example relates to Mectizan@ supply and storage systems which in many projects are not only within the government system but are the same as systems used for the supply and storage of other programme drugs in the country. fssues at dffirent levels of the systen Higher levels, NOTF and State 3.3.5 Many integration issues fall outside the role of CDTI, to the overall health services (MoH) and sometimes even outside MoH, to donor agencies and international organizations. 3.3.6 Integration will not just happen. While MoHs and NOTFS acknowledged that integration was the lifeline for sustainability of CDTI, evaluations on integration revealed almost non-existence of "integration plans". 3.3.7 The CDTI staff in most of the facilities visited had other responsibilities, which is good for integration and sustainability. l0 Pu.urn",".. assessed included: Whether there is "a written work plan and if it shows how implementation of activities in an integrated manner wrll be achieved"; Staff combines two or more tasks on a single trip such as monitoring / supervision, training for CDTI, HSAM, fetching records and delivering of Mectizan and staffcombines CDTI activities with those of other programs, where this is relevant (e.g. supervision, training, HSAM, Mectizan delivery). FLHFs JAFl1.10 Page 35 3.3.8 While FLHFs placed as they are at the critical level of supporting communities, previous evaluations in practically all projects showed that this was the weakest link in the health system, which is not good for sustainability. CDTI may be sustainable at the community level, but this will not be possible unless FLHFs and districts/LGAs are able to provide ivermectin and supervision regularly. The present evaluation team was pleased to note that some improvements are taking place partly in response to the 2000 adverse findings. Staff in most of the FLHFs was knowledgeable on CDTI and increasingly taking responsibility for training and supervision of CDDs but under diffrcult circumstances, for example with no transport and a charged schedule. In the past FLHF higher-level staff mostly bypassed staff. Community level 3.3.9 The piecemeal approach in the delivery of health care at he community level, each agency or prograrnme with its own package makes it practically impossible for communities to integrate their own activities with those being directed from outside. Even studies aimed at finding ways of enhancing "integration" come with their packages to test which additional interventions can be added without rocking the boat. They forget that integration at the community level is not about selecting this or that intervention but arriving at an effective and efficient mix of service that can be combined with community's own activities to improve health. 3.3.10 A recent development in APOC, support to high level meetings organized byprogramme managers of programmes involved in community interventions, in three countries, augurs well for the future. In one of the countries CDTI strategy was made a national policy for all community based programmes in the whole country. The activity can be seen as being on the borderline between APOC and MoH mandates. What is certain is that the approach and momentum being built is unique. This step has evaded many prograrnmes including Primary Health Care whose progress was diverted by red herring technicalities like developing essential health packages. The initiative has potentials of being one of the most important contributions of APOC and CDTI to health systerns, but it has to be nurtured. Impact of CDTI on the quality of local health services 3.3.11 CDTI has contributed a number of approaches and tools that have enhanced the improvement of health services. A number of programmes are also using tools developed under CDTI, modified as necessary. Examples include monitoring and evaluation tools and instruments. Training of medical personnel and CDDs, has contributed to improvement in quality of care. Nurses in the field indicated that CDTI had given them a better understanding of communities and their needs, which has enabled them to improve services to communities. Recommendation 21 (a) As overall integration of health services is beyond CDTI, Ministries of Heath, in collaboration with partners including donor agencies, should find ways of enhancing integration of health services and develop an appropriate strategl and plan. (b) In view of the many operational problems being encountered in the integratton of health services, Ministries of health, together with partners should identify critical problems and carry out operational studies in selected geographical areas (Including districts and communities) to find solutions). APOC and NOTF should intensify effort to make CDTI experiences and lessons readily available to other programs. (c) JAFl1.10 Page 36 (d) Each NOTF should organise a special meeting on integration of CDTI. The meeting should take stock of achievements to date as well as issues and define future strategies and activities. In view of the importance of this item it would be useful for APOC management and or a member of TCC to participate in the meetingtl. (e) APOC should take stock of experience gained in CDTI integration in all participating countries and make lessons learnt widely available. 4. Partnership 4.1 The various partners in synergy at all levels 4.1.1 The APOC Programme rests on a wide partnership, at the international and national level, each covering one or more of the following areas, all ultimately converging towards sustainability: funding; planning; prograrnme development, coordination, supervision and monitoring; health education, sensitisation, advocacy, and community mobilization (HSAM), training re-training and capacity building in general and research. 4.2 The international level 4.2.1 APOC has an intemational partnership network largely inherited frorn OCP (see Introduction). Several of the international partners are likely to phase out and frnally withdraw when APOC winds up. But some assets are likely to remain. WHO, is expected to continue to play a prominent supportive role, notably through the Regional Offrce for Africa (AFRO). Efforts are being made by AFRO to integrate onchocerciasis into their agenda, and gradually cover needs for surveillance, overseeing, and exchanging of information. AFRO already keeps up-to-date with APOC activities, participates in CSA, and is invited to TCC and JAF. Also, within its "lntegrated disease surveillance strategy", AFRO bas included onchocerciasis in its priority list of communicable diseases for surveillance. 4.2.2 Furthermore, it is expected that the WHO Multi-disease Surveillance Centre (MDSC) established in Ouagadougou, which inherited the regional surveillance infrastructure that OCP developed over the years, will gradually join the APOC partnership, and define a post- APOC role. Other United Nations organizations, such as UNICEF, ffioy contribute in various ways to be determined. Finally, on the intemational level, though the contributing parties will dissolve as an institutional group, resources may be sought from individual donors prepared to adopt bilateral strategies of assistance to CDTI in specific countries, or focus their assistance on particular targets. 4.3 The national level 4.3.1 The national level rests essentially on the Ministry of Health (MoH) and health services at different levels throughout the country down to First Line Health Facility (FIHF). There are several important mechanisms at national level: National Onchocerciasis Task Force (NOTF), and its operational branch, the National Onchocerciasis Control Programme (NOCP); the NGDOs; country offices of international rr (A second item that might be taken up by the special meeting is future of NOTF) JAFll.l0 Page 37 organizations; and an others involved in making CDTI effective, such as churches and denominational missions, few National NGDOs, as well as the communities themselves. NOTF/NOCP 4.3.2 While NOTFs function under government chairmanship, as a high level coordinating body and a suitable meeting point and forum for partners and stakeholders they are not a standing body, and work through periodic meetings. In order to give concrete expression to their deliberations and conclusions, they must fully rely on their operational arms, the NOCPS. fssue of concern 4.3.3 The mid-term external evaluation (2000) had already called for urgent rethinking on the future of NOTFs. "It is clearly essential that when APOC ends there should be a mechanism which continues to promote partnership in planning, monitoring and research around onchocerciasis control. The nature of such an arrangement is however far from clear. There is therefore an urgent need for NOTFs to plan for the future of partnership. Since countries and NGDOs differ substantially, it is logical to expect different models to be developed in different situations. Bach NOTF should be asked to develop a strategic five-year plan. This plan should indicate whether they see their role as phasing out and gradual integration with MoH, or whether they are going to continue to exist as a partnership, and to support onchocerciasis control activitiesr2". There is no evidence of action having been taken to implement this recommendation to which the evaluation team fully subscribes. NGDOs 4.3.4 Within NOTF, representatives of international NGDOs are active partners, with a substantive role. In general terms, they contribute to and have a catalytic effect in various Programme aspects. Concretely, according to the countries and the agencies, NGDOs offer various forms of support to NOCPs and to projects: management, financial assistance and equipment, logistics including facilitation of procurement, storage and distribution of ivermectin, and, as an important contribution, capacitybuilding.In the latter, they deploy a wide range of activities especially in the form of training in a wide range of fields: health education, sensitization, advocacy, community mobilization (HSAM); data management and management information systems; monitoring and supervision. NGDOs 4.3.5 The NGDOs involved, in a joint endeavour, meet twice ayear in an NGDO Coordination Group for Onchocerciasis Control ''. Their presence in the field is not linked to APOC duration. ln partnership t2 Report: External mid-troll evaluation. Spt-2000, APOC 13 Tlr" members of the Group are: Christoffel-Blindenmission. The Carter Centre, Helen Keller International, Interchurch Medical Assistanc.e, Inc. (IMA), Lion's Club International Foundation (LCIF) Mectizan @ Donation programme, Organisation pour la Pr6vention de la C6cit6, Sight Saveurs International, US Fund for UNICEF. Associate member: Merck & Co., Inc. Observers from the health field attend the meetings, and the Group is open to new memberships. JAFl1.l0 Page 38 with governments, long before APOC started its own operations, they were active in health programmes. In a number of countries, they were involved in onchocerciasis, including the ivermectin delivery aspects. For the future, at its 25th session (Ouagadougou, 10-12 March 2005), the Group reaffirmed, in its "Conclusions and Recommendations", the need to maintain "its role in supporting onchocerciasis control up till and after 2010". Some have already reasserted their long-term commitment, while the Group can see a role for increased training and surveillance. ln the Group's deliberations, the need to avoid that onchocerciasis becomes a "disease of the past" was emphasized at one point, as well as the need to do advocacy at all levels as a $oup. Operationally, the evaluation team believes in the importance of NGDOs stepping up their presence even further, to act as implementing partners including for new projects to be launched. Moreover, they should actively participate in future planning exercises for after APOC. Financially, each agency bas its own resources and funding capacity, in a decentralized pattern. One agency, though strongly committed for the long-term, expressed clearly that it had no intention of filling any post-APOC financial gap. The evaluation team felt that this was a general trend. .Issues of concenr 4.3.6 Some degree of dissatisfaction is voiced at the functioning of the partnership. ln one country with a large number of projects, NGDOs complained about instances of improper coordination with them by NOCPs when sending correspondences out, or of NOCP's failure to provide NGDOs concrete information on government counterpart fund contribution. In another instance, NOCP complained about NGDOs sending funding requests to APOC without government clearance. 4.3.7 NGDO contributions, in cash and kind, at any level in the countries, are not always presented with accuracy, leaving doubts as to what exactly is Some agencies expressed concern about their long-term funding and the incentive donors may have in the long-run Though there is a general will to continue activities beyond APOC, for some agencies, future commitment beyond 2010 is not firm The NGDOs involved are intemational. National NGDOs are the exception, and not part of -or represented in- the NGDO Group. It seems unclear to the evaluation team where exactly responsibility lies for identiffing local NGDOs with right potential, after training and capacity building. International organizations at country level 4.3.8 Among country offices of international organisations, WHO has a specific role to play. At present, WHO offices provide various forms of assistance, e.g.: support to APOC and APOC-connected missions; clearing entrance of Mectizan into the country in some countries, and forwarding it inland; purchasing equipment; channelling Programme funds to NOTFs. Also, administrative and finance assistants are out posted by APOC in some WHO country Offices. ln the framework of post-APOC arrangements, the evaluation team considers that the need for increased support from WHO will arise, and will have to be defined as functions are devolved to countries. In this approach, it must be bome in mind that WHO is well established and deployed in many countries, with no time limitation, and close to health authorities not only at MoH but at all levels. Also, many WHO offices are involved in work towards integration of flteld activities, and in one country visited, they had mobilised CDDs for vaccination campaigns. ln future, the WHO network in the Participating Countries, with support from AFRO and HQ, could be extremely useful for organizing, or incorporating into their agendas, inter-country exchanges on experiences of mutual concem in the onchocerciasis field. One other potential partner in CDTI is IINICEF. ln one of the countries visited LINICEF carries out clearance of Mectizan. Again in some of the countries visited, I-INICEF JAFll.l0 Page 39 expressed interest in participating more in CDTI. In one country UNICEF is gradually "revitalising" health areas, which should lead to beneficial interaction with CDTI. Other partners at national level 4.3.9 Churches and denominational missions offer various forms of support. In one country, a mission, working in the framework of community development under an arrangement with the government, is the implementing agency for one project. ln other instances, such agencies afford logistical help in many ways, and such immediate assistance as safe fund keeping, or shelter to staff on mission. Recommendution 22 (a) Since effective partnership will remain essential for sustainability, it is recommended that, considering the phasing out and Final withdrawal of one major group of international partners (Donors, World BanHTrust Fund) as well as of APOC, the role, contribution, andwillingness of other partners -existing or still to join the partnership -be defined clearly and in good lime, wtthin the overall objectives of the post-APOC activities in mind towards a sustainable CDII. Particularly, with ifs phasing out in view, and with the assistance of LYHO at appropriate levels, APOC should formulate plans for increased involvement of ll/HO country ffices in CDll Programme. Similar plans should be formulated with any other organisation, which could mate a substantive contribution such as UNICEF. l'yith APOC assistance and in close liaison with governments, plans should be preparedfor the continuation, and strengthening where required; of NOTFs or similar structures, to ensure their uninteruupted activities as long as necessary. NOTFs, or any successor stakeholder group, should take the lead in the discussions on future partnerships and; with APOC support, include the international level such as WHO, UNICEF and other such potential partners, with a view to enlisting as much supportfor CDll as possible. International NGDOs, who have played so far an important part in the Programme development, should actively join future joint planning exercises to be caruied out on the longer-term future of CDTI. They should envisage stepping their presence eyen further as required by current and emerging projects. They should; jointly with NOTFs, identify local NGDOs with the potential to act effectively on their own after training and capacity buildtng. Intra partner communication should be strengthened through the establishment of country onchocerciasis-website funded by APOC to enhance information sharing among all stakeholders. APOC management should prepare an analytical paper on issues related to NOTFs responsibility for covering 25% of the ivermectin distribution project costs, encotnpassing relevant information including that related to monitoring of sustainability plans, for review and decisions by TCC and JAF. Meanwhtle efforts should be stepped up by APOC in keeping track of contributions by governments and NGDOs while prompting compliance with the obligations. APOC should develop and implement, together with NOTFs, a mechanism whereby planned and actual contributions by local partners can be better monitored. This mechanism should produce triggers for coruecttve action on the part of NOTFs and, if needed, APOC, such as targeted advocacy nteasures from within or outside the country. (b) (c) (d) (e) JAFl1.l0 Page 40 a lfiith the help of APOC NOTFs should undertake detailed analyses of partners' contributions, and take strides to ensure that adequatefunding is available, includingfrom governments, for the orderly implementation of the CDTI Programme in all its aspects, particularly with a view to the phasing out of APOC Programm 5. Conclusions on findings and way forward after APOC This section pools and analyses the findings of the extemal evaluation team. Two critical findings are (a) progress towards sustainability of projects is uneven within and between countries and (b) a number of projects will still be under APOC implementation by 2010, decreasing gradually until 2015, requiring some fonn of continuation of the Prograrnme beyond 2010. This section also suggests a course of action forthe period after 2010 and to outlines preparatory measures that need to be taken for "when APOC is no more". 5.1 Will APOC achieve its objectives? 5.1.1 There is no simple composite measurement that could be used to find an answer to the above question. A qualitative assessment on the findings in sections 3 and 4, focusing on the sustainability of projects and delays in their implementation was carried out. For sustainability, adequacy ofi Policy; Planning and leadership; Support of the health system; Integration of support; Firiancing/funding; Human resources; Monitoring and evaluation, coverage and related aspects of APOC and CDTI systems were kept in mind. The picture that emerges from the assessment as well as deliberations of the evaluation team is mixed. Annexes 4 summarizes for easy reference the salient positive findings and emerging threats. Balance sheet 5.1.2 The findings show clearly the soundness of the CDTI system. Most of threats identified are not new. Considerable ongoing and planned work is aimed at finding solutions these issues. A big worry is unexpected events like SAEs and delays/stoppage of funding. But here the findings sliow ability of countries with appropriate support to overcome emerging threats. Pooling the findings show that APOC is moving towards its objectives. However there is a serious problem of delays particularly in projects in conflict situations. The conflict threat is detrimental to the 2000 objective. At the time of the evaluation a number ofprojects have not even stated. 5.1.3 Winding up in 2010 would be premature). The proposal for extension calls for reflection on the calendar for phasing out and measures to ensure that APOC has adequate capacity to support key activities related to enhance CDTI sustainability after APOC. 5.2 The what after APOC? 5.2.1 Considerable effort by APOC has gone into finding ways of improving the performance and sustainability of CDTI projects on the lines outlined in Sections 3 and 4. The hallmark of APOC for enhancing sustainability of projects, "when APOC is no more" has been the development of project sustainability plans under government leadership and increased funding (by the government). It can be hoped but it cannot be assumed or taken for grant that such sustainability plans will continue to be so in a rapidly changing national and global context. The EET feels strongly that CSA and JAF need to address this issue of enhancing sustainability of projects, squarely as a matter of urgency and put in place innovative strategies for the post APOC period. JAFI1.10 Page 41 5.2.2 Ensuring the success and sustainability of CDTI will be a local and national responsibility Governments should be in the lead for mobilizing resources from national and international sources, based on sound and solid budgets. Appropriate partnership and solidarity support through collective pressure and advocacy will ensure that onchocerciasis remains "on the screen" of public awareness and gets adequate funding from the government and other sources. The need for such partnership is now recognized by other pro$ams including the "big three" (HlV, malaria, Tb) and others. There is always the danger that present and emerging health priorities might relegate onchocerciasis to a position of little importance in terms of national and international funding, thus endangering decades of successful efforts. The Oncho partnership will have an oversight function, which could be the flag-bearer for what APOC (and CDTI) and OCP(now SIZ) have stood for. Such a function will be needed to coordinate efforts in areas such as research, project evaluation, impact assessment, advocacy, continued exchange of information between countries, including in areas such as the effect of CDTI on interruption of the parasite, SAEs, Ivermectin resistance, and new experiences in organizing CDTI and its impact, incentives for CDDs, 5.2.3 An added financial complementary mechanism, at the international level, may be devised to support the functions. ensure that OCP's and APOC's core activities research, dissemination of information and SAEs achievements and help ensure that the core activities outlined in this report are carried out. Though it is premature to propose any precise formula, the team is of the opinion that any set-up should be placed under the auspices WHO, which might consider the creation of an extra-budgetary onchocerciasis fund. Modalities and procedures would be defined with the WHO in due course. A basic role would rest on the country level, where NOCP, with the help of NOTF or similar mechanism would prepare plans of activities to be supported by the fund. Some of the agencies that have spearheaded APOC may want to play role in NOTF at the country level. The WHO Representatives in the countries concerned would approve proposals. As at present, NOCPs would continue to receive funds from govemments (regular budget, SWAP, PRSP) and other sources." 5.2.4 APOC should develop a plan of action for implementing whatever recommendations are eventually approved by CSA 5.2.5 The External Evaluation Team would like to conclude the report with its viston/broad strategies of "after APOC" as follows: 5.2.6 CDTI is fully integrated in health systems; High therapeutic coverage (65+)\is maintained; Assertiveness and demand of communities for ivermectin treatment provides/maintains political pressure for CDTI; CDTI is seen as a component of poverty elimination; Adequate budget provided by government; Solidarity and global interdependence ensures that all eligible populations are getting treatment; Active support to find and prompt use of new drugs and technologies, particularly macrofilaricides and alternative drugs to Mectizan (Move towards elimination?). Operational research enhanced; Strong monitoring and evaluation, critical analysis of data and surveillance for equity: More synthesis and dissemination of experiences, enthusiasm generated and maintained at all levels.a JAFl1.l0 Page 42 Annex 1 Terms of reference for the external evaluation of the African Programme for Onchocerciasis Control (APOC) for the period 2001-2004 1. Introduction and Background (i) The primary purpose of the African Programme for Onchocerciasis Control (APOC) is to establish sustainable, community-directed ivermectin (Mectizan) delivery approaches covering approxirnately 90 million people in 19 countriesra which fall outside the scope of the earlier Onchocerciasis Control Programme (OCP) in West Africa. These systems are being established to become sustainable " by 2010. It is estimated that a minimum of 30 million people living within the APOC countries are currently heavily infected with onchocerciasis. (ii) The formal objectives of APOC approved by the Joint Action Forum in 2001 in Washington reads as follows: "To establish, within a period of 12 to 15 years, effective and self-sustainable, community-directed ivermectin treatment throughout the endemic areas within the geographic scope of the Programme, and, if possible, in selected and isolated foci to eradicate the vector by using environmentally safe methods. The attainment of this objective is expected to contribute towards the eliminstion of onchocerciasis as a disease of public health and socio-economic importance throughout Africa and so contribute to improving the welfare of its people. It is worth stressing that the duration of APOC operations is limited in time and that the Participating Countries will assume, without major external support and well beyond the end of APOC, full responsibility for the continuation of community-directed treatment with ivermectin, set in course by the Programme." (iiD The principal tool for controlling and eventually eliminating onchocerciasis as a public- health problem is Mectizan, which is being grven free-of-charge by the producer, Merck & Co., lnc., for "as long as needed." One dose of this drug given annually reduces the load of microscopic, larval worrns in the human body by up to 950lo, without serious side effects. It thereby relieves intense itching and prevents occurence of blindness. It is estirnated that at least one billion free Mectizan tablets will be distributed over the life of APOC, having a value exceeding one billion US dollars. Angola, Burundi, Cameroon, Central African Republic, Chad, Democratic Republic of the Congo, Republic of Congo, Equatorial Guinea, Ethiopia, Gabon, Kenya, Liberia, Malawi, Mozambique, Nigeria, Rwanda, Sudan, Tanzania, Uganda Sustainability has been defined by an APOC special task force in2004 as follows: "CDTI activities in an area are sustainable when they continue to function effectively for the foreseeable future, with high treatment coverage, integrated into available health care services, with strong community ownership, using resources mobilized by the community and the government." ? t4 JAI11.l0 Page 43 (iv) The total cost of APOC over the l5-year period is estimated to be US$135 million, to be contributed by the donor community and an additional of 25o/o of this amount are borne by the Non-Governmental Development Organization (NGDO) Coalition and the African countries.The average cost per person treated per annum after establishment of sustainable community-directed treatment with ivermectin (CDTI) is approximately US$0.25. APOC will protect the donor community's substantial investment of US$600 million in OCP. OCP has succeeded over the past 25 years in virtually eliminating onchocerciasis from a ten- country subregion in West Africa. By controlling onchocerciasis in Nigeria, APOC will protect the entire eastem flank of the OCP subregion from re-invasion by infective blackflies, which could otherwise re-establish the disease in the OCP countries. Twenty donors and all 19 African countries signed a multilateral agreement, bringing the program legally into force as of April 1996. Twenty donors have committed US$65 million in financing the Phase II (2002-2007) and the Phasing-Out Period (2007-2010) program, and 1 I 7 proj ects in 1 6 countries had been approved for financing by the end of 2004 to alleviate suffering and prevent blindness among a total population of nearly 50 million people per annum. Obiectives the Review The objectives ofthe external review are: (i) To assess progress towards meeting the Program's objectives, including an evaluation of operations, and to make appropriate recommendations on the Program's strategies in order to fulfill its objectives by the year 2010. (ii) To make recommendations on how best to sustain CDTI post-2010 in order to eliminate onchocerciasis as a public-health problem throughout the APOC countries. (iii) To review the program's progress in light of the impact data. 3. Terms of Reference 3.1. Programlmplementation (1) To review and evaluate the performance of the Program to date, with particular emphasis on the Cornmunity-directed Treatment (ComDT) strategy and the effect on coverage and sustainability. (ii) To review treatment and evaluate also the performance and impact of focal vector elimination projects. (iii) To make recommendations on actions to be taken in areas where periplieral health systems are weak or non-existent. (iv) To examine the strategic prospects for achieving the Program's objectives by the year 2010. To assess the adequacy of monitoring procedures and to suggest improvernents, which might be made. 2 a a (v) JAFl1.l0 Page 44 (vi) (vii) (viii) To assess the adequacy of the impact assessment of APOC operations and make recommendations for future impact evaluations. To evaluate operational research and application of its findings To assess the role of the Program in conflict/post-conflict oncho-endemic areas and among displaced-person populations. 3.2 Coverage and Sustainability of the Program (i) To review treatment coverage achieved by the Program since its inception, as well as methods to expand coverage further. (ii) To review progress made towards enhancing the long-term sustainability of ComDT including the role of partnerships and community involvement and make suggesti ons for improvement. 3.3. Integration of ComDT into the Health Systems (i) To evaluate progress towards integrating ComDT into existing health systers in the Participating Countries and, within this context, the utilization of the ComDT system for the integrated delivery ofother health interventions. (ii) To examine APOC's contribution to health-systems development in Participating Countries, particularly the achievements of the Program in local capacity building. (iii) To review the impact of the Program on the quality of local liealth services, including Primary Health Care, especially in the poorest communities. 3.4, Binancial integration of APOC (i) To assess APOC financial management systems and propose ways to progressively integrate them into national health accounts and general national public financial management systems. (ii) To review present APOC financing systems and propose alternative ways of financing APOC under programmatic and general budget support, i.e. SWAP and PRSC respectively. (iiD Carry out a progmm expenditure tracking survey in a proper sample of APOC projects. (iv) To provide a preliminary assessment of the participating governments' willingness to continue financially supporting APOC program operations after 2010 and to propose steps which could facilitate the progam integration. t JAFl1.10 Page 45 3.5. Program Management (i) To assess the adequacy and effectiveness of management and staffing at the Program Offrce in Ouagadougou and the role of the Technical Consultative Committee (TCC) and the Committee of Sponsoring Agencies (CSA) in light of consolidating and ensuring sustainability of Program operations. (ii) To examine the financial management and controls of APOC vis-d-vis projects in the field. (iii) To review program management at the different levels of the national health systems and make recommendations for better integration and effectiveness. 3.6. Partnership (i) To assess the technical, financial, and operational contributions ofthe partners at the country level, namely the Ministries of Health, APOC, NGDOs and other partners. (ii) To assess the scope of the partnership and how it works at the country level. 3.7. OrganizationalAspects (i) The evaluation will be organized by the Committee of Sponsoring Agencies (CSA). (ii) The evaluation will be carried out by independent experts. The CSA, with input from the donors and the NGDOs, will finalize the composition of the Evaluation team, insuring that the required expertise is represented, and affange for secretarial support. (iii) The evaluation shall take place during 200412005. APOC staff will provide information to the Evaluation Team, and the Mectizan Donation Program (MDP), the NGDO Group, and members of the Technical Consultative Committee may be invited to act as resource persons where required. (iv) Evaluations of the roles of the various partners, g.g. NGDOs, Participating Governments, and local communities will be grouped together and separated from the evaluations of APOC pel se in the body of the report. (v) Field visits to the APOC area will be arranged as required. (vi) The evaluation report will be considered in final draft form by the CSA and presented to the December 2005 session of the JA-F for adoption. 3.8. Skills Requirement The six members of the Evaluation Team would have the following skills: Epidemiologist, Social Scientist, Entornologist, Economist, Public Health Specialist, and an expert with experience in Administration and Finance. a t JAF1l.t0 Page 46 Annex 2 Analysis of treatment results all APOC since inception Updated on October 11.2005 Communities Population Coverage (%) Country Year Total Treated Total Treated Geographical Therapeutic Angola 2005 122 46 90 527 6 062 37,7 6,7 Qaqreroq!"1 Cameroon 9gnqrqpn Ceqqrgq1 cqg!qro,"9n Cameroon 1999 2000 2001 2002 2003 2004 2 490 3 387 5 684 5 852 5_894 7 532 2 913 3 l0z 4 qqg I911 5 q49 7 489 1 379 405 1 599 Z6_0 1 868 278 _? qq1 911.5 3 161 522 4 334 712 529 967 727 921 1 018 117 1 5_50 385 2 482 672 3 180 769 81!8 91,7 7.1$ 86,7 99,E 99,4 38,4 45,5 54,5 5q,3 79,5 73,4 c4B CAR CAR c_Alr CAR 1_9e9_ 2_90q, ..?,q01 2002 2003 I0_-14 5 q_11 F911 5 014 5 014 t ?9+ q 0_11 1.59t.: t 9p2 4 653 1 375"637. 1lE.?.6..4 1 454 891 1 15J 991 1 176 572 _ e94 1_5,q 1 916 ?9? _ 778 _9_89 *"_492287 890 747 91,1 199,0 "91,6 93,4 92,8 67,9 7q,0 53,5 !2,7 75,7 9ong-o cg.ogo 9orlg.q- Congo 2001 2002 2093 2004 748 118 718 770 425 4qq 719 765 578 812 58-318_0 569 652 580 582 "228 220 1 95 950 ._359 ?8.1 383 302 56,9 Q2S 96,J 99,4 _39,4 33,6 62,0 66,0 9_Bq P.B_q . Dtsg DRC ?og1 2002 2003 2004 10_ 020 .12 394 17 507 15 760 2 6Qq L 919 11 78? 9 469 q 5qp 19_3 6 516 142 e es8 B!3 8 613 655 _ _141.9T0 I 816 Be8 4403 838 3 898 234 26,3 39,9 67,3 60,1 13,4 27,9 44-,2 45,3 Eq._Gulngg Eq. pyi49a. !"q. Qgiqe_a_ -fq. G1'pe_a Eq. Guinea Eq. Guinea 1999 2000 2q01 2002 2003 2004 129 129 1?e 129 129 129 15 68 67 95 128 63 1ll 64 Q9.1 66 122 67 679 70 903 1 236 7 882 10 797 10 874 48 551 1J,9 5?,7 51'9 73,6 99,2 2,0 12,2 16,3 16,1 68,5 Ethiopia Ethiopia EJhjopiq Ethiopia 2001 2002 2003 2004 3 277 3 277 5 244 20 368 le4 2 088 I ?50 16 364 902 687 928 691 1 563 635 4 965 717 233 309 516 217 1 025-859 2 960 696 15,1 6?,7 B1,q 80,3 25,8 55,6 65,6 59,6 Liberia Liberia Liberia 2000 2001 2002 2 596 2 596 2 596 2p1 e.9B 628 2 083 435 ? 149 021 2214 610 497 662 900 000 900 000 10,8 76,6 62,7 23,9 41,9 40,6 Melewj Melqwi Melqwi Melewi Malawi Malawi 1 999 2000 2001 2002 2003 2004 690 ? 174 ? 174 2"174 2 170 2 186 . . _.34_2 . , -"- 9_qq _ _ 5_96 " ..8q1 7qe 2 186 . q39 957 1 383 824 1 412 I 1+9 1 477 1 604 qt1 392 153 453 228 014 2s9 1?o 310 056 " 473 506 47.7 103 1 095 530 .5J,q -21,4 ?7,4 _ 9_9,6 39,0 100,0 42,8 21,2 21,9 32,8 32,3 68,3 ' o JAFl1.10 Page 47 Communities Population Coverage (%) Country Nigeria Nigeria frligerja Nigeria Nigeria Nigeria Nigeria Year 1998 _lgee 2000 2001 ?902 2003 2004 Total 2 531 20 301 32 530 32 530 35 783 35 299 34 939 Treated .1 39q 2 8_90 .??_296 29 1L9 _3"1 Lq1 -3?-.344 31 482 Total 605 693 109 1?9 77.9 019 618 844 9q9 9i1.1 170_612 683 209 1 21 ?1 .24 _25 ?5 26 Treated 1 175 27',1 12 434 247 15 154 631 16 s39 589 18,389 925 19 511 290 19 511 819 Geographical . Q2,7 14,2 71'6 q6,6 ggrg 91,q 90,1 Therapeutic 73,2 "59,9 63,2 67,2 _ 70,9 77,5 73,1 9q_Qen Sudan 9rCen S_qdan 9qQen Sudan 1999 2000 2001 2002 2003 2004 311 549 2 600 2 600 326 618 ?1e 478 890 94e 238 357 1 1 .1 1 q19 q16 384 188 Boq 7?0 q77 +1zi 586 625 557 813 431 329 423 824 789 935 495 97-0 248 180 253 089 7o'1 Q7, -1 34,2 1?,4 29,0 57,8 32,8 30,6 43,7 ?6,_4 42,3 45,4 ia ia ia Tqn4?l Te!4a!'r Tanzanta Ienzere _ _ Tanzania Tanzan 1999 2000 2001 2902 2993 2004 3?2 442 592 592 633 884 -31J 433 qqq 58q 633 874 _ 6qq z9.s q59 4l_1 1-.2591?4 1 311 047 .1 _s?9179 1 785 133 504 746 540 757 8q_6 136 866 527 058.352 271 453 -99,4gp,g s9,3 99,5 1.09,q 98,9 7e,3 63,6 6!,1 66,1 6e,q 71,2 Ugcn"d.e" !,ggf'_d_q Ugan4a Vgcn_Ce* !.,gelda Uganda 1999 2000 2001 2902 2003 2004 3.122 3 613 3 96"6 3 e_99 4 186 4 785 9 t2? _3 6-13 3 eq9 .3 q3q 4 lgq 4 785 1 Zq9 958 1l!_9 211 1 960 935 _2 12_0.Q9e 2 129 314 2 416 898 1 226 494 1 402 018 1 543 926 1 6s2.987 1 756 418 1 761 700 1q9,9 I90,0 109p 1q9,0 100,0 100,0 _ 69-,7 78p 78,7 77,9 82,6 72,9 a t (nl bl trl trl <t (D € f e4 H E!!;i! .iE -r E-B i i".;itE >.rEE= !Ei tEI r T::ErEg * ETIS!2r.9 :7i - itteg*i€ E5' F _EFF T 5ir:3 {8. i! .I irc Li IEI E -fF = 3EqF ^5 giot AliK:,tx-';:8, .tEriEE.E:zliE .orE! . E EEE+E ;E F! I T b I I! It I i RI E $ ?o f e .!- H $E ? E t 5 4E+iii! !a a -/N 1E tJ o g N] ,t F o 6F(, U a E ou o d. aI tA raeoFI ft c, O cl 9/' Io oLc !(u olioa U o Cr D E5{ I E sn E^Ei€ =c.Bt;'rg ^oE mEE erE ail E?E-G.. : E:: E .!49. o 2<u (aoo*$ Eg,dd iiO. (D I L d E U E * E u 3 0ia ri I$l i TI II I I I I !:tr trl5rl Eri & atI o !9 +:-i alE u: ai:E.E: I i i <oou=6F 9 E 7 = iq9: ["rt EI T II iI I I :. I li U! l: I I I fi I taI F U- I & I I, t,... i r T I l I I I i IlFl Zl i !${tFl I E li li q) , Ir 5 I =t 4l Efl rl lr. *l Elltf i i !i 3l:1 fl I g" .I I! T , TI t t Aa _4 q I TI - lt l{ i s J 6 A s T i _a_i n '\li ; I lt ,l :llii{ :iIilI c I s U x.> Lr-! I I! I .E 9 !ale II ir c, t: s ! I Bt a EE ;co E ae o ooo ojq 5E<g a C o o 6o J .g(, Jt €€ =-o9'o _EE 3FIIJ 5o o fotE >9a! E<fcoLoo)9s E!n63o e ooo e :2oo E.! ofr oa)9EFE.9oE keE i, o- E3 I t3 o d o I oq F .d= >f EA3!rlll = uto o$d- s5 IOoE eo o ooo a6 o ooU' a,oE-oi,tr6iEa t.E {E oo 'Eo oo oo CI! b E oo g 6 tr =oo o - .9o6 Eo c: ooo .2o il- E" E , >o EERoocOG<E do.E; !'gr<oo e co o oo t -cI d. rlU6o =ll E3OIJ 26 u, lC o Ioo t uJ C' trr o IJJo-Z 'r\ <E z_@E- € F u, oo =o t u,ILro6 lLl L z z L rI ,oo9<E .o o.:ElroU' .0 o c 3go:<: .o .9 oo o c3r ES <= .to go! o6 oIoI tro E ooo Io6EasgE'E e a6 o ooo ooE ooooE ((g I aI EE a o o o oo dE .9c oo o s o oE p. ona fiF d oF 5oz fiF xi c o r lrJ co aF x o o vg(, B o E o oo G a9otrF u.r<E 2o E6 8U(J.... o :14 t; .96 ;Tootroooo = Do o ooo zo* kf;a Er=P6- = otb!6j €!o !g o (( U d, o o .z o ! Io e o o o o v, o o F CD .s 6 =I o o iEc>E6 r"iyo Efr !c o = .2o o Is ,o EE o 'tr o o o o oo to 6 o (, oo a o o ooo xo oo oe =grEu zI ,EDUJ^F9a(l)[< =oo o .2 o Erz d E$3 oOEc6q'6 OE .:odo o =t.oE u1JJqzI9a <58Eg=)u,q, i; fr6 I o 6 Fc;o <e LOoI 6.9tJ, ooG dE .D EJ2 d r{36ooog,ErEt' $l xlOO\ 6)l -v trl EFiI < Iri (, F\v Rr D EI GIEI c)l EI;tNI 'El dl oaLI 3l OI Arl <t I I I I I I Ia !