Report of the meeting of french-speaking countries on the sustainability of CDTI projects

t I I AFRICAN PROGRAMME FOR OCHOCERCIASIS CONTROL (APOC) PROGRAMME AFRICAIN DE LUTTE CONTRE L'OCHOCHERCOSE REPORT OF THE MEETING OF FRENCH-SPEAKING COUNTRIES ON THE SUSTAINABILITY OF CDTI PROJECTS Rapporteurs: . Dr Joseph Enyegue Oge . Dr Jupa Mukengeshai . Dor Nouhou Konkoure Diallo . Cyrille Evini . Dr Francois Banyombo . Mr Hyacinthe Balla Balla I Content i. Abbreviations and acronyms ii. Summary 1. Introduction 2. Expected results 3. Methodology 4. Procedure 4.1 Speech from the Director of APOC 4.2Examination and amendment of the guide for the evaluation of sustainability 4.3 Examination and amendment of the instruments for the evaluation of sustainability 4.4 Presentations of country health systems 4.5 Evaluation plan for project sustainabiliry in each country 4.6 Closure 5. Modilications of the English version of the instruments for evaluations of sustainability 6.Conclusion 7. Recommendations 8. Other important issues 9. Annexes L Agenda II. List of participants IIL History of the development of the guide and instruments for conducting evaluations of sustainability VI. New organization chart of APOC V. Organization chart of the SIZ VL Presentations of country health systems VII. Instruments for the evaluation of sustainability VIII. Report of the moderator i. APOC DC JAF FLHF NOTF IEC OCP NGDO NOCP CAR DRC STZ PHC CDTI Abbreviations and acronyms African Programme for Ochocerciasis Control Community Dishibutor Joint Action Forum First Line Health Facility National Ochocerciasis Task Force Information, Education, Communication Ochocerciasis Control Programme Non-Govemmental Development Organization National Onchocerciasis Control Programme Central African Republic Democratic Republic of Congo Special Intervention Zones Primary Health Care Communify-Directed Treatment with Ivermecti n Summary In order to finalize and allow a common understanding of the guide and instruments for the evaluation of sustainability, a meeting was held in Ouagadougou from February 24-28 with NOTF representatives and NOCP coordinators of ex-OCP from French-speaking countries. Group discussions for the amendment of the above mentioned documents have led to a clear and common understanding of the concept of SUSTAINABILITY of CDTI projects, which includes the following aspects: effectiveness, efficiency, simplicity, integration, attitude of the actors, resources, community ownership. A guide with four instruments for conducting evaluations of sustainability for CDTI projects were studied: Instrument l: National/Central/State/Province/Regional/Department/ District Level Instrument 2: Health District/ Periphery Level Instrument 3 : FLHF (sub-district/health arealhealth center) Instrument 4: Community Level (villages, communities) All those documents for conducting evaluations were corrected and amended. The major amendments were also reflected in the English version. A schedule for the evaluation of CDTI projects in the six APOC countries (Cameroon, Congo, Equatorial Guinea, Central African Republic, Democratic Republic of Congo, Chad) represented at this seminar was adopted. It also appeared that all the participants had a clear and common understanding of the group of indicators and the Guide for conducting evaluations of sustainability as well as their use at the different levels of the health pyramid. In addition, the participants requested that APOC intensified advocacy activities so that governments mobilize the resources necessary for the implementation of post-APOC sustainability plans for CDTI projects. 1. Introduction From February 24-28, a meeting was held in Ouagadougou (headquarters of APOC) to discuss the sustainability of CDTI projects of the APOC programme in French-speaking countries. The objective of the meeting was to allow the participants to familiarize themselves with the guide and instruments for conducting evaluations of sustainability. The opening ceremony was presided over by the Director of APOC, Dr. Azodoga Seketeli, and Professor Detlef Prozesky served as moderator. The meeting was attended by the representatives of NOTF of the six countries of the APOC zone (Cameroon, Congo, Equatorial Guinea, Central African Republic, Democratic Republic of Congo, Chad), the National Coordinators of five ex-OCP countries (Benin, Guinea, Mali, Senegal, Togo), the staff of APOC headquarters and the SIZ, and a representative of the World Bank. The detailed list of participants is attached in annex. It should be noted that after the evaluation of sustainability in the ten CDTI projects in English-speaking countries supported by APOC, it became necessary for French-speaking countries also to get together and exchange their views. They need to have a common understanding of the description and importance of the evaluations of the sustainability of CDTI projects with the instruments that have been developed and already put to use. 2. Expected Results In his introductory message, Dr. A. Seketeli, Director of APOC, made the expectations of the Direction of APOC clear to the participants (see annex) as follows: - A clear and common understanding of the definition of SUSTAINABILITY as defined in the implementation of CDTI projects; - An unequivocal understanding of the group of indicators of sustainability as well as the instruments and the guide for conducting evaluations at the different levels of the health pyramid. Amendments have to be made if necessary. - Take into consideration the new version of the sensitive documents developed to date, namely the guide and instruments for conducting evaluations of sustainability for CDTVAPOC projects. - Keep in mind the planning for the evaluations of sustainability for the year 2003 in their respective countries. - After all the discussions, clarifications and a perfect understanding of the objective of the evaluations of sustainability, develop post-APOC plans for CDTI sustainability by identifying their funding sources, or, more specifically, by indicating who is likely to fund what (material, financial, human) for the implementation of the sustainability plan. Dr. U. Amazigo, Head of the Medicine Distribution Unit presented the objectives, which were similar to the above mentioned expectations. 3. Methodology The methodology consisted in the following: - Clarification of documents and topics for group work; - Group discussions of the guide or instruments depending on the case; - Presentations, discussions and adoption in plenary of the conclusions/recommendations; Group work was carried out in two ways: - Study and amendment of the documents submitted to the participants, - Case study for sampling activities, data collection and analysis and overall appreciation ofthe projects. After the modifications were adopted in plenary, the rapporteurs had to include them in the French version of the documents. The modifications that were considered fundamental were reflected in the English version as well. 4. Procedure ofthe sessions 4.1 Speech from the Director or APOC The speech addressed the following issues: - New organization chart of APOC After the closure of OCP, the structure of the management of APOC underwent some important modifications. The main innovation worth mentioning is the creation of the position of Coordinator of the Office of the Director of APOC. Dr Laurent Yameogo who was, until December 2003, the Head of the Vector Control Unit of OCP fills this position. Besides, several other staff members from ex-OCP were transferred to the management of APOC as the new organization chart required. It should be noted that some positions on this chart are still vacant (see in annex). - Special intervention zones (SIZ) The Director of APOC was mandated to implement the plan of action that was approved for special intervention zones during the last session of the JAF/CCP (?) which took place in Ouagadougou in December 2002. The objective was to make the SIZ healthier by intensifying Mectizan distribution in the regions under consideration in Benin, Ghana, Guinea, Siena Leone and Togo in addition to vector control in northern Togo and Benin. Dr Laurent Yameogo is the team leader of SIZ, concurrently with his duties as Coordinator of the Office of the Director. It should be noted that the management of APOC is not legally responsible for SIZ. Besides, the funds allocated to SIZ do not come from the budget of APOC. For more details on SIZ, refer to their organization chart in annex. Dr Azodoga Seketeli, Director of APOC, after showing appreciation for the presence of the representatives of APOC as well as ex-APOC zones, encouraged the participants from the two zones to take advantage of the sessions to share their experiences, increase their knowledge and improve the performance of their respective projects. - Historical overview ofthe development ofthe guide and instrurnents for conducting evaluations ofthe sustainability of CDTI projects Following the recommendations of the extemal evaluation of APOC programme that was conducted in May 2000, the management of APOC took an initiative to help the countries under consideration with strategies that would make the achievements of the programme sustainable once APOC support comes to an end. In effect, two experts, Dr E. Tarino and Pr D. Prozesky were appointed in May 2001 to conduct the elaboration process of the instruments and the guide for evaluations in order to measure the sustainability of CDTI projects. Those documents were used from May-June 2002 to evaluate nine CDTI projects in Tanzania, Malawi, Uganda and Nigeria. From the results of this evaluation and the lessons leamed, recommendations were formulated for the amendment of the guide and instruments. Those recommendations were actually made at a meeting of the National Coordinators in June 2002 at Abuja, Nigeria. Thus, the instruments and the guide were amended in September 2002 in Ouagadougou at the second meeting of the NOTF/APOC representatives. Given the fact that the amended documents were in use in English-speaking countries only, it was necessary to hold another meeting on sustainability before the beginning ofevaluations in the French-speaking zone. P os t-A P O C s us tainability p I ans The Director of APOC reminded the participants that the objective of the evaluations of sustainability was the elaboration and implementation of post-APOC sustainable CDTI projects. The present meeting constitutes an opportunify for the National Coordinators whose projects are in their fifth year at least, to exchange views with those of their peers whose projects have already been evaluated. This is also the right time to start working on post-APOC sustainability plans that will need to be polished after the evaluations have taken place. In addition, the Director of APOC wondered if it was not a good idea that all APOC countries meet after all the evaluations that are scheduled for 2003 to analyze the results together and draw conclusions. 4.2 Examination and amendment of the guide for conducting evaluations of sustainability This document provides background information and guidelines for the preparation and management of an evaluation mission. It also describes the analysis process ofthe data and explains how to organize feedback and planning workshops for sustainability. After the presentations of the different groups' work and plenary discussions, the guide was amended (see annex). The following specific issues were raised during discussions: - Self-sufficiency and autonomy. The two concepts have the same meaning during evaluations of sustainabiliry; - Community ownership as an aspect of sustainabiliry, and forgotten in the French version, has been added. - The sequence of the groups of indicators must be revised as follows: activities - resources - results. - Sampling must be done realistically at the district and sub-district levels and randomly in the choice of communities. However, it important to be flexible during the sampling exercise because the objective is to guarantee the representation ofcoverage and endemicity levels in the different geographic zones, including the remote ones of the project under evaluation; - It is necessary to elaborate a practical guide for data collection; - It is imperative during the analysis of the data to always consider the characteristics of the indicator; - During the analysis of the data and the elaboration of conclusions and recommendations, it is only the data collected that have to be strictly taken into consideration. No personal opinion must be taken as fact. - The evaluators must start with a verbal description of each characteristic before assigning a grade. - The data generated by the interviews are as important as the information obtained from studying the documents, which will enrich the conclusions of the team of evaluators. - Concerning the feedback/planning workshop, the team must feel free to decide on its duration depending on the realities in the field: availability ofthe evaluators and resources. - On this issue, the final decision depends on the 'scout' and the team leader. By way of example, NOTFs in the different countries have determined the duration of the evaluation missions as follows: * Cameroon: 2 weeks, but two and a half for the projects located in the Southwest, the North, and Adamaoua. * Congo: 2 weeks * Equatorial Guinea: I week * Central African Republic: 3 weeks * Democratic Republic of Congo: 3 weeks * Chad: 3 weeks - The feedback workshop must start with the Dishict level ( I day). All the authorities (evaluated or not) at this level must attend for retro-information. The second workshop (central/project management) will take two days, that is one for feedback and the other for planning. - During the feedback meetings arrangements must be made to have the highest ranked possible authority as chair of the opening ceremony. 4.3 Examination of the instruments for the evaluation of sustainability For data collection instruments have been specifically designed for each level - Instrument l: National/central/state/province/regional/departmental level - Instrument 2: Health district/peripheral - Instrument 3: FLHF (sub-district/health arealhealth center) - Instrument 4: Communiry level (villages, communities) Each of these instruments has a group of indicators and sources of information to allow for the appreciation ofthe activities, resources, or results by taking their sustainability into account. Corrections were made to the four instruments. Those corrections concerned their form more than their content. See the corrected instruments in annex. 4.4 Presentation of country health systems The presentations of the National Coordinators have allowed the participants to understand the health systems of their countries on the one hand, and the level of application of the instruments for conducting evaluations on the other. Table I shows the instrument to use and the people to interview per country during an evaluation for sustainability. The presentations are attached in annex ofthis report. Table 1: Instrument for the evaluation of the sustainability per level and country I3 €fr!Eb i 'X ? ETH f; ; E EE ,q.E ! i E" !EYO a '6 +g s,Y fiaeEiEE;u€aa*s 4{,gEXqEEZE?e"tt i'BdNemS EEg F : EE9 E - rr(/lA O o\ r -- & ( EEEE.eHEq .ATXEE E5 i a-^ 2 ! = r\Z*, ox& cooiD trt a o() .N ^6HO :YO .= bc o-2> o q) oUl 60bu q) aoo-coc.--coo o ii P E OI XP q E = FF 7E + #Ef: E;EI qE{O O o -'= a<. ti- E E E E.E :.8 e.E.E .!E.:EAe>=>eo.QoO€U bt be .u .z .q) .az !o d--(!(€ o I IJrO d CE E EB = E 'r A -r't 6 o Ea9E99ai;? - - -._ - o '= o'a;'a a o t o o6 P,6X668;8a .!-,;.!.x!!.axa-LL-L-vrr-Q-o- .v ootEoub \J= 6A oo bot, 2>- o o.Ytr o.co=oNL/]7) (! o o 00 c!, H.-O> 6t 19 cgts q) Q ! : =o ,o)cd=6 .. i: E.= 6 v (u;9 X1' g=€.9 EO oSEbHUo l- u =v L'='CJ; a v I H a'9 = Q >,.9U ir - A = ;r ; .= L ! dq!.-6.q=E!> o.= d 6 0.= EJ: E'51?fiz.a.a.E-Et -a<)aL C. X tr!?E 6 t 6.'j90tEg.E Eryi E =a8&} gFi E EH ;eaH:uEA:e:E A FA Fg rgZ$t$ a' .&,&,a o o- o- o: :E .U5 =od9!E E g-3 E= > T3?EtN o o o0(! U> 6t 6) 6l o 6l ri .a E Li:- oox !-Y!A o v 6- 5 !'! c>€ 8.g 9*9EAEE5 -: !i: E -z o A.Y :a6v x! EgAO QOZJ . ;.r.,i . a E d o !.-O> o u0 o U Lt=rP9aO EgEEoOo,9\-,/ o v ()E o O ora9=9ts'EE .= d.= E 6Aqfii62.z El -.ea6l EE-oil 5tE € zrE .E ElrEEi q=ElE i ElEEif*E3=lEkEh Ela $E t g EBEiE s:E 5 6FO CILo^ i!eoLCo; OE =;'H eocgtrt.qtu.Q o C) o((! U> o 6 U a'! I EE=*+ }E E dff,Ea o! 5 pir-3'iE8B E iu? T;8.: :E b ;1,] E :: E.E ;tsE3E;=sEE3 og e= .EU qE .2i r=! ..?T 55 'Et f, * E.9E EIEEIEfi :l 6 6 q.3.E it 660;=6il O O = tr .= el IE E ca ! o{t .{ EEgiE.=OEE ,? gE H E L =v9!Ootr! :l F=trtrE <.q.q Eq qvv;i o o=E;o +L-;:!G o E o Y o.={-Y{-*A ,t/ z. cB o (.) b0d U> E*Fe' a s \ e q) L e \ -! Table III: Availabitity of the evaluators No Name Period 1 Dr. M. TRAORE June-July, August 2 Dr. NTEP Mid August-l't week of September J DT. BEFIDI-MENGUE Unavailable 4 Dr. A. DARE Mid-May, June and July 5 M. C. EVINI July and September 6 Dr. NGIMA MAWOUNG April, May and June 7 Dr. J. OYE June and July 8 M. F. BANYOMBO June, July and September 9 Dr. MEBENGA Tamba Luc April, May and June l0 Dr F. MISSAMOU July and August l1 Dr. A. SIMA July and August t2 Dr. J. N'DOYO April, May, June and July t3 M. R. NALKE-ASSOULA April, May, June, July, August, September l4 Dr. E. NGUEMBI April, May, June, July, August, September 15 M. J-B GONDA April, May, June, July, August, September 16 M. H. BALLA BALLA April, June, July and August I7 DT. NAMANGUE July and August 18 Mrs. T. SABAYI-CUPPEN Late May, June, August, September 19 Mrs. NDAGANO From September l5 20 Dr. N. K. DIALLO Mid May, June and September 2t Dr. L. DIAWARA April 15 - May - June 8 22 Dr. E. GABA May through June 23 Dr. M. KUPA July, August and September 24 Dr. B. KEBELA- ILUNGA May, June and September 25 Dr. BISSEK ? 26 Dr. D. Prozesky August thru November (one two-week evaluation only 27 Dr. BODZONGO Mid June thru July 4.5 Plans for evaluations ofproject sustainability in the different countries The evaluation plan that was adopted at the session includes the list of projects to be evaluated per country, the schedule ofthe evaluations and the period when evaluators are available (tables II and III) 4.6 Closure The closure of the meeting was presided over by the Director of APOC, Dr. Azodoga Seketeli. In his speech on the occasion, he congratulated the participants and moderators for their patience and courage that they have shown during the five days of the session. He reminded the participants that the modifications of the content of the French version of the instruments must be reflected in the English version to bring the two documents on the same level. During this ceremony, the participants requested the advocacy activities of the management of APOC to sensitize their respective govemments on their role as agents in charge of mobilizing the necessary resources for the post-APOC plans of sustainability. To conclude, the Director of APOC expressed the wish that Burundi and Angola be included in the process of the evaluation of sustainability, which will allow them to benefit from the experience of other countries even though they are only in the starting phase. 5. Modifications of the English version of the documents Corrections were made to the four instruments. Those corrections concerned their form more than their content. See the corrected instruments in annex. As far as the guide is concerned, the moderator passed the latest English version on to the rapporteurs by two days before their departure. Given the weight of the details and the richness of the amendments in the latest version, the French version of the guide was now obsolete. Therefore, it was put aside in agreement with the management of APOC. The services of a translator were requested for the occasion, but it should be noted that an entirely revised and relevant French version ofthe guide was not available yet. 6. Conclusion After the session, it appears that the objectives were achieved. In effect, from the consensus on the definitions of the concepts and the modifications to the guide and instruments, it is clear that the participants have had a common understanding of the topics that were discussed. The consensus that was reached on issues pertaining to the content of the documents was reflected in the English version. With a clear understanding that the aim of evaluation of CDTI projects is the elaboration of post-APOC plans for sustainability, the participants have requested that APOC intensified advocacy activities to Country CDTI PROJECTS DATE OF CLOSURE OF .LETTER OF ACCORD' EVALUATION PERIOD Cameroon Province ofNorth Province ofAdamaoua Center 3 South West I Western Province S"rth w"rt II(7!"*) October 2003 November 2003 November 2003 August 2003 December 2003 March 2003 l7-30 April 2003 I 7-30 April 2003 5 -l8 May 2003 l6-30 June 2003 5-18 Mav 2003 April 2l-May 5 2003 Central African Republic CDTI CAR October 2003 August 3 l-Sept. 14, 2003 Chad CDTI Chad September 2003 2-16 June 2003 Dem. Reo. Conso CDTI Kasai (3d year) March 2003 l-14 Julv 2003 Congo CDTI Congo (3'd year) December 2003 I 5-30 September 2003 Equatorial Cuinea 4'n year November 2003 7-13 July 2003 sensitize their respective govemments on the mobilization of the necessary resources. In conclusion, a planning for the 2003 evaluations in the countries under consideration was adopted. 7. Recommendations 7.1 The guide and instruments for the evaluation of the sustainability of CDTI projects that were corrected during this session must be passed on to the NOTFs in French-speaking APOC and ex-APOC countries by the management of the programme by the end of the year 2003. 1.2 ln order to have the same guide and instruments in all APOC countries (anglophone and francophone) the modifications of the content of the documents in French must be taken into consideration in the English version. This must be done by the rapporteurs of the workshop and the management of APOC will have to send copies to the respective countries. 7.3 The participants recommend that after the evaluations for sustainability in 2003, the following people must attend the feedback workshop in a francophone country (to be determined): Representatives of the NOTFs (national coordinators and representatives on NGDOs) of all APOC countries as well the teams of evaluators. 7.4 ln order to have a standard method of data collection, the participants recommend the elaboration of a practical guide that the management of APOC will make available to NOTFs (anglophones and francophones) before the next evaluations for sustainability. 7.5 All the countries benefiting form APOC funding for the implementation of CDTI projects must start elaborating of post-APOC plans for sustainabilify right now, no matter the year of their implementation. 7.6 ln order to ease the mobilization of the resources necessary for the implementation plans for the sustainabiliry of CDTI projects, the management of APOC should intensify advocacy activities to sensitize government offi cials. 8. Other important issues arising Outside the activities of the session, workshops were organized to discuss several other issues. Some of the important ones included: - The necessity to organize a training session on the effectiveness of the management of the funds of CDTI projects and the regular delivery ofcompleted and accurate financial reports; - Brainstorming of NOCP Coordinators for the identification of the country that must host the next meeting of NOTF coordinators; - Planning of the review mission of IEC materials for the Central African Republic; The meeting of the coordinator of Benin with the management of APOC on the transnational border problem between Nigeria (APOC country) and Benin (OCP country), the infection and re-infestation problems in order to preserve the achievement of OCP; - The meeting of the Director of APOC with the representatives of the Democratic Republic of Congo and Congo-Brazaville on the collaboration in the region of the transnational border between the two countries. Ann, I. II. III. IV. VI. VII. VIII. exes Agenda List of participants Historical overview of the guide and instruments for evaluations of sustainability New APOC organization chart Organization chart of SIZ Presentations of country health systems Instruments for evaluation of sustainability Report of the moderator