APOC management review: final report

APOC Management Review Final Report Allison Beattie and Richard Johnson co-authors 4Iuly 2Ot4 ,, Æ_V AFRIcAN PRocRAMME FoR \Yrl oNcHocERclASls coNrRoL Table of conten Executive Summary Summary of Re commendations 1. Introduction and background to the study: t.t. Why is this study being done now? 1.2. Approach and Methodology .8 12 15 15 t6 2. Findings 2.1. Current stffing 2.1.1. Frameworkforemployment. ... . 2.1.2. Staffing levels and locations 2.1.3. Roles, duties and responsibilities. 2.1.4. Remuneration and O_ualifications 2.1.5. Nationality and Gender...... .......... 2.1.6. Consultants. . 2.1.7. Conclusion on current staffing.. . 2.2. Managementstructure .... ...23 2.2.1. OrganizationalStructure.... . ... ......23 2.2.2. Partnership Structure....... ....... .23 2.2.3.Resourceflowswithinthepartnership ............. . 27 2.2.4. Management of APOC Functions 29 2.2.5. Managementof projects.. . .......... ..32 2.2.6. Recent and foreseen changes to classic model 34 2.2.7. Communication, data disclosure, presentation of results.. .. 36 2.2.8. Conclusiononmanagement structure..... ... ... ... ..36 .... 19 . ...19 19 .... 79 . ...20 .......21 22 22 ......22 2.3. Implicationsfor management and costs of expanding the scope of the programme ... 2.3.1. Addressing the Cumulative Challenges 2.3.2. Possible Implications of Incremental Changes... .. 2. 3. 3. Key challenges for transformation.... . 2.3.4. Implications for management structure and costs ... ... 2.3.5. Conclusion on implications for management and costs 2.4. Benchmarks 2.4.1. Methodology for establishing benchmarks... 2.4.2. Benchmarks:indicators andassessments . 2.4.3. Conclusions on benchmarks 2.5. Strengths and Weaknesses; Opportunities and Threats 2.5.1. APOC Secretariat Strengths ... 2.5.2. AP OC Secretariat Weaknesses 3. Overall conclusions . . ... . ... The Challenge The context Towards new modalitie s ... Transparency and openne s s 4. Recommendations Annexes Annex 1: People interviewed and questions . .. .. .. ... Annex 2: Documents and papers consulted and reviewed Annex 3:Terms of Reference. ... ... 37 37 38 40 42 42 43 43 M 48 49 49 49 51 .51 .51 52 54 55 60 60 62 63 Tables and Figures TABTES Table 1: Table 2: Table 3: Table 4: Table 5: Table 6: assessment FIGURES Figure 1: Figure 2: Figure 3: flows Figure 4: APOC Timeline Summary of statutory meetings and functions APOC expenditure on Statutory Meetings in the last Biennial (2012-13) Start-up and Project Sequencing at Country Level lndicative Management Transformations required for pENDA Benchmarking management capacity in APOC as of 2014 -indicators and APOC staffing levels by department and type APOC Organogram Simplified graphic presentation of APOC partnership relationships and resource to countries lncremental adaptations to meet the requirements of PENDA Acronyms APOC African Programme for Onchocerciasis Control CDTI Community-directed treatment with invermectin Committee of Sponsoring Agencies GWG Governance Working Group Joint Action Forum lymphatic filariasis MDP Mectizan Donation Programme Memorandum of Understanding MOPAN The Multilateral Organization Performance Assessment Network NGDO Non-Governmental Development Organization National Onchocerciasis Task Force Neglected Tropica! Diseases ocP Onchocerciasis Control Programme Preventive Chemotherapy PENDA The proposed "Programme for the Elimination of Neglected Diseases in Africa" Results-based management Techn ical Consultative Committee Special Programme for Research and Training in Tropical Diseases of United Nations Development Programme/World Bank/WHO Terms of Reference wHo World Health Organization Acknowledgements The authors gratefully acknowledge the support of the whole partnership that comprises the African Programme for Onchocerciasis Control in commissioning, facilitating and contributing to this study. We are grateful to representatives of the sponsoring agen- cies, NGDOs, national ministries of health, and to representatives of the World Health Organization, based in its headquarters, in the Afrlca Regional Office and in APOC itself. We would like to thank the many stakeholders who gave up time for interviews and discussions ln person, by phone and by email. We are particularly grateful for the willing and practical support we have received from the Director and staff of APOC's Ouagadougou office. We are also grateful for the quality of verbal and written comments on the Discussion Draft, both at the Committee of Support Agencies and thereafter from its members and APOC staff. We have not been able to agree or include every point, but the quality of these contrlbutions has enabled us both to correct erors, improve the quality of the reports findings and analysis, and to provide more specific recommendations. The depth of analysis and strategic thinking by the CSA and its sub-groups was also powerful evldence of the value added by APOC's governance structure. We have found it a great prlvilege to be able to hold open, forward-thinking discus- sions with individuals and institutions that have had an unprecedented achievement in controlling the scourge of river blindness across Africa. We have welcomed the opportunity to play a small part, as APOC and its partners prepare a new and more ambitious programme to take an integrated approach to the elimination of neglected tropical diseases. Executive summary APOC's partners have agreed both a concept note and a strategic plan of action for a new Programme for the Elimination of Neglected Diseases in Africa (PENDA), to take forward control and elimination of onchocerciasis and other NTDs after the expiry of APOC in 2015. APOC asked for this management review to describe and analyse current APOC structures, for consideration in the development of PENDAs future management. Section 2.1. addresses the APOC Secretariat's current staffîng APoc staff are employees of wHo, and are recruited, appointed and remunerated on WHO terms of service. APOC has 82 approved posts of which 68 are currently filled. The Iargest group is the Office of the Director (OD), which includes 48 posts at head office and 15 country-based programme posts for technical advisers and financial staff. The large number of finance and admin staff reflects the demanding processes used of administrating finances for some 124 projects in 31 countries. The provision of support staff is generally appropriate to the requirements of the open- tions. This is a busy office that operates smoothly but there were wide concerns about workloads. We conclude that the quantity and skill level of human resource currently employed is required for the range of activities currently undertaken. To different degrees, APOC staff have demanding work, requiring a full working day or more. We found no evidence of over-staffing or obvious failures in efficiency. Section 2.2. addresses APOC's mdnagement structure The broad stakeholder representation of the governing body, the Joint Action Forum (JAF), was widely cited as a cornerstone of APOC's success. The public-private mix, the important role of NGDOs, the involvement of other international Organizations and country participation at a high level were features of a maturing partnership. Interviewees commented favourably on the composition, preparation and conduct of meetings of the JAF and Committee of Sponsoring Agencies (CSA), particularÿ on their abil§ to bring together people with diverse perspectives into a common programme. Both JAI' and CSA have been instrumental to the approval of the Concept Note for trans- forming APOC and the associated Strategic Plan for PENDA in 2013. The costs to the APOC budget are proportionate totheir significance inthe management and direction of APOC. Most of APOC's roles are concentrated onthe APOC Secretariat based in Ouagadougou. APOC delivers these functions through a densely organised structure based on three departments: Sustainable Drug Distribution (SDD), Office of the Director (OD) and Epide- miology and Vector Elimination (EVE). This review was conducted before the announce- ment of a new post of Corporate Services Manager. This is expected to be followed by further changes in the structure. The project remains the dominant modality for APOC funding. In the past year, APOC has initiated some significant changes to its approach, particularly a shift away from environmental zones to geographical areas based on health districts. However, we note that where a country still reports on the basis of a large number of districts (e.g. 10 districts instead of 15 projects) the burden of project management is scarcely reduced. APOC's communication strategy is not fuIly developed and as a result, is limited in scope and depth, particularly beyond the partnership. There is a lack of an open approach to results management and data disclosure. We conclude that the current management structure of the APOC Secretariat is ration- ally designed to fulfil the mandate of APOC, and is properly accountable through the Director to the JAF and to \MHO AFRO. Similarly, the financial structure likewise is ration- ally designed to manage the flow of funds from the Trust Fund to national projects. However, as currently defined and subject to anticipated changes, the role of the Office of the Director is very broad, and overlaps those of the other units. In contrast with the Office of the Director, SDD and EVE are relatively small units. This report is not an evaluation of the effectiveness of any of APOC's units. However, It is hard to see these are adequate to address the very large numbers of activities within their terms of reference. The evidence indicates that the Secretariat is principally focused on undertaking the administrative and financial requirements to direct implementation of a large and increasing number of projects. This leaves only a limited capacity for a wider current agenda in relation to onchocerciasis control and elimination. Section 2.3. provides an indication of the adaptations required for APOC to support the shift to oncho elimination, then to merge with the LF community, and then to take a wider role in the elimination of 5 PCT-NTDs. These incremental changes go beyond a minor oI even moderate adjustment of APOC's management. They will require major changes to the governance, functions, skills and systems as well as its formal structure. The adaptations required by the greater scope of PENDAs mandate derive from three fundamental challenges: Challenge 1: The changing nature of the partnership PENDA will have a larger, more complex partnership (less linear or structured in nature) that involves many more partners. The partnership will be founded on country capacity and leadership. It will need a fresh approach to communication and openness to func- tion, including greater transparency. Challenge 2: An evolving model of funding and operations PENDA will move from project support to another model incorporating results-based management and funding methods, with a stronger focus on enabling than operational management. To meet the requirements of elimination, PENDA will have to help coun- tries and health systems drive down cost and overcome barriers to a more widespread and sustainable programme than has been achieved by the project model. challenge 3: An evolving leadership based on evidence-led protocors, knowledge management and improved transparency PENDAs leadership here will come from buiiding on APOC's comparative advantages for the NTD community. Based on APOC's past and current performance, PENDAs leader- ship will be around advocacy, community-based programming, identifying best prac- tice, attracting resources, maintaining the profile of NTDs, and contributing to health systems strengthening . In a newly framed Organization, PENDA should also support countries to lead, and build an open forum for scientific discussion and best practice in treatment protocols. In this widened community PENDA will be a strategic partner in the control and elimi- nation of NTDs, complementing the growing capacities of other partners, helplng to solve operational and science-based problems, and doing only what PENDA is uniquely placed to do. We conclude that, because of the extensive transformation required for the formation of PENDA, its management system should not be evolved from ApoC by adding-on, cutting-back or re-organising. In order to deliver the much more ambitious agenda set out ln the draft PENDA document, it wilt require a balance of visionary leadership to develop clear ideas and then particlpatory engagement in the analysis and deslgn of the transformed Organization to meet the challenges set out in the Concept Note and Strategic Action PIan for PENDA. This will require decisions on the operational model for PENDA, particularly on flnancing and direction of operations. The existing project model is tikely to be unsustainable economically, as well as at odds with the basic principle of the Concept Note that country ownership and leadership are of paramount importance. Once these decisions have been taken, it should be possible to develop an operational action plan for PENDA, identifying both the human and financial resources required. This in turn will provide the essential data for creating a management structure for a new PENDA. Because the new Organization will be the result of transformation, rather than adjustments, of APOC, it would be a grave mistake to propose or to endorse any structural arrangement for PENDA without a clear commitment to the transformation required. section 2.4. uses five benchmanks, drawing on estabrished benchmarking processes. These are: 1. Success in Partnership; 2. Science-led, evidenced based decision-making; 3. Building an innovation and learning culture; 4. Adaptability and responsiveness; 5. Value for money. The research indicated that APOC demonstrates clear strengths especialty in success in partnerships and in value for money. APOC demonstrates strengths but also challenges in relationto science-led, evidence-based decision-making; building an innovation and Iearning culture; and adaptability and responsiveness. Section 2.5. summarises the strengths and weaknesses of the APOC Secretariat The APOC Secretariat has a number of strengths that have made it successful as a single disease control programme operating in difficult to access places across a range of countries. These strengths are listed in the main t ert.They are considerable and their achievements should not be put at risk during the transition process. Overall conclusions (section 3) The review's conclusions centre mainly on the proposition that the management choices available to PENDA (as the Organlzation that will absorb APOC) depend fundamentally on the decisions taken about its mandate, approach, strategy and objectives. While recent and proposed changes to APOC's management have potential to equip APOC for change, they should not be seen as any indication ofthe future staff or resources required by PENDA. APOC has successfully hetd its ground and maintained an effective control programme during a 15-year period when much donor funding became concentrated on global mechanisms to combat a fairly narrow range of prevalent health challenges identified through the Millennrum Development Goals. Combined, the management and partner- ship strengths that dellvered this result make a sound foundation for APOC to evolve towards the kind of Organization that will meet the challenges posed for PENDA. APOC's limitations can and should be addressed as it moves forward with the expanded mandate. Furthermore, as aid modalities develop overtime and countries become more confldent and assertive about leading their own NTD control programmes, APOC should expect to become more flexible in its approach to working wlth countries in order to remain relevant and useful. The three over-riding management challenges emergedfromthe evidence and analysis presented in this report: how to meet the challenge posed by the changing nature of the partnership;the urgent need to evolve a model delivering expanded, integrated and results-based implementation, in the context of a shift from projects to country-led, joined-up NTD programmes; an evolvlng leadership based on evidence-led protocols, knowledge management and improved transparency. It is an excellent trme to make the changes required for these challenges. NTD support continues to benefit from steadily increasing awareness, commitment and support (underpinned by large drug donations). Prevention and the options for reducing morbidity are recognised for their contribution to reducing the social and economic determinants of extreme poverty. However, the window of opportunity is always under pressure as overall aid funding shrinks and other priorities surface of building and maintaining commitment to the broader NTD control effort. Summary of recommendations We recommend a staged process for taking forward the management design of PENDA. Recommendation 7: agree the functions required. The management design of PENDA should be driven by a clear understanding of the functions that it will use to undertake its day-to-day tasks, in the context of the PENDA Concept Note and Strategic Action Plan. It should consider the functions required for: a) funding and implementation; b) technical capacity; c) research; d) communications. a) The fu nding and implementation models adopted by PENDA need to be matched to the needs and capacities of each country, with a clear path towards full country leader- ship and management. This should require less Organizational focus on operational and financial management of projects, and more focus on supporting countries to take the lead in identiÿing their own approach to NTD control and elimination. These changes will require fundamental changes in PENDAs staffing and skill requirements, decision- making structures, work flow and location of operations. b) PENDA's technical capaciÿ model should drive its provision of technical staffing and capacity in the regional centre and in-country. It needs to ensure that it has both the range and depth to address the key areas required for its technical leadership, and to ensure that countries have the technical support they need, complementing technical support from other national and international providers. c) PENDA's approach to research should drive the way that PENDA contributes to the development of evidence and knowledge for NTD elimination.It should define PENDÀs contribution to the generation of evidence and science, reflecting the needs and capaci- ties of countries, the partnership and still wider stakeholders. d) PENDA's communication model should be integral to its direct funding, technical support and research. Again, this will be a major driver of Organization design, including a comprehensive communication strategy updated annually. Re commendation 2 : implementation plan Once the functions of PENDA have been agreed, the proposed partners in PENDA should undertake an indicative operational planning and budgeting exercise, probably over 2-4years. . The plan should take into account both the existing commitments of its constituent Organizations, and the future strategy of PENDA. It will thus draw proposed activities from: ) existing operational plans of proposed partners in PENDA including APOC and partners supporting LF elimination; ) actlvities proposed in the PENDA Strategic PIan. Actiüties should be incorporated withln the proposed operational plan, taking account of the agreed functions (recommendation 1). A preliminary budget and staffing requirement should be developed for the proposed operational plan. A provisional staffing structure may also be set out at this stage as a basis for discus- sion and consultation. Recommendation 3 : staff workloads This report has notedwidespread concerns about current staff workloads. These needto be properly identified by managers, and taken into account in planning and managing the last 18 months of APOC's existence. This process be led by the managers in charge of each of the units and sub-units of the secretariat, under the overall coordination of the director. Recommendation 4 : transîtion timetable Finally we note that this review takes place in the context of a larger change process. The transition process should continue to be monltored closely by all stakeholders in APOC and its partners.

background to the study 1. Introduction and 1.1. Why is this study being done now? Objective of the Management Review This management review was recom- mended ln the'APOC Donor Dialogue" report and was agreed by the 141'tmeetlng of the Committee of Sponsoring Agencies (the CSA) in Hammamet, Tunisia in July 2013. There is nowa consensusthatAPOC should restructure into a new entity that will 1) seek the elimination of Rlver Blindness, 2) combine thls goal with the concomitant elimination of Elephantiasis, and 3)where appropriate, seek ways to assist WHO AFRO in implementlng the regional plan for controlling other Neglected Tropical Diseases. The objective of the management reviewis to determine whetherthe current manage- ment structure and system, includingthe level of human resources, is appropriate to perform the current duties of APOC at the Headquarters level in Ouagadougou and in the 31 countries served bythe APOC, and to comment onthe staffing and manage- ment implications of expanding the scope of the program if it is restructured into a new entity beyond 2015. The wider goal of the management review is part of a process of guiding the transi- tion from APOC to just such an Organiza- tlon. Although not formally constituted, it is understood that this will be known as the Programme for the Elimination of Neglected Diseases in Africa (PENDA) and is referred to as such in this report. APOC's partners have agreed both a concept note and a strategic plan ofaction for a proposed Programme for the Elimi- nation of Neglected Dlseases ln Africa (PENDA), to take forward control and elimination of onchocerciasis, lymphatlc fllariasis (LF) and other NTDs after the expiry of APOC in 2015. This management review aims to describe and analyse current APOC structures, in light of the PENDA proposal, in order to meet the new challenges of NTD control and elimination across the African region. As the Concept Note for PENDA empha- sises, country ownership and leadership is of paramount importance. Context Onchocerciasis and other NTDs have continued to attract worldwide funding, support and research interest, along with support for AIDS, TB and malaria. This is in a context of increasing focus of global funding on the main drivers of prevent- able mortality, especially in women and children. Over the last 20 years, progress has been made to the extent that elimination of one or several NTDs, including onchocer- ciasis, can be considered a real prospect. Blindness has become more rare, and the plague of itchy skin, so common a couple of decades ago, has also largely vanished. What has been required to reach this position is a sustained focus, continuity in resource mobilisation, a compelling strategy and successful coverage across more than 20 countries over many years. In the final stages of control and ellmina- tion, when cases have diminished signifi cantly andthe disease is almost unheard of amongst young people, the political will required to maintain funding and commitment becomes increasingly hard to mobilise. Yet, the alternative is a slow (or not very slow) return to the previous situation that puts all the investments over decades at risk. The challenge to APOC in the current global context of declining resources, increasing pressure on prioritization and the pursuit of sustainable, lasting impact is to replace APOC with an Organization that will drive the elimination of a number of NTDs across a the whole region. To do this successfully, and to retain the confi- dence of its partners, APOC needs to work attentively with partners to identiÿ and sift out cumbersome or outmoded prac- tices while ensuring the new Organiza- tion retains and builds on its comparative strengths. Recent development Since the research for this review and completion of the draft report, APOC Secre- tariat has appointed a new Manager of Corporate Services. Further developments are under consideration, and together these could both to reduce the amount of APOC's business requiring the direct intervention of the Coordinator and Director, and increase the scope for delega- tion of decision-making and budgetary authorityto unit managers. These in turn could greatly increase APOC's capacity to respond to the immediate and longer term challenges set out in these recommen- dations, by empowering APOC's senior management team to work on managing their staff's workloads and to take a fuIl part in shaping the operational plans and resources for PENDA. 1.2. Approach and methodology Approach Our starting point for this management review is that the design of an Organiza- tion should be based on the purpose for which it has been established and the chal- lenges it exists to overcome.These in inturn should inform its objectives, activities and the structures put inplace to achievethem. Table 1: APOCTimeline 1975 1995 2002 2009 2012 2013 2014 2015 2016 . Onchocerciasis Control Programme established . APOC established . Phase ll of APOC starts . Emergence of evidence that ivermectin treatment alone can lead to the elimination of onchocerciasis . JAF 18 agrees to target elimination of Onchocerciasis and coordination with lymphatic filariasis elimination . Phase-out of APOC starts . Adoption of Concept Note and Strategic Plan of Action for PENDA . JAF 19 agrees to formation of PENDA . Governance Working Group and Transition Task Force start work on transi- tiON tO PENDA . Management Review . PENDA governance finalised . Management structures in place . APOC closes . PENDA to start work The context for this study, conducted in May and June 2014, is APOC's longer-term plan, summarised inthe Strategic Plan of Action for the establishment of PENDA. This plan seeks to ensure that appropriate governance and management structures and systems are in place intime for PENDA to start work in 2016. The ToR require the consultants to comment on the staffing and manage- ment implications of expanding the scope of the program if it is restructured into a new entity beyond 2015. However, because PENDA, as this entity is called here, will have an expanded scope,we do not assume that it will be "more of the same", or even that APOC's efsting tasks will be the same. Organizations with similar purposes may have very different structures: for instance a postal service and a mobile phone company may share a purpose of improving communications, but achieve this in a very different manner. In the tran- sition from one form to the other, some functions are reduced or fall away entirely, whereas new ones are required. A management review cannot, there- fore, be conducted in isolation. In order to be able to comment on whether APOC is doing things right, it has to comment on whether it is currently doing the right thing. Therefore, without seeking to be a strategic review itself,this report needs to reflect the strategic choices already indi cated or made available to APOC and its partners in the formation of PENDA. Methodology The process included review of available documents and strategies (listed inarmex Z); a visit to the APOC Secretariat ln Ouaga- dougou in M ay 2014 allowing the scrutiny of planning, finance, administrative and project implementation documents in APOC headquarters, and; interviews with a wide range of stakeholders, partners and accountable staff (listed in Annex 1). Regrettably some key staff were on leave, on duty abroad or sick. For this reason, further interviews were conducted during the forthcoming CSA, before finalisation ofthe report. The consultants received excellent support and assistance from APOC during the visit and afterwards; all requests for relevant information held by APOC were satisfied where that information existed or was held by APOC. Theterms of reference (TOR) (Annex 3)and delays in setting up planned interviews before our visit to Ouagadougou did not allowfor as much direct engagementwith stakeholders within national programmes as we would have liked. However, we were able to interview a range of stake- holders including APOC staff, senior health officials and implementing agencies in several countries. While their contribu- tions were very helpful in building our understanding of how APOC functions, its strengths and challenges at operational (country) level, the report is on much more solid ground in relation to the APOC secretariat and its work in Ouagadougou, including the implications of APOC's approach of funding multiple projects in each country. Any management review is concerned with economy, efficiency and effective- ness. The examination of productiüty in a complex Organization is a challenge. Techniques for assessing productiüty in repetitive industrial, manufacturing and service operations and not appropriate here, and the consultants have had to draw on broader tools of observation and assessment. In Section 2, Findings, below, the report structure follows structure of the Main and Specific Objectives of the TOR, which are noted at the head of each section. Section 3 identifies key conclusions and section 4 sets out our recommendations.

2. Findings The findings below reflect the Main and Specific Objectives of the TOR. 2.1. Current staffing ü-earlhs-reuf--el-Befie!euei- Determine whether the level of human resources is appropriate to perform the current duties ofAPOC. 7. Descrtbe the dtn,:ent lanel of APOC statling at the seoetartat and countryt l*els, their qualÿîcations, their remuneratlon and their roles andrcsponsiDflfties. > The qualifications of staff, their locations, their support stnrctures and their respective roles, duties and responsibiliües. 2.1.7. Framework for employment APOC staff are employees of WHO, and are therefore recruited and appointed on WHO terms of service, as set out in \MHO Staff Rules and Regulations, and in the WHO e-manual III Human Resources. This proüdes a detailed and accountable framework for employment including recruitment, appointment, grading, sala- ries and benefits. Employment conditions are therefore non-negotiable. Figure 1: APOC staffing levels by department and type Staff are recruited on an international basis for director or professional grades. Staff are recruited locally for national professional officer posts and general service posts. 2.1.2. Staffing levels and locations APOC has 82 approved posts of which 68 are currently filled (vacanry rate is 18%). The largest group is the Office of the Director (OD), which includes 48 posts, Vaccant I Di rector/professionnal staff I National professionnal Officer I General Service staff SDD: Sustainable Drug Distribution OD: Office of the Director EVE: Epidemiology and Vector Elimination of which 9 are vacant. 15 country-based programme posts for technical advisers and financial officers also report to the Office of the Director. Of the remaining 35 posts in the Director's Office, 13 posts relate to budget management, finance and accounting, largelyto support imple- mentation of projects in-country. The remaining staff are largely focused on reporting, administration, procurement and inventory, support functions and translation. Other staff are shared between two smaller departments: Epidemiology and Vector Elimination (EVE) and Sustain- able Drug Distribution (SDD) with 12 and 7 posts respectively. The deployment of these staff within APOC is discussed further in Section 2. 2.7.3. Roles, duties and responsibilities Approved job descriptions and grading details were provided for each post. Job descriptions show the purpose, broad objectives of the post and supervisor of each job together with the expected qualifications and experience required to fill the post. The AFRO Director in Congo- Brazzaülle approves the creation of each professional grade post along with its associated job description. General grade posts andjob descriptions are the respon- sibility of the APOC Secretariat. The func- tions of the three units are described in Section 2. The large number of staff needed in the finance function is prominent. The Budget and Finance section of the Director's office has 13 posts, more then either SDD or EVE. This reflects the demanding process of administrating finances for some 124 projects in 3l countries. Much of this work is done on paper, as there is limited use of electronic accounting and reporting systems in-country. Staff in all units are required to provide prompt response in order to service the large numbers of projects. Because of the limited use of delegated authority, the director's authorisation is frequently required. This adds to the office workload. The ratio of general staff to professional grade staff is approximately 2:1. General staff are predominantly administrative, secretarial, financial, procurement and accounts staff with a small number of maintenance technicians and drivers. The provision of support staff is gener- ally appropriate to the requirements of the operations in an Organization which a high administrative workload. This is a busy office that operates smoothly. Docu- ments requestedwere generally easily and quickly retrieved. Staff appeared to work in an orderÿ and productive way. The working day was long although in common with many country-based UN offices, Friday was a short day. The working weekwas 40 hours as prescribed by the UN system in country: 07:30 to 18:15 from Monday to Thursday with a two- hour break at lunchtime; 07:30 to 12:30 on Friday.In other similar settings, public service staff have been gradually moved over to a more economical system based on 7.5 to 8 hourworking days (7:30 to 15:30) with short lunch breaks. This may be better for employees as it limits the time and cost of commuting to once a day. It is not clear whether this format is under consideration in Ouagadougou but as a UN office it will continue to followthe hours established for all UN operations in Burkina Faso. There were wide concerns expressed by many staff about workloads across the HQ office. Some staff reported that they were at the limit of their capacity. This workload appears to be linked to the large number of processes required to deliver high quality, effective projects. But it is also probably linked to the number of hours I II ) I spent at work given the practicalities of taking a two-hour break in the middle of the day in practice (transport home and back twice each day). We found several staff members at their desks (often eating lunch at their desks)throughout the week we were there, suggesting that for many general grade employees, their working day is probably routinely 9 to 10 hours a day. 2.7.4. Remuneration and qualifications AII staff are remunerated according to United Nations and the WHO classifica- tion system, which are stipulated for both grade and location. Average remunera- tion costs for each grade are determined centrally and are published inthe annual Plan of Action and Budget (PAB) 2014-15. This report does not offer a comparable assessment between employment costs of \MHO and other international bodies. A review of all staff CVs was done. CVs were assessed against the approved post description including qualifications and experience required. Staffin post appeared to hold appropriate qualifications and experience for the posts they held when measured against objective requirements. Some of the higher grade general staff (G6 and G7) had sufficient qualifications to satisÿ requirements for P1'posts (a university degree), but theywere few and the posts they occupied were administra- tive and financial posts. No evidence was found that staff were significantly either over- or under-qualified for their posts at the time of their appointment to those posts. Amongst professional staff appointed in country (programme coordinators or tech- nical assistants at country level), appoint- 1 The UN professional grades (P grades) run from P1 (lowest) to P5l6 (highest). A Masters degree and several years'experience are required for a P3 post. Those without Masters degrees and few years of experience are appointed to P1 or P2 grades. ments tendedto be at P2 grade (with one P3). Ail country based technical assistants were qualified enoughto be appointed at P3 or even P4 grades based on minimum entry requirements. However, the posts were advertised at P2 grade and clearly attracted very strong applicants. APOC has succeeded in attracting and retaining high quality staff in posts for which they are qualified. Approximately half of professional staff have served for five years or more. There is a preponder- ance of older staff at senior grades, espe- cially in the SDD department, with well- established reputations prior to entedng service with APOC. Professional staff are active in maintaining andimproving qual- ifications and professional track records. For example, many of the laboratory staff have won scholarshipsto do post-graduate studies at an American university with which the lab has developed an on-going relationship. Recent appointments of staff with qualifications and experience in busi- ness and health economics strengthen these skills areas in an Organization where natural sciences and health disciplines have been predominate. Withinthe next 4years, based on a manda- tory retirement age of 62,25% of current professional grade staff will be retired. Ofthese, 4 ofthe 25 staff at professional grades will have reached retirement age within two years, and a further 4 within four years. It is understood that the UN may adjust the mandatory retirement age for staffto 65 from 2014; a decision about this is expected in the course of 2014. Nonetheless, the decision would delay a major loss of staff rather than address it. Continuing education or skills upgrading was not assessed. While for general staff, there are a number of online and class- room opportunities for skills improvement within the WHO and UN systems gener- ally, the concern is more about whether professional staff would be in a position to build experience in newways of working with other partners and knowledge about new health systems strengthening modal- ities at the country level. Analysis ofjob definition, grading, recruit- ment and remuneration showed a high level of rational allocation and account- ability. No injustice or manipulation of these areas is apparent. There is also now a reasonable transparency in remuneration Ievels, following a decision of the CSA at APOC Secretariat's request that remunera- tion costs be shovrn in financial reports in full rather than distributed across activiÿ budgets. 2.1.5. Nationality and gender APOC is predominantly African in both its location, as required by its MoU, and also in its staffing. Altogether 42 out of 68 staff are from Burkina Faso (17% of professional grades representing 4 out of 29 people and 86% of general grades). Among the professional grades, staff are drawn from 18 countries. This represents a reasonable balance of nationalities for an Organiza- tion that works across the African region and where selection is based on merit. 30 out of 68 staff (44%) are women. However, at the professional and director level, only 4 out of 23 staff (14o/.) are women, whereas 26 out of a5 $7%) national professional officers and general staff are women. 2.7.6. Consultants APOC agreed 14 consultant contracts in 2013 with individuals for periods of between 5 days and 12 months. Four contracts covered a period longer than 3 months. Six contracts included visits outside Ouagadougou. This suggests that APOC relies on the technical resource provided by permanent staff in Ouaga- dougou and each country. Similar programmes might use more consult- ants, particularly from within the African region, to proüde more flexible assistance, bring comparative experience and new perspectives. Pay bands, per diems, honorariums and other costs associated with the use of consultants all appeared to be in accord- ance with clear and explicit \ÂIHO policies. 2.7.7. Conclusion on current staffing We conclude that the quantity and skill level of human resource currently employed is requiredforthe range of activ- ities currently undertaken. To different degrees, APOC staff have demanding work, requiring a full working day or more. We found no evidence of over-staffing or obvious failures in efficienry. In the next section we discuss the appro- priateness of the activities undertaken, and particularly the large amount of administrative and financial tasks related to the management of projects. 2.2. Management structure F r orn tft gle rmç. OI &eft rc ase; Determine whetherthe current management and structure and system... is appropriate to performthe current duties of APOC. 2. Descrtbe the management stflrctt re of APOC, especialÿ the rclation- ship, including roles and rcryrcnslbilities of stafr, betveen trOC and the national gonernment prqrums. > The organogram and organizational connections; > How funds flow, including the relationship with \/VHO systems; > The accounting process, induding the relaüonship with WHO systems; > The reporting system within and beyond the Secretariat; ) How management meetings are organized and theirfrequency; > Any extemal audit queries and management response; > The management decision makins processes; > The coordination mechanisms within the Secretariat; > The communication process and strategies within the Secretariat; > Govemment participaüon at the regional and countrylevels. 2. 2. 1. Org anîz ational Structur e The structure and governance of APOC is defined by the'Agreement/ Memorandum of Understanding" (MOU) for the African Programme for Onchocerciasis Control, most recently updated and signed in 2011. The organogram below shows how the Programme Director reports both to the Joint Action Forum (JAF) of the APOC part- nership and to the executive agenry, \ÂIHO, of whichAPOC is a programme. The lower part of the organogram shows the three units that, at the time of this review, comprised APOC's functions and the key staff, mostly at professional level, within them. AII posts are in Ouagadougou except for the in-country posts shown in the Office of the Director. The in-country teams are supported and freely consult with the other departments in APOC but they are managed by the Office of the Director's department. From the above, the Programme Director has twin reporting responsibilities: to WHO as Executing Agency; and to the Committee of Sponsoring Agencies (CSA) and the Joint Action Forum (JAF) under the MoU. Such twin reporting responsibili- ties are not uncommon, and, even with sensitive handling, may expose divergent interests. This has to be weighed against the benefits of twin accountability. In this case, these benefits include the high degree of common ownership and resource mobilization provided by the partnership; and the political status and commitment, management systems and guidance provided by WHO. At the end of June 2014 the appointment of a new Corporate Services Manager was announced, reporting to the Director, alongside the Coordinator post, also knornrn as the Programme Services Manager. The latter post is vacant following the retire- ment of the former Coordinator at the end Figure 2: APOC Organogram JOINTACTION FORUM COMMITTEEOF SPOT{SORING AGET{CIES DIRECTORGENERAL wHo REGIONALDIECTOR WHO/AFRO ofJune 2014.The implicatlons of this couid not be addressed directly in this review, as the appointment was announced only after the discussion draft had already been produced and considered, andthe Organi- zational changes remain to be decided. However, they offer potential to increase delegated decision-making and budgetary authority. 2. 2.2. Partnership Structure The Memorandum of Understanding (MoU) defines the partners and their rela- tionships in APOC. . APOC includes Participating Countries in the elimination programme. . The programme is executed by the Executing Agency, WHO, represented by its regional office, AIRO. EPIDEMIOLOGY& VECTOR ELIM]NATION .2 epidemiologlst . Blostatlstlcs& mapping officer . lnformation technologyofflcer . lnformation systems offker . Technicaladvisor- vectorj . The World Bank is the Fiscal Agent for the fund, receiving funds from contrib- uting parties, and paying them out to WHO as Executing Agenry. . ContributingParties are govemmental, intergovemmental or private Organiza- tions that contribute funds to APOC. . Sponsoring Agencies are intergov- ernmental agencies supporting the programme: they are currently the World Bank, \MHO and African Devel- opment Bank (AfDB). The statutory meetings of APOC are set out in the MoU. Partnership representation and mix The broad stakeholder representation of the Joint Action Forum was widely cited as a cornerstone of APOC's success. TECHNICAL CONSULTATUE COMMITTEE SUSTAINAIAEDRUG DISTR!BUNON . Communitÿ ownarshlp& partnerrhlp,officer . Health systems speclalist . Operatlonnal reserrth scleôüst . Communicatlon and advocayofflcer PROGRAMME DIRECTORAPOC OFFICE OF DIBECTOR . Coordinator . Admlnlstratlon irfflcer . Programme meetlngs offlcer . ,Budgetand flnance offlcer . Flnance oflicer . Tran3lator ln-countryposts . 6teclmical advlsers . Country flnance offlcer . 4countryflnance assltantg Table 2: Summary of statutory meetings and functions Joint Action Forum (JAF) Committee of Sponsoring Agencies (csA) . Contributing Parties . Participating Countries . Sponsoring Agencies . NGDO Coordination Group members . Merck and co . Others by invitation . Sponsoring agencies . NGDO Coordination Group representative . Merck and co . MDP . Reviewandapprove Plan of Action and Budget (PAB) . Assess financing requirements . Decide on expansion of APOC's activities or countries . Reviewandapprove adjustments to PABs . Examine reports from Sponsoring Agencies and transmit to JAF . Advise JAF on expansion of APOC's activities or scope . Act on behalf of JAF between JAF meetings . Advise Programme Director on criteria for implementation ofCDl and other disease control; funding parameters; project proposals; technical and operational research . Review applications for funds . Review progress towards elimination of onchocerciasis and integration of CDI into the health system . Review new national plans . Coordination . lnformation sharing . Technical support . Annually or biennially, as it decides . Costs paid by JAF members . Four times a year . Meetings may be held by teleconference . Costs paid by CSA members . At least once a year (normally 2) . Costs paid by APOC . Costs paid by NGDO members Technical Consultative Committee (TCC) NGDO Coordination Group . 11 members appointed by APoc on recommendation ofCSA, ofwhom 3 proposed by NGDO Coordination Group . MDP or other ivermectin donation programme . NGDOs and institu- tions collaborating in CDI The public-private mix, the important role of NGDOs, the involvement of other international Organizations and country participation at a high level were features of a maturing partnership. The role of the Mectizan Donation Programme (MDP) was widely referred to as an important driver of the partnership and a factor in main- taining and accelerating momentum first towards control, then elimination. Several partners suggested that, in the wake of the London Declaration (signed January 2012), the partnership needed refreshing to take account of the rapid growth of country-based river blindness projects and increased interest in NTDs generally, and in a context of declining resources inAPOC. The CSA has a narrower representation than the iAF but meets more frequently. Its members include the Sponsoring Agen- cies, \MB, \MHO and African Development Bank, with NGDO representation. It meets more often and works at a more execu- tive level to support APOC to implement the decisions taken at the JAF. Recently, it was decided to convene an expanded CSA that includes many more partners and covers a practical, implementation- focused agenda. An expanded CSA was held in June 2014. Effectiveness of Partnership Interviewees commented favourably on the composition, preparation and conduct of these meetings (JAFs and CSAs), particu- larly on their ability to bring together people with diverse perspectives into a common programme. Documents were generally well prepared and distributed on time. One interviewee reported that some documents were distributed late some of the time. There were some ques- tions raised about the appropriate time APOC should spend planning the JAI' every year. It takes several months of prepara- tion usually. However, the JAF is the most important meeting of the year for APOC, making decisions about strategic direc- tion and setting APOC's agenda for the subsequent 12-month period. As such, it is appropriate for this meeting to be well planned and prepared. Given lts role in the partnership, there was agreement that poor preparation of the JAF would be wastefuI and damage the effectiveness of the partnership. The MoU allows JAF to meet either annu- ally or biennially. Several interviewees questioned whether the JAF could be held once in a biennial given its strategic role especially if the CSA was always the expanded type. JAF and CSA have been central in APOC's key strategic directions, including the move towards onchocer- ciasis elimination, greater integration of approaches with LF and other PCT-NTDs, exploring options for introducing new treatment regimes to hasten elimination results, and a move away from projects to integration with local and national health systems. Both IAF and CSA have been instrumental to the approval of the Concept Note for transforming APOC and the associated Strategic Plan for PENDA in 2013. Less frequent meetings could have delayed these. Having made these strategic decisions in principle, APOC appointed new members to the TCC in order to better reflect and integrate LF partners'interests and views. It established a Governance Working Group (GWG) and a Transition Task Force (TTF) reporting to the CSA, to develop both the governance and technical arange- ments for PENDA. While the pace of progress seemed slowto some partners, it is important to balance this with a longer view on APOC's achieve- ments. APOC Phases I and II have imple- mented and sustained a radical, evidence- based intervention over a 20 year period. During this period there has been equally radical change elsewhere in approaches to NTDs and basic health services, much of it informed by APOC's experience and methods of mobilising communities. APOC has taken a measured approach to change as it approaches closure. Some interviewees were concerned that APOC is slow to adapt and implement policy changes. Progress towards the establish- ment of PENDA started well ahead of APOC's planned closure in 2015. The Stra- tegic Plan sets out the transition tasks and timetable (Annex 3: What needs to be in place before the programme starts?), and this work has started. However, both the Transition Task Force and the Governance Working Group are only starting to get under way in 2014. Partnership costs The costs of the statutory meetings are proportionate to their significance in the management and direction of APOC. The amount that APOC spends on these meet- ings is shown in the table below'. The costs incurred for the JAF and CSA relate only to the costs of hosting the meetings and APOC staff attendance. Although we have not scrutinised these costs in detail, we note the costs per meeting and do not see these as neces- sarily out of proportion to their value. The costs of the TCC include those of funding most of the participants to attend. Given the different character of the TCC, this is considered appropriate. 2 As each participant pays their own way to attend the JAF and the host country fund5 the venue and other costs, the funds shown here are APOC'S expenditure on these meetings rather than their cost. 2.2.3. Resource îlows wîthin the partnership This review is primarily not about the financing arrangements for APOC, its sources of funds, amounts and accounta- bility arrangements fromAPOC to donors. It does not consider these elements of APOC in any depth beyond a brief description (below) in order to provide the necessary background to discussing the more relevant parts of this review: budget management, prior§ setting, the administration and delivery of projects, and cost control. Resources to replenish the African Programme for Onchocerciasis Control (Phase II) Trust Fund have been raised from over 20 donors since 2001.Inthe last bien- nial,10 donors made resource grants of around USS31m into World Bank admin- istered Trust Fund. Additional resources are granted to APOC from other sources. For example, WHO AFRO has funded the renovations to improve and upgrade secu- r§that are being undertaken in mid-2014. Merck provides the drug donations for all Onchocerciasis programmes through the Mectizan Donation Programme but they also contribute financial resources to the Trust Fund. Some NGDOs transfer resources to APOC for specific purposes. For example, Sightsavers provides ring-fenced funding to support technical capacity building. Resources also flow direct to projects and countries from donors. From a management perspective, there does not appear to be a unified statement of all resources available for onchocerciasis control either in APOC or in countries. It is Table 3: APOC expenditure on Statutory Meetings in the last Biennial (2012-13) JAF csA TCC 2 8 119,851 20,346 40,781 239,702 162,771 163,125 not clear what the full resource envelope is for APOC or for onchocerciasis. Accountability for resources is to the Iarger JAF forum (a11 partners), to WHO AFRO and headquarters, and, individu- ally, to countries where APOC operates. An annual audit is undertaken, the most recent of which led to some helpful recom- mendations about the presentation of accounts but no concerns about the use of funds. Budgets are developed each bien- nial (and subsequently for each year) based on activities. This is a useful practice in many ways but falls short of the more effective (and generally best practice) approach based on results-based management (RBM). Over the last year, APOC has been putting in place a series of reforms to improve its planning and accountability. At the centre of this is a Managing for Development Results system, and Enterprise Resource Planning. These are intended to allow APOC to plan, monitor and evaluate its results /outcomes, outputs and activities, against input costs and the use of avail- able resources (for example, staff, tech- nical capacity etc.). However, while the financial components of the system are in place, the monitoring of results and outputs lags behind, andthe system is not yet widely understood or adopted within APOC.There are aspects of communication and reporting that appear to be behind as well. If it can be successfully implemented, these reforms would provide the RBM approach required to allow APOC - and in due course, its successor - to clearly iden- tify its results and then to allocate human and financlal resources to achieving these results. It will also contribute to a sound framework for accountability for results. Ultimately, in terms of monitoring and then reporting, RBM will allow all part- ners to see clearly what the full package of resources is, how these are allocated to results (rather than just activities), how successful the achievements in-year have been and the cost per result (effectiveness measures). RBM takes activity budgeting further and contributes to performance monitoring because it allows greater accountability and openness in the use of resources l. Delivery is direct through projects at country level operating in endemic areas and usually as partial contributions along- side NGDO, national and other funding. APOC's country budgets used to include the financial support it provides direct to projects and its own costs for delivering project support. The recommendation of a recent audit that APOC's costs (for example, APOC's staff costs) should be presented as a consolidated amount and not apportionedto projects has improved the transparency and reduced confusion. Thus, resource flows and lines of account- ability are complex. APOC tries to build sustainability in countries by reducing its own funding commitments over time. In reality, in many cases, this creates a gap that others fill where countries have not built sufficient resource commitments from their own budgets. APOC funding is insufficient to support any individual project entirely and it is almost always one of several funding sources for project delivery (and sometimes only a small part). As mentioned, audits have been done at the right time and have been generally helpful. All requests forfinancial expendi- ture were satisfied; APOC has records of all its expenditure and can allocate this to activities. Thus, APOC is in a good posi- tion to rapidly strengthen and improve its reporting even further. 3 WHO has committed to strengthening results-based management (in the MOPAN and in its response to DFID's multilateral aid review). ln this case, WHO's partner programmes would need to shift their budgeting approach as well. Figure 3: simplified graphic presentation of APoc partnership relationships and resource flows to countries ----> Partnerships --)Resourceflows 2.2.4. Management of APOC functions APOC is a multi-faceted Organization incorporating a wide range of roles and responsibilities across several distinct partners. Most, but not all, of these roles are concentrated onthe APOC Secretariat based in Ouagadougou. APOC and its associated structures cover awide range of functions includingthose shovrm below, Setting goals and objectives (withJAF); Policymaking including treatment regi- mens (with advice from TCC); Standards and norms (in cooperation withWHO HQbut not always clear); Fundraising (APOC, JAF, WB); Advocary; Planning and budgeting; Training (of trainers, entomologists, epidemiologists); i ffi #r*M e6*ffii dYrffi ffitrw,$. ffi: p]ryM wy'ryl ffiÏ}tr;{';ir-At i.-- --L--- - -- J- -----rI *i&ri,.wi*i!**{§l]Mr*i,i;æii,i**?;r:r*Àt. ffiffiffiàilf,FË.è i.iffi aAl --L------l------l . Implementation and project delivery (with countries and NGDOs); . Distribution of drugs and verification (withMDP); . Reporting (financial and programmatic withNGDOs); . Monitoring and evaluation; . Surveillance (both disease prevalence/ incidence and vector); . Accountabil§management; . Communication and awareness-raising. APOC delivers these functions through a densely organised structure based on three departments: Sustainable Drug Distribution (SDD), Office of the Director (OD) and Epidemiology and Vector Elimi- nation (EVE). By far the largest of the three units is the Office of the Director, with 39 filled posts and a further 9 vacancies. In-country tech- nical advisers andfinance staffalso report I I I I I I I I L- I ffiWff W -e3ffi:§ ffi# i#ffiH"'ï§:Effi : ,ry "*t.:-{#/+AAA a a Figure 3: Simplified graphic presentation of APOC partnership relationships and resource flows to countries * Partnerships --)Resourceflows IilI i-___{_____ _a_ _ __ _, WTT . Implementation and project delivery (with countries and NGDOs); . Distribution of drugs and veriflcation (withMDP); . Reporting (financialandprogrammatic withNGDOs); . Monitoring and evaluaüon; . Surveillance (both disease prevalence/ incidence andvector); . Accountabiliÿmanagement; . Communication and awareness-raising. APOC delivers these functions through a densely organised structure based on three departments: Sustainable Drug Distribution (SDD), Office of the Director (OD) and Epidemiology and Vector Elimi- nation (E\Æ). By far the largest of the three units is the Office of the Director, with 39 filled posts and a further 9 vacancies. In-country tech- nical advisers and finance staff also report 2.2.4. Management of APOC functions APOC is a multi-faceted Organization incorporating a wide range of roles and responsibilities across several distinct partners. Most, but not all, of these roles are concentrated on the APOC Secretariat based in Ouagadougou. APOC and its associated structures cover a wide range of functions including those shownbelow: . Setting goals and objectives (withJAF); . Policymakingincludingtreatmentregi- mens (with advice from TCC); . Standards and norms (in cooperation withWHO HQbutnot always clear); . Fundraising (APOC, JAF, \ArB); . Advocacÿ; o Planningandbudgeting; . TraininB (of trainers, entomologists, epidemiologists); . advocacy, communication and health education; . analysing CDTI project plans, budgets, performance and evaluation. SDD's terms of reference includes several different types of engagement: . project analysis, linked to project approval and management; . contributions to guidance andtools; . facilitation, training and capacity development; . advocacy and communication; . research and research dissemination. It is notable that SDD's duties in analysing and commenting at key stages in relation to 121CDTI projects are only one portion of a much wider role. This is a very wide- ranging brief for three professionals and three support staff. b) Epidemiology and vector elimination department EVE's objectives are to: . contribute to the elimination of oncho by 2025 and to support Member Coun- tries on disease mapping, programme data management and analysis and monitor the trend of epidemiological indicators for oncho; . assist member states to determine when and where ivermectin treatment can be safely stopped and to provide güdance to countries on preparing to stop ivermectin tre atment; . play a leading supportive role in providing guidance to and build ento- mological capacity of countries to monitor and evaluate the progress of their interventions towards intemrption of transmission within the framework of elimination of oncho by 2025. EVE's terms of reference include: . coordinating oncho and LF evaluations; . disease surveillance; . entomologicalsurveillance; . eliminationplanningandmanagement; . integated disease mapping of PCT-NTDs; . information system management, anal- ysis and sharing; . collaborationwith\MHOininformation systems, monitoring and evaluation; . cost analysis; . health impact; . capacitybuilding; o training planning. As with SDD, EVE expects to undertake a wide range of interventions, including: . country and project level interventions; . progrumme wide evaluation; . facilitation, training and capacity building; . integrated approaches to epidemiology and vector elimination across PCT-NTDs; . data management and sharing; . development and delivery of training. As with SDD, this is a very large agenda. EVE comprises 4 professional and 8 general service staff. c) Office of the Director The objective of the Office of the Director is to provide, in an integrated manner, all management and administrative support services to the APOC senior management and technical programmes. The scope of the department covers all functions that support technical programme delivery in an efficient and effective manner, including: . strategic planning; . operational planning and budgeting; . perforrnance monitoring and evaluaüon; . advocacy and stakeholder management; o resources mobilization and external relations; . management of financial and human resources; . provision of all logistic support; . communication; . translation; . management of informationtedrnology; . appropriateaccountabil§andgovem- ance mechanisms across all areas. The office is headed by the Coordinator of the Office of Director who holds a very broad terms of reference. These go far beyondthe normal brief of an administra- tion fund as described above.In addition to the above, these include: . maintain and reinforce the Programme linls with the Ministries of Health in the endemic countries, the NDGO Group and the \ÂIHO representative offices; . assess training needs of participating countries and overseeing capacity building in countries; . oversee and coordinate vector control and drug distribution in the Special Intervention Zones. Notably, the Coordinator is also respon- sible for all technical advisers and finance/ administrative officers in endemic countries. There is considerable overlap between the fu nctions of the Coordinator and those of the other two units. . SDD has the objective to support Member Countries to establish simple, effective and self-sustainable CDTI systems in endemic areas; . EVE plays a leading supportive role in providing guidance to and build entomological capacity of countries to monitor and evaluate the progress of their interventions towards intemrption of transmission yet; . the Coordinator maintains and rein- forces links with ministries, NGDOs and the \MHO in-country assesses training and oversees capacity building. It is difficult to see how SDD can fulfil their leadership in these areas, while the Coordinator retains such central responsi- bilities, and manages all in-country staff. However, despite these possible contra- dictions, APOC's centralised model of management underthe Director and Coor- dinator has delivered results for control of Onchocerciasis. Because of the constraints noted in Methodology above, we did not have sufficient contact with country offices or national programme managers, to be able to assess the effect ofthis centralisa- tion onthe development of authentically autonomous management and leadership in-country. 2.2.5. Management of projects Classic CDTI project model The following sketch outlines the steps of the establishment of a classic oncho- cerciasis control programme, made up of multiple projects within a country. Although APOC has moved away from zone-based projects to a district-focused programme,the project remains the domi- nant modality for APOC funding. l Table 4: Start-up and project sequencing at country level APOC guidance ldentify project sites Mapping consultant Consultant contract !dentification of need and development of National On<ho Control Plan Mapping to identify hyper- and meso- endemic areas Establish Nationa! Onchocercia- sis Task Force (NOTF) Start designing projects lst year CDTI in at least 1/3 communities 2nd year CDTI - at least 2/3 communities 3rd year CDTI - all communities 4th year CDTI - all communities 5th year CDTI 6th year CDTI and onwards . APOC guidance. Support for "Head Office Project" includes vehicle and other capital costs, National Programme Coordinator and other salary top ups, consumables. . Technical Adviser and or Finance and Admin Assistant Capital and recurrent costs for project from APOC directly to projects . Workplan and budget 6 months in advance . Letter of Agreement between MoH and WHO/APOC . 6 monthly Expenditure reports . Staff contracts . As above . Annual Report . TCC Reports . lndependent participatory monitoring report As above As above As above As above As above + mid term or As above As above As above . Replacement capital As above equipment funded . Capacity building . No recurrent costs funded any longer 2.2.6. Recent and. foreseen changes to classic model Inthe pastyear,APOC has initiated some significant changes to its approach, mainly in light of the shift to an elimi- nation strategy. The most important of these is a shift away from projects based on environmental zonesto coverage based on health districts (or other national administrative health units). However, we note that where a country still reports on the basis of a large number of districts (e.g.10 districts instead of 15 projects) the burden of project management is scarcely reduced. In addition to this, other changes have been made to the classic model. These include: . 3-monthlyreportingreplaces6monthly reporting; . projects include plans for co-implemen- tation with LF; . inte8ration with other PCT-NTDs; . development of national master plan may supersede a national oncho plan. Projects used to be sent to the Technical Coordination Committee for approval at the planning stage. With experience and improved training, but also because of the burden of managing large numbers of projects,the TCC stopped approving indi- vidual projects, but reports from countries in their first 7 years of implementation were reviewed. Now, it is understood that the TCC will look at country reviews at certain periods of development. The project cycle, once agreed and under implementation, requires direct reporting to APOC in Ouagadougou. Financial management standards, as reported, appear high, and funds are released only when reporting is accurate and complete. Recently, APOC took the decision to shift from its existing accounting system which required projects to report onlywhenthey had spent 70% of project funds, before the final 30% was released.In practice, some projects failed to report at all, foregoing the final 30%. This has been replaced by a requirement for trimestral reporting. This means that project accounts will be scrutinised more frequently, but the reporting template has been simplified to make reporting and analysis easier. In addition, in 7 countries, APOC operates "HO_ Projects" which are programmes of support to national coordination struc- tures. These countries are Sudan, Nigeria, Cameroon, Ethiopia, DRC, Tanzania, and Angola. The HQ project supports equip- ment, vehicles, training, monitoring, reporting and recurrent costs. After 5 years, costs for staff (like a programme coordinator in the MOH, a driver or a data collection officer) and for recurrent expenditure (such as petrol/ diesel) are no Ionger eligible for cover from APOC. This is a rule that aims to ensure country-based onchocerciasis programmes become well-established but also seekto become sustainable and nationally owned. There are a number of limitations with the project model: . projects are not likely to be fully inte- grated into national health system structures where they are planned around environmental zones rather than existing district or national health structures; . project accountability is primarily to APOC rather than to district or national health authorities; . NGDO co-funding is usualiy required as APOC funding is not sufficient to cover all costs; most projects, at least inthe earlyyears, are dependent on topping up salaries to ensure the availabil§ or dedication of additional key staff; projects only receive funding on approval of previous accounts. This is bound to cause some delay if projects are implemented punctually and receive no pre-payrnent. We were not able to ascertain exactly how many account reports are outstanding, but note that a trimester system could double the number of such delays. The financial management response from APOC when asked about reasons for the shift to trimestral reporting was that some projects do not report onfunding use prop- erly under the existing arrangement and they often used funds for purposes not agreed in the initial project plan. Further- more, many of these projects do not, in practice, come back for the second annual tranche offunds. While the integrity of the financial record is important, the accounting system to improve project management is gener- ally not best practice. If projects are not spending at high enough rates or against the budgets initially agreed, the reasons for this should be identified. Problems of this kind will likely get worse rather than better as the number and complexity of projects grow. APOC is aware of the need to improve tech- nical reporting from the field of activities and outputs, and seeking to address it. However, putting this in place will have important implications for the types and volume of resource needed in the field and Ouagadougou. 7ÇF'i'd' \f ,, .t i, l:T :'l; i 'uollEulruqê pue Io4uoJ sIseIJ -raJoqJuo ol uollBler uI BpueSe ]ueilnl repu e roJ dllede) pel1url e dluo se Bel srql'spaford;o raqurnu a8rel e Jo uoq -eluaualdul perlp o+ slueruennbar prc -ueuupue e llerlslurupe aq1 Sur4euepun uo pesnroJ dlpdmurrd sI ]eue]anes er{} ]eq+ salErIpul eroJarsq] aJuepl^e eqJ ]eÆI IEuoIlEu 3q+ ueq+ reqlBr Ie^aI +lulsIp aq+ ]E s]lErluol roJ sr pelueuraldu-rr eq o] Iepour eq+ JI 'peoplro^ eq] eJnper ol papadxa +ou sI Suryodar paseq-+lulslp o] uolllsu€rl V 'ruad e rno; ol o^ + ruor; selqnop d;uanba4 Euryodar aq] sE dlluecr;ru8ls eseerJul ol papadxa sI peoplro ^ sIqJ'speoplro ^ daeaq papodar JJEIs )odv 'spa[ord;o 'luaura8uueu IEIJuEuIJ aq1 dlprcadsa 'luaura8eueru ar{}ut pesnJoJ ere saf,rnosar uerunq s,f,Odv ]o {1nq aql q8noql uarrg 'eJueraJer Jo sure] req] un{I,nn, ser}r,rtpeJo sreqrunu aSreldraa aq1 sserppu o1 alenbape are eser{} aes o} preq sr 1l're e^ oH'slun s,366ygo due Jo sseu -a^lpa#e eqlJo uollenle a uE 1ou sr yodar srqJ'slrun ileu-ts dlarulelar ere [r\[ pue OCS 'ro]lerc er{}Jo a3-IJJo eq} q}yv\ }sBr}uoJ ul 'ernl1nr eq] eBueqr o1 3ur{r1 sr ropeIg eq} q8noq11e artpurd uI pelrul sr dlrroqlne ;o uorleSalap lut{} se}BJIpq's}tun raq}o puE rolJeIIC 3ql Jo aJ-IJJO eql uêeff\}eq salor Jo uotlelldnp eq1 qlllvr paldnot 'suorsr)ep 1aaa1-q8rq 3ur1e1 aa]]nuruo) luaura8eueur 8uor1s € Jo {JeI aqJ '1ad uotssursuer] Jo uol] -dnrralur spJB^ o] suollue^relul reql Jo ssarSord aq1elenle a pue rolluoru o] salr] -unor;o dlrreder pctSoloulolue plnq pue ol aruepmS Eurprlord uI elor aruyoddns s,E^f, pue:seere Jlruepua ut srualsds IJCf, alqeurelsns-]les pue anrpa;;a'aldurls l-{sl -qelse o] selrlunoJ requaw yoddns o1 aarpalqo s,COS elBJIIdnp o1 sruaes sIr{I 'd-rlunoc-ur ;;els JodV IIe sa8eueur dlpalp pue Surplrnq-dlnedet pue Suturerl saas -r3^o'serrlunoJ Jrluspue uI sreploqe{B}s raqlo puE sel4slultu qlm suollBIer roJ dlrroqlne sulelar 1l 'slrun reqlo er{} Jo asoql sdepa^o pue 'peorq d-rarr sI roperlg êq] Jo arIJJo 3q] Jo slor eq] 'ra^e^ oH 'spalord IEUoI]eu o] punJ ]snrJ eq] ruorJ spunJJo ^ oU aq1 a8eueu o1 pau8rsap dlpuoqer sI asIMe{I ernpnr}s FIIUBUIJ aq1 dlrepurs 'oàJV OHM o] pue cvt eq+ o+ rolrarrq aq1 q8norql alqelunorre dlra -dord sr pue ')odv Jo a]€pueru eql IIJInJ o1 pau8rsap dlleuoller sI ]euelarles f,odv eqlJo arnpnrls luauaSeueu ]uerrnJ eql atnpn4s luawaîDuout uo uolsnPuo) '8'z'z drqod ]uerrnr s,OHM r{}u auq ur e^r]€Ilrul druaredsuerJ plv 1euolleu -ralul et{} o} dn pau8ts }ou ser-l pue dc11od ernsolJsrp elep e e^Eq ]ou seop f,odv '{Vt ar{} ro oH^ uror} luauaar8e rtg:rrads +noqllm serrlunoJ Jo JIBqaq uo >1eads o1 areld rraql ro elor req+ sI sil-tr] {un{} },uop daql asneraq uolleruroJul roJ selrlunoJ aq] o] uraql sreler lodv'slsanbar rauyed Cvt-uou rog'+I ro; Euqse rauyed er{} o} lno puas pue lsanbar uo +no UIs uaq+ IIr^ ]EIrBlerJes eq+ qJlqff\ uollEturoJul o+ sseJJe seq cvt eqlJo requaur due 1eq1 pres JJBIS )OdV'Surpodar ]noqe pe{se uaq^ ' ^ereà plSoloru;apldg OH171 eq] ur leJrpoued pue spodar {Vt }o }xe}uo) aqlur pesEelar 3rB sllnser slsEllreJoqJuo 'palepdn ueeq ]ou e^eq ]nq 9167 ur dn ]as are^/\ slunoJJE sqnlnoÀ puE {ooqaJEJ 'reprlvrJ'8002luor; aq ol sreadde elsqe^ . eq+ uo Elep ]uerer +soru aq1 '(a11sqaat uato sU e pq ]ou sêop +t) atts ôU Ogm er{} uo a8edqaar e ser{ pue JVt aq1 o1 syodar 364y synsat [o uollDluasatd' atnsolcsqp D loP t uollo )lunwuo )' l' z' z 2.3. Implications for management and costs of expanding the scope of the programme F:ern the fetrr,;_ OI BClgB.nçg i Comment on the staffing and management impiications of expanding the scope of the progmm if it is restructured into a new ent§ beyond 2015. S.Comment onthe implicationslor management and costs if the soope of the work îs to be expanded to address fl LF as well as River Blindness (country numbet increasesfrom gtto 34,two dlseasesto be addressed using common platforms) or z) all 5 prenentable llllfDs (country number increases from 3t to 47, more than one d,elivery platlorm required): > Assess worldorce implication of paradigm change (control to elimination); ) Identiÿ changes needed to the management structure to meet the needs of elimination of onchocerciasis and LF; > Comment onthe abüÿ of the APOC secretariat to change in response to the expanded scope of wor§ and the processes required to facilitate this. In setting out our approach,we statedthat our starting point for this management review is that the design of an Organiza- tion should be based on the purpose for which it has been established and the chailenges it exists to overcome. These, in turn, should inform its objectives, activi- ties and the structures put in place to achieve them. We note that, from the analysis of the current situation, APOC Secretariat appears to be principally focused on the administrative and financial require- ments for directing projects, which leaves only limited capaciÿ for a wider current agenda. This has important implications for a future Organization's managerial capacity to address the proposed chal- lenges, which are larger in both scale and in scope. 2.3.1. Addressing the cumulative challenges The adaptations required for APOC to successfully expand, shift, transform or develop in ways that ensure it remains fit for purpose in a changing enüronment can be considered cumulative,each phase building onthe last (Figure 4). Each phase will challenge APOC to make manage- ment, human resource and budgeting adaptations. At each level these will need to build on incremental changes made to deliver the previous phase. Figure 4: lncremental adaptations to meet the requirements of PENDA 2.3.2. Possible implications of incremental changes This section considers in detail the possible implications of meeting the incremental changes required to meet the challenges of PENDA. It would not be appropriate forthe consult- ants to offer a definitive statement of the adaptations, management changes and workforce changes required. This needs to be developed on the basis ofthe Stra- tegic Plan of Action for PENDA, in consul- tation with a wide range of stakeholders, critical those in country leadership roles. Institutional and partnership arange- ments around the expanding mandate that will effectively close dornm APOC and open up a new sort of structure needs to be established on the basis of extensive consultation within APOC and within the other relevant bodies including NTD communities (especially LF) and country health authorities. A new Organizationwill require increased management and human resource provision in some areas, and decreased resources in others. The following table of adaptations, management and workforce requirements is therefore highly indica- tive and is presented as an illustration of the sorts of management and institu- tional adaptations likely to be required based on experience elsewhere. Table 5: lndicative Management Transformations required for PENDA Oncho Elimination Readiness to innovate espe- cially around drug regimens, working in hypo-endemic environments; strengthening surveillance. Shift from the ,,endemic zone" approach to district-level working and from focused projects to country-led national health systems streng- thening. . lnnovation management and investment in building a learning Organization. . Risk identification and management. . Technical support to national NTD planning, budgeting and implementation structures. . Funding transfers to national, provincial or district programmes. . Less financial management of projects, as these are phased out. . More focus on science for elimination and management of elimination including surveillance, training, capacity building. economic analysis, monitoring and evaluation. Sustained or increased capacity for conducting operational research, or engagement with external researchers. Sustained or strengthened provision of contracted technical support especially for: . ln-country skills for entomology, e pidem iolo gy, esp. for elimination . Economic analysis, health system strengthening, strategy and programme development and aid effectiveness . Community mobilisation . Disease surveillance Reduced project admin and finance: - An lncreased programme support in - Programme strategy and coordination - Planning and budget, - Results monitoring and evaluation. iI t I Table 5: lndicative Management Transformations required for PENDA (continueed) Joint Oncho-LF programme in 34 countries 5 PC.NTD programme in 47 countries . Allthe adap- tations set out above AND adapt techni- cal approach to integrate effectively with LF. . Consider whe- ther and how to operatio- nalise support to new areas such as morbi- dity manage- ment. . Provide ex- panded reach to increase countries from 3l to34. . lnvest in buil- ding national leadership and commitment Allthe adaptations above and: . Expanded approach to NTDs intro- duces many new partners (and new medicines/ chemothera- pies); . Countries expand to 47 . Mixed approaches to control and elimination, prevention and morbidity management required . More and better coor- dination with WHO NTD programme directorates (AFRO and Geneva). . Forge and manage par- tnership across oncho and LF community in- country and globally. . Ensure outward-facing harmonisation of WHO HQ-WHOAFRO-APOC technical and mana- gement approaches to LFIOncho. . Build a common approach to country and community enga- gement between LF and Oncho. . Shift funding to support joint pro- gramming based on national/ sub-national plans. . Build skills in aid effectiveness, funding modalities, national and district health sys- tems strengthening. . Forge and manage partnership across PCT-NTDs which will require PENDA to be: - Responsive to a wide range of partners - Open and transparent reporting - Operationally efficient - Wider engagement with NTDs and preventable disease communities. . lnvest in communi- cations, result- based management systems, budget transparency, new funding modali- ties and more surveil- lance at country level. . lncreased staffand technical assistance capacity in: morbidity management health systems strengthening (supply side, service delivery as well as community mobilisation side). . Merged LF and oncho human resourceS distributed very differently across countries, institutions and NGDOs. . lncreased joined up working with WHO AFRO and HQ around mandate, technical capacity, deployment of human resources. . Training for HR across the LF and onchocerciasis commu- nities to build consistency in approach and unity of purpose. . lncreased capacity in partnership management including communications, results monitoring, evaluation. . More technical assistance for: Planning Health systems strengthening - Community engagement - Economic analysis . Much more joined up wor- king with countries around building single NTD plans and supporting countries to take the lead in developing and implementing these; . PENDA will have to bring together several different communities of vertical disease capacity and build a functional, joined-up approach: much expertise will be required in forging institutional arrangements, negotiating plans, integra- tion of strategies in ways that builds something stronger out 5 disparate and distincti- vely branded groups. While the above table is only indicative, it shows the extent of the transformation that may be required of PENDA. These cumulative incremental changes go beyond a minor or even moderate adjust- ment of APOC. They will require major changes to the governance, functions, skills and systems as weII as its formal management structure, bringing together existing elements, whether from APOC or elsewhere, but also identiÿing new ways of working. We call this change a transformation because of the large difference between APOC Secretariat's current operation, with its resources principally focused on the centralised management and control of projects, from what may be required by PENDA in the future. A future PENDAs management system will not be evolved from APOC by adding- on, or cutting-back.lt will requlre a deeper analysis of the challenges required for a transformed Organization. The next section looks at the key challenges that a transformed management system will be likely to address. 2.3.3. Key challenges for transformation Our understanding of these challenges is based on discussions with a wide range of partners and stakeholders, the Concept Note and the Strategic Action Plan for PENDA. Broadly, this review indicates that the transformation required by the much greater scope anticipated in PENDAs emerging mandate will require three fundamental challenges to be addressed and are thus the challenges that PENDAs management systems and structure will need to address: Challenge 1: The changing nature of the partnership PENDA wiII engage with a larger, more complex partnership, globally, re gionally and in-country, than at present. It may be Iess linear or structured than at present, andwill involve manymore partners. The partnership will be founded on country capacity and leadership. It will need a fresh approach to communication and openness to function efficiently with a much broader partnership base, including greater transparency. Challenge 2: An evolving model of funding and operations PENDA will need to move from project support to another model incorporating results-based management and funding methods, with a stronger focus on enabling than operational management, which is already becoming unsustainable. To meetthe requirements of elimination, PENDA will have to help countries and health systems drive down cost and over- come barriers to a more widespread and sustainable programme than has been achieved by the project model. PENDAs ambitious agenda makes it urgent to evolve alternative programming models especially in large and/or well-supported countries. Challenge 3: An evolving leadership based on evidence-led protocols, knowledge management and improved transparency PENDAs leadership will come from building on APOC's comparative advan- tages for the NTD community. Based on APOC's past and current performance, PENDAs leadership maybe around advo- cacy, community-based programming, identifying best practice, attracting resources, maintaining the profile of NTDs, and contributing to health systems strengthening. In a newly framed Organ- ization, PENDA will also support coun- tries to lead, and build an open forum for scientific discussion and best practice in treatment protocols. This is a facilitating role that will require a capacity in APOC/ PENDAto support countries in a different way.It will also require PENDA to pass some of APOC's current roles to others.

IIn this widened community PENDA will be a strategic partner in the control and elimination of NTDs, complementing the growing capacities of other partners, helping to solve operational and science- based problems, and doing only what PENDA is uniquely placed to do. 2. 3.4. Implications for manag ement structure and costs Because the new Organization will be the result of transformation, rather than adjustments, of APOC, it would be a grave mistake to propose orto endorse any struc- tural arrangement for PENDA without a clear commitment to the transforma- tion required. The structure of PENDA will depend on its response to strategic chal- lenges such as those we have posed above. The predominance of administration and finance in APOC means that the future arrangements for programme financing implementation will be the major cost driver. Although the Concept Plan and Strategic Action Plan for PENDA give a good indication of the challenges that a new Organization will seek to address and its ambition, neither document adequately explains how the burden of project management will be reduced to ensure the organization continues to be able to make financial grants in a cost effective and efficient way and to make room for new strategic priorities.lf PENDA seeks to fund and control implementation of a large number of local projects forthe elimination of oncho, [F and several other PC-NTDs, it can expect to face very high and costly requirements for management, financial administration and in-country oversight or technical assistance. In order to establish the human and finan- cial resources of a transformed Organiza- tion, the strategic action plan will need to be used to form an operational plan. This will showthe activities to be undertaken, for which staffing and financial require- ments can be calculated.In the process of establishing the activities required to achieve PENDAs objectives, the require- ments for coordination will provide the basis for identiÿing an effective structure. 2.3.5. Conclusion on implications for management and costs Because of the extensive transformation required for the formation of PENDA, it is our judgement that a future PENDAs management system should not be evolved from APOC by simply adding-on, cutting-back or re-organising. In order to deliver the much more ambitious agenda set out in the draft PENDA document, it wiil require a deeper and highly partici patory engagement in the analysis and design of the transformed Organizationto meetthe challenges set out inthe Concept Note and Strategic Action Plan for PENDA. This will require decisions on the opera- tional model for PENDA, particularly on financing and direction of operations. The existing project model is likely to be unsustainable economically and mana- gerially, as well as at odds with the basic principle of the Concept Note that country ownership and leadership are of para- mount importance. Once these decisions have been taken, it should be possible to develop an opera- tional action plan for PENDA, identifying both the human and financial resources required. This in turn will provide the essential datafor creating a management structure for a new PENDA. 2.4. Benchmarks Ef pm-t:b e lçrmspI-Beferenc e : 4. Benchmark these findings against approprtate management standards where possible.In partianlar to assess the extent to which the procedures follow approprtak standards and, if not, how this mightbe remedied. 2.4.1. Methodology for e stablishing benchmarhs Benchmarking is a process that aims to plot management capacity against a clearly defined range. The process of benchmarking can be helpful for a number of reasons: . building consensus around desirable qualities and behaviours; . tracking progress towards improvement; . focusing capac§building efforts; . supportingtransparenry. In this case, the provisional rankings offered here may help APOC to form consensus around the desirable qualities and behaviours required by PENDA, and therefore provide a useful contribution to testing proposed Organizational designs. In order to undertake the benchmarking exercise below, three established bench- marking processes were consulted and used as a basis for developing appropriate qualities and behavlours suitable to APOC and the kind of Organization it is. These processes were: . The Multilateral Organization Perfor- mance Assessment Network (MOPAN) Common Approach+: a review of multilateral Organizations conducted regularly by 18 development agencies 4 The MOPAN Common Approach is explained here: www.mopanonline.org . The most recent review of WHO was in 2013. The Annex for that review is here: http://www.mopanonline.orglupload/documents/ MOPAN_201 3-_WHO_Vol._ll.pdf . The United Nation's Results-Based Management Guidelines and Handbook5; . The UK Department for Intemational Development Multilateral Aid Reüew (MAR), 2011 and update in 2013 6. From these assessments and frameworks, indicators were identified that would provide relevant benchmarklng in APOC. The indicators were grouped into five thematic areas that have been identified as critical markers for successfullytaking on future challenges. These are: 1. success in Partnership; 2. science-Ied, evidenced based decision-making; 3. building an innovation and learning culture; 4. adaptability and responsiveness; 5. value for money. The selection of benchmarks and specific lndicators as well as the process of posi- tioning performance on the scale is not an exact science. These indicators were chosen because they seemed appropriate to the circumstances and challenges faced byAPOC inthe comingyears as judged at the time of assessment. 5 Available here: http://www.undg.orglcontent/ programming_reference_guide_(undaf)/un_country_ programming_principles/results-based_management. 6 The MAR update can be found here: https://www.gov.uk/ government/uploads/system/uploads/attachment-data/ filel297523lMAR-review-decl3.pdf . The WHO updated assessment is on page l8l-2. Benchmarking indicators start on page 9Z 2.4. 2. Benchmarks : ind.icators and assessments The table below sets out the indicator with an appraisal of where APOC scores in 2014. Comments give the context and direction of travel expected in the next couple of years. They were all identified during the research phase as particularly important competences for the future success of an expanded and more complex partnership z. 7 These are our choices for the best indicators but it is worth noting that there are many indicators that would be suitable. Table 6: Benchmarking management capacity in APOC as of 2014 - indicators and assessment i 1. APOC is a partner focused Organization with a clear mandate, widely understood goal, and strongly articulated objectives 2. APOC's senior J management shows leadership on mana- ging for results right through the Organi- zation including at country level 3.APOC is commit- J ted to transparency in data disclosure. Examples would be: signatory to the lnter- national Aid Trans- parency lnitiative (lATl) and is actively participatings; or a regularly updated website accessible to all containing all data relevant to APOC. APOC scores highly for this indicator. lt is a one-stop shop for onchocerciasis, has the advantage of clarity of mandate and is a well-established, mature partnership based in Africa and building African country capa- city. The challenge will be (a) in maintaining clarity of purpose as the mandate changes with PENDA and (b) serving countries wit- hin the partnership in a way they best need in the future rather than continuing in the current mould whether suitable or not. APOC has many senior staff who are tech- nically competent, dedicated and focused. Senior staff should work as a management team to ensure that each department has clear results commitments in relation to the Organizations annual/ biennial results framework and each individual staff mem- ber has clearly articulated objectives and performance indicators that contribute to this framework. APOC currently releases data on request to JAF or CSA partners, including countries. APOC does not publish its results (inclu- ding project by project results, financing data etc) in a simple webpage linked to other WHO NTD sites such as the Global Health Observatory (http://apps.who.int/ gho/data/node.ma in.A1 629NTD?lang=e61. A more open communication strategy and better joined-up working with other NTD leaders will be an important part of successfully managing the shift to PENDA. There are many ways that results can be released that do not overstep the bounds of country sovereignty and ownership: other Organizations grapple with these issues all the time (eg GAVI, the Global Fund for AIDS, TB and Malaria etc) and there are good examples in WHO. J I 4. Programming is developed using the best available evidence taking account of technical, economic, financial and implementation evidence and where necessary, operatio- nal research supports improved decision- making. 5. Evidence from multiple sources is considered in decision-making and decisions are open, justified and based on consensus amongst partners. 6. APOC is geared towards supporting countries to play a leading role in building, using and implementing evi- dence in their natio- nal programmes. J J While APOC is fairly strong in this area as regards the use of entomology and epidemiology to determine project sites, it could strengthen further by incorporating better economic analysis that would help improve evidence about the cost-effectÈ veness of one approach over another. For example, does the full economic cost of elimination over 8 years rather than 14 or 20 years justify increasing to two or more MDA distributions per year? Under what circumstances is doxycycline cost-effec- tive? How can countries build their own capacity to undertake surveillance and mapping? Putting science at the heart of future decision-making does not neces- sarily mean increasing in-house research, but may mean strategic partnership with others undertaking research, through involvement in research strategy, making data available, supporting trials etc. There is some evidence that the decisions taken within the APOC partnership may be overly dominated by the relationship with the Special Programme for Research and Training in Tropical Diseases (TDR). APOC should take the lead in ensuring that its advice and guidelines for detection, mapping, treatment match up closely with those of WHO HQ and AFRO. APOC currently operates a fairly top-down delivery system, reinforced by centralised management within APOC itself. lt has good partnerships with countries but is less effective where countries take the lead themselves with other partners. APOC should become better able to support genuine country leadership: that means finding alternative ways to channel re- sources to countries (beyond projects) and participating in elimination programmes as a supportive partner rather than a leading one. PENDA will need new modalities to guide its engagement with different kinds of countries. 7. Are systems in J place to effectively measure results? Does it have a com- prehensive results framework that covers the whole of the results chain from inputs through to impact? 8. Operational research is relevant and can be clearly linked to improving the implementation of effective pro- grammes 9. ls the Organization J always striving to improve its approach and adapt it to new circumstances, evidence and country needs? 10. APOC is able to lead from the front where necessary but also take a suppor- ting or advisory role where countries are leading their own programmes? J APOC does not use a comprehensive results framework that links inputs to out- puts, outcomes, and impact with timelines and annual evaluations. A shift to results- based management (underway across WHO) will support better planning and accountability, improved use of available funds, clear prioritisation of resources and accountability for results. lt will improve APOC's management credibility amongst partners and help sustain interest and wil- lingness to fund onchocerciasis and other NTD programmes through APOC. APOC operational research programmes are already well established and are cited as a strength. lmproved communication about the research agenda, open dialogue about what research is to be done and the costs will improve performance further. Although the Technical Coordination Committee meets regularly, there has been little change in protocol in the recent past despite challenging evidence that different regimes may be more cost effective and efficient in the longer term. APOC should seek to be at the cutting edge of operatio- nal research to design better, cheaper and more cost effective protocols to achieve elimination rather than demonstrate conservatism around its current approach. APOC is well structured and highly functional as a command and control operation. lt can successfully articulate and deliver programmes in many diverse settings in cooperation with countries. lt seems less comfortable taking a backseat and allowing others to lead. ln the future, PENDA will need to be able to verify technical standards but not always take the operational lead on elimination efforts. To be sustainable and to deliver a five-di- sease mandate, it has to move away from a top-down approach. This will require training and capacity building as well as new systems. 11. Delegation of operational decision- making and manage- ment responsibility promotes task-shif- ting, accountabi- lity for results and efficiency. 12. Do its financial systems give it the flexibility to use the right instruments in the right situations (e.9. in both mature programmes and in fragile states)? 13. Cost per person treated &/or cost per DALYe saved is routi- nely assessed in each project/ country? 14. Does APOC strive for economy in pro- curement and have targets for reducing administrative and implementation costs? 15. Does it have strong policies and processes for finan- cial accountability (risk management, anti-corruption, qua- lity of external audits, fiduciary risk)? J It is clear that APOC leadership is trying to build delegated decision-making. ln a setting where decisions have always been taken or approved in a centralised way, it takes time to make this sort of change. Continuing efforts, mentoring and support will all help strengthen delegated decision- making. ln principle, WHO systems can allow for a range of different financial instruments. ln practice, these are not always used to their maximum effect. As the oncho-LF effort moves towards country-led national programmes, APOC will have to shift away from projects and it will be challenged to find ways to maintain excellent financial accountability (see below) in these new contexts. Cost-effectiveness benchmarkin g: Oncho- cerciasis treatment is US$ 37l per DALY saved; Cost of treatment is US§ 0.45'0. By comparison: LF annual MDA is USS 15/ DALY and cost per treatment ranged from USS 0.40 to USS 5.87'r; Albendazole treated helminthic infections USS3/ DALY saved; Average PC-NTD treatment cost is USS 0.49 to USS 0.51 per treatmentl2. APOC demonstrates some economy: travel policies, for example, mean that costs of meetings are shared with partners and all staff travel economy class. This year's budget has been trimmed to match avai- lable resources but it is not clear what has been cut. APOC could improve its manage- ment of cost savings by being clear about where it will save costs, setting savings targets and engaging staff in helping to reduce unnecessary expenditure. lt would be especially admirable if APOC could demonstrate how it will reduce costs in a way that preserves expenditure on priority results. APOC, as a programme of WHO, has well established strengths in this area. Partners have a high degree of confidence in APOC's financial management and accountability for funds. Audits have been generally free from qualifying statements and have pro- vided helpful advice to improving trans- parency in the presentation of accounts. APOC's challenge will be to maintain this level of accountability as it moves to new funding modalities. J 2.4. 3. Conclusions on b enchmarks The benchmarks were chosento reflect a range of competencies that APOC should demonstrate in order to take on the chal- lenges of its expanded mandate. The research indicated that APOC demon- strates clear strengths especially in success in partnerships and in value for money. APOC demonstrates strengths but also challenges in relation to some aspects of science-led, evidence-based decision-making; building an innovation and learning culture; and adaptabil§ and responsiveness. I I I 8 The lATl Registry (www.iatiregistry.org) is an index of data published on international development activities. The registry itself stores no data, but provides a searchable index of metadata, feeds, and links to datasets hosted by donor agencies, development Organizations and partner countries. Organizations (currently 240, including bilaterals, multilaterals and others) publish lATl data onto their own websites. The lATl registry provides a convenient single point ofaccess for users to find and stay current with these different sources of information. 9 DALY5 are a measure of years lost to death and disability caused by disease or other causes WHO benchmarks DALYs against GDP per capita:'highly cost-effective' if cost per DALY is less than GDP per capita;'cost-effective' if between one and three times GDP per capita; and 'not cost-effective'if more than three times GDP per capita. ln 2010 (World Bank data) the GDP per capita was around S1,150 in low-income countries and 52,050 in 5ub-Saharan Africa. 10 This data comes from Disease Control Priorities 2 (www. dcp2.org) unless referenced otherwise. 1l Economic costsfrom Accelerating Workto Overcome the Global lmpact of Neglected Tropical Diseases: A Roadmap for implementation; WHO, Geneva 2012. 12 Towards an lnvestment Casefor NeglectedTropical Diseases Including new Analysis ofthe cost of lntervening against Preventable NTDs in Sub-Saharan Africa; Anthony Seddoh, Adiele Onyeze, John Owusu Gyapong, Janet Holt, Donald Bundy. Lancet Commission on lnvesting in Health, July 2013. 2.5. Strengths and Weaknesses Opportunities and Threats From the Terms oJ Reference : 5. Summarize the strengths and weaknesses of the APOC Secretariat and. provid.e compartson with estabtished good practice. Based on the above assessment, a review of available documents, interviews with stakeholders and our own observa- tions, we propose a summary of APOC's strengths and weaknesses here. 2.5.1. APOC secretariat strengths The APOC Secretariat has a number of strengths that have made it successful as a single disease controlprogramme oper- ating in difficult to access places across a range of countries. These strengths are listed below. They are considerable and their achievements should not be put at risk during the transition process. What they amount to is an Organization that has "fought its corner" during a 15 year period when much donor funding became concentrated on global mecha- nisms to combat a fairly narrow range of prevalent health challenges identified through the Millennium Development Goals. Combined, these strengths make a sound foundation for APOC to evolve towards the kind of Organization that will meet the challenges posed for PENDA. . Well-established, mature, capable and well functioning Organization. . A knowledge-led, experienced, dedi- cated and hardworking team across all three clearÿ structured departments. . Evidence of successful large-scale delivery through CDTI providing good value for money. . Strong internal management means that each individual has a clear under- standing of his orherrole andfunction (and limits of responsibility). Although the decision-chain is a little cumber- some, it nonetheless works at a reason- able pace. Manages partnership successfully (20+ years and still going strong):. plays a convening role andis strong onbringing onchocerciasis partners together; has donor confidence - including that of Mectizan Donation Programme - based on high delivery rate of drugs to people and contlnuingly growing results. Successful fund-raising capac§ from a wide-range of donors including uncon- ventional ones sustained over time. As a \MHO programme, APOC is widely accepted by countries as a legitimate partner able to challenge their perfor- mance;APOC gains access to political authorities and has the necessary status to be able to raise difficult questions. Countries have a strong Ievel of confi- dence that \MHO - and its programmes like APOC - are not going to walk away as a result of lack of funds or altemative interests; has a remit to facilitate cross- border issues and programmes. Evidence-based decision-making: on the whole, APOC programmes are led by science and have been since the beginning. Created and evolved the community directed approach and has long expe- rience of this modality, including its potential and limitations. Is very stronglyembedded inthe region and has good relationships with coun- tries. Trusted as a home-growrr effort, APOC is anAfrican Organization estab- lished to support African countries to address an African problem. . Is technically strong and has substantial capacity in entomology, disease mapping, epidemiology and surveillance. . Has developed a goodtrack-record on training - formal and informal, directly in countries and through bursaries. . Interesting and useful peer review processes to support country-based advisers to visit and support each other (eg among financial officers; among country coordinators). 2. 5. 2. APOC s e cretariat we akness e s APOC's approach has resulted in some notable faults. These can and should be addressed as it moves forward with the expanded mandate. Furthermore, as aid modalities develop over time and coun- tries become more confident and assertive about leading their development, APOC will be expected to become more flexible in its approach to working with countries, to remain relevant and useful. . Top down approach to management and decision-making, within the secre- tariat, with APOC staff in-country and with national programmes.l3 . Directive rather than facilitative of onchocerciasis control. . No comprehensive results framework so it is not possible to link investments with outputs, outcomes and results. Weak data disclosure policy/practice and insufficient opennes s /transparenry about programmes and results. Unclear knowledge management poliry and process: APOC appears to have diffi- culty resolving views about the way to treat onchocerciasis in different settings. There is insufficient knowledge about l3 lt is relevantto notethatthe currentdirector is already working to shift to delegated and distributed decisio n-making. what will be the most cost-effective approach overtimela andno clear option for resolving these outstanding ques- tions in a way that all partners agree and accept. APOC appears to be slow in responding to new evidence treatment and preven- tion: whether the evidence is sound or practical to implement, APOC has an important role in opening up debate and being seen to be examining ideas and evidence very carefuIly and trans- parently; and to evaluate the potential for change in management of NTDs in a form/ format that all stakeholders can participate in and agree. Insufficient integration with the broader NTD commun§ including other \AIHO NTD programmes. 14 For example, there is no cost-effectiveness analysis that demonstrates the value of treating with ivermectin once a year rather than two or three times: while input costs are higher for the latter, the full economic benefits (faster elimination) which would consider the benefits to communities in economic terms as well, have not been analyzed. Another outstanding question is the circumstances under which doxycycline should be used for treatment ofadult infections (outside Loa Loa Infection areas). The fact ofthese apparently outstanding questions and their lack of resolution within the APOC community (according to several of those interviewed) is the issue, not the substance ofthe questions themselves. 3. Overall conclusions The challenge APOC remains a professionally staffed and effective Organization, but faces major new challenges. These will requlre a trans- formatron of its management to fulfil new functions reflecting the new purpose of the proposed PENDA and new ways of working to support country leadership. The appointment of a new Corporate Services Manager, announced just as this review was approaching completion, and further promised reforms of the structure are welcome pre-conditions for such a transformation. They have potential to equip APOC to address the challenge of transformation, as set out in the recom- mendations below. While therefore, they may be seen as appropriate to prepare APOC for change, they should not be seen as any rndication of the future staff or resources required by PENDA. The management choices for the future depend fundamentally - as do other elements of the new Organization - on the strategic choices made for PENDA. There is a clear tension emerging in the onchocerciasis strategy as it moves from control to ellmination. The PENDA concept note states that country ownership and leadership is of paramount impor- tance. This requires a community-based strategy and health systems strength- ening, whereas oncho ellmination may appearto require a command and control approach based on top down delivery, more similar to APOC's historic approach. As more diseases are merged into a unified programme and as the number of coun- trles expands, it will become increasingly untenable (if it isn't already) to continue with a top down model based on projects and where APOC is the centre of the effort. It is vital, therefore, ln light of the expanding mandate, that APOC's manage- ment structures, systems and resources reflect rts strategic approach to working with partners, whether through mobi- Iisatron of resources, research, advocacy, implementation. Three over-ridlng management challenges emerged from the evidence and analysls presented in this report: . how to meet the challenge posed by the changing nature of the partnership; . the urgent need to evolve a model deliv- ering expanded, integrated and results- based implementation, in the context of a shift from projects to country-led, joined-up NTD programmes; . an evolving leadership based on evidence-led protocols, knowl- edge management and rmproved transparency. The context It is an excellent time to make these changes. NTD support continues to benefit from steadily increasing awareness, commitment and support (underpinned by large drug donations). Prevention and the options for reducing morbidity are recognised for their contribution to reducing the social and economic determi- nants of extreme poverty,that also involve land use, access to water, education, social stigma, disability, and the diseases of neglected people. However, the window of opportunlty ls always under pressure as overall aid funding shrinks and other priorities surface, notably,the rising burden of NCDs. The global economic downturn has put pressure on all aspects of development and the new post-2015 development goals to follow the MDGs will harness much of the available funding for the foreseeable future. As an Organization, APOC (and, in due course, PENDA) needs to demonstrate its comparative advantage and value for money much more effectively in order to remain at the forefront of the NTD elimi- nation effort. Politically, it will become more and more difficult to raise funds and sustain interest and commitment the closer to elimina- tion countries get. As local communities "forget" what a disease looks like, they are harder to mobilise especially - as the polio eradication effort demonstrates - in a context where these communities are unable to access the other services for which they feel a strong need. Iver- mectin and albendazole have the signifi- cant advantage of being the treatment of choice for a wide range of irritating condi- tions including intestinal worms, scabies and so on which communities continue to value even after the more sinister condi- tions are largely eliminated. But nonethe- less, from a cost effectiveness perspective, it is vital that NTD prophylaxis is deliv- ered in awaythat maintains commitment until elimination is achieved, in as short a time as possible. Joining up with other service delivery programmes to embed NTD prophylaxis in abasket of services is one strategy. There are others, of course, andthe over-arching challenge is to opti mize interventions and drive down costs in ways that will achieve optimal health outcomes across the greatest population covenge and disease profile. Towards new modalities One of the outstanding features of APOC's functions identified in section 2.2.4above is how many APOC leads on or is almost exclusively responsible for. Not only does APOC lead on objective setting, strategy and policy-making, it also Ieads (with others) on planning, implementation, vectors control, mapping and financial management and then, accountability, monitoring and evaluation. Alongside, it aims to do operational and some basic research, partnership building, advocacy, results management - the list goes on. While impressive for a relatively small Organization, there is a limit to what can be done effectively and as APOC moves into other diseases, it will not be fully in the lead on all these areas across all diseases covered. In addition, the most robust Organizations have a separation of functions which helps build transpar- ency, keeps them open to challenge and stimulates continuous improvement. In APOC's case, it will be advisable to iden- tiÿ which core functions (comparative advantages) PENDA will embrace and lead on. It is unlikely to be able - financially, logistically, technically - to do everything across 5 diseases and 41 countries. Moving away from projects to country-led systems will certainly be vital to APOC's ability to maintain a meaningful role in programme implementation since under its current approach, it is reaching the limits of capacity: staff are working very hard,the number of projects is increasing, expansion to elimination, and in due course to other diseases, will increase the pressure on project development and complexity. Furthermore, WHO's accounting standards and the rigorous requirements of trimestral reporting, while reassuring on one level, do require high standards forfund management that inevitably cause delays in disbursements and thus the delivery of project outcomes. There is a high risk that by tightening APOC's accounting practices in isolation of other funding partners in country, the pressure will build for NGDOs to step in with funding support to keep programmes going as financing delays caused by insuf- ficient reporting, create operational gaps. APOC's increasingly rigorous accounting may not therefore have the intended impact on better reporting while there are other partners also working in the operational areas and who can step in with other funds. This is a critical issue that needs further evaluation. Another reason to shift away from projects, however, is that they are no longer best practice in aid effectiveness. In most other areas of development, strength- ening aid effectiveness based on Paris Declaration principlesls has put national authorities in a position of implementa- tion leadership. Partners provide support and financing to a national strategy, often pooling funds for use where they are most needed and against an agreed plan and budget. For APOC, shifting to this model of funding would have advantages and risks. The main advantage would be Iess financial management and increas- ingly demanding project administration with consequences for freeing up staff time; the main risk would be ensuring that programme quality and results were maintained. New ways of working should not mean sacrificing value for money. On the contrary a new partnership will need to focus strongly on results, not just on project activities and input costs. APOC develops biennial (and then annual) budgets based on activities that are helpful for identi- l5 The OECD Paris Declaration on Aid Effectiveness (http://www.oecd.orgldac/effectiveness/ parisdeclarationandaccraagendaforaction.htm ) promotes harmonization, country leadership and ownership, mutual accountability, and, critically, alignment by which is meant that donors align behind country systems and strategies. There is a set of principles for working in fragile and post conflict settings as well (http://www.newdeal4peace.org ) endorsed by the United Nations Development Group. fying where its resources go and how staff time will be used. Results-based manage- ment links inputs to outputs, outcomes and impact. It also helps contextualise successes (and failures), assists funders in seeing how much of the total resource envelope they are funding, and allows countries to identify opportunities for improved cost effectiveness. Country leadership also changes the nature of partnership: whereas the current modality suggests that APOC drives delivery of onchocerciasis control programmes with the support of others, a country{ed programme funded through pooled arrangements makes APOC requires different kinds of partnership and leadership skills, in relation to NGDOs as well as government. This shift has fundamental consequences for the management fu nctions of PENDA. In a context where countries have joined up NTD plans, what will be PENDAs role in surveillance, training, project delivery funding, monitoring and evaluation? NTDs have attracted renewed funding commitments in the last few years, linked to ambitious elimination targets but global development efforts aim to build national capacity, country lead- ership, health systems strengthening, universal access. Therefore, the challenge is to deliver results on NTDs while also strengthening country commitment and control. There is an increasing shift away from vertical, single disease programmes: perhaps guinea worm and polio are the only two programmes iikely to retain a single disease focus until eradication. The point here is notto identify an exhaus- tive list or proüde any guidance on tech- nical approaches but rather to call atten- tion to the need - from a management perspective - for APOC now, and PENDA in due course, to be on the front line of supporting the analysis and thinking in each country individually about the options, given its funding, its partner base, its operational challenges and its capacity. To do this, APOC/PENDA requires a broad and flexible set of skills and expertise, including economic analysis, aid effec- tiveness, national policy and planning and others. Transparency and openness Finally, a word about transparency and openness. The consequences of working through country-led modalities are several, including requirements for data disclosure, clearer statements of results, and gener- ally improved transparency and account- ability. Both existing and prospective stakeholders, lncluding countries, donors, NGDOs and researchers, value access to information and consistent, authorita- tive data for scientific and programmatic purposes. Country data belongs first and foremost to countries and APOC has a role in helping to bulld capacityto collect, interpret and use data appropriately. But demonstrating the results of investments and the whole value chain is also APOC's responsibility. Pressure from investors to have concise and reliable results data will increase as the number of diseases - and stakeholders - grows. Maintaining all part- ners' confidence in the value for money achieved by APOC in its work will require an openness to benchmarking, objective scrutiny of results and the regular publica- tion of impact that shows what has been achieved, at what cost and how (together with the drivers of costs). The only sustain- able approach in the long term is to support countries to publish their own data in a common location (like the Global Health Observatory) or by agreement through an annual APOC report. Many interna- tional Organizations face this challenge (GAVI, Global Fund, research Organizations and others) where the baiance between data ownership, partner confidence and public scrutiny can be in competition with one another. Whatever the path chosen, improving access to information will be an important component of building and maintaining commitment to the broader NTD control effort. 4. Recommendations We recommend a staged process for taking forward the management design of PENDA. in summary: Recommendation l proposes that an exercise be undertaken to agree the functions required by PENDA. Recommendation 2 proposes that these should be used to guide the development of an implementation plan, including budget and staffing requirement. Recommendation 3 addresses the concern of stakeholders withthe imme- diate workload of APOC staff. Recommendation 4 addresses the need for a clear timetable to manage the tran- sition process. Recommendation 1.: agree the functions required The management design of PENDA should be driven by a clear understanding of the functions that it will use to undertake its day-to-day tasks. The first step in design should be for the Transition Task Force to agree the functions required for the activities that PENDA will undertake. We provlsionally suggest that four require- ments that also should be considered in specifying PENDAs functions, in the context ofthe PENDA Concept Note and Strategic Action PIan. A. Funding and implementation B. Technical capacity C. Research D. Communicatlons It is essential that there is a broad consensus on the requirements and work- flows of these functions before proposing a new workplan or any new structure to support it. A. The funding and implementation models adopted by PENDA need to be matched to the needs and capacltles of each country with a clear path towards full country Ieadership and management. This should require less Organizational focus on operational and financlal management of projects, and more focus on supporting countries to take the lead in identifying thelr own approach to NTD control and elimination. Examples of change need to lnclude: 1. Where there are several sources of funds at country level coalescing around a national NTD plan and funding mech- anism (making APOC one of several partners engaged in NTDs or even in onchocerciasls control), the partnership in that country should be reframed and funding flows re-considered. These will be considerable changes and should be piloted ln one or two countries first in order to build experience. Transition as soon as possible from project-based to nationally integrated, or pooled funding modalities in countries where these are already established. Supporting country coordinators (APOC programme staff in country) to join country based NTD coordination processes where they exist orto initiate these where they do not. A separate funding modality for donor orphan countries where PENDA may be one of the only funders based on notional pooled funding allocated to different areas in the country. ). These changes will require fundamental changes in PENDAs staffing and skill requirements, decision-making structures, work flow and location of operations. B. PENDA's technical capacity model should drive its provision of technical staffing and capacity in the regional centre and in-country. It needs to ensure that it has both the range and depth to address the key areas required for its tech- nical leadership, and to ensure that coun- tries have the technical support they need. PENDAs approach to technical capacity should complement technical support from other national and international providers. It may include more compre- hensive skills around aid effectiveness, economic analysis and country-focused partnership management. C. PENDAs approach to research should drive the way that PENDA contributes to the development of evidence and knowl- edge for NTD elimination.It should define PENDAs contributlon to the generation of evidence and sclence, reflecting the needs and capacities of countries, the partner- ship and still wider stakeholders. PENDAs contributlon to research is likely to include developing, articulating and communi- cating research requirements from the partnership, facilitating the contribu- tion of evidence from NTD elimination programmes, and supporting the process of putting evidence into practice. PENDA needs to clarify whether or not it is its own role to conduct research, or whether its best contribution will primarily be in advocacy and support for research conducted by dedicated speciahst research bodies. PENDAs strategy for supporting research needs to be published transpar- ently so that it is open to a wide range of stakeholders and considers a range of views about priorities. D. PENDAs communication model shouid be integral to its direct funding, technical support and research. Again, this wil1be a major driver of Organization design. A comprehensive communication strategy updated annually should incorporate the following: 1. Communication objectives to support the overall objectives of PENDA. 2. Strategic choice of channels, whether scientific journals or social media. 5. Use of advocacytools to maintain and develop dialogue on NTDs. Scientific collaboration and transpar- ency, by embedding onchocerciasis results in the WHO Global Health Observatoryl6 and other locations where NTD data is captured.; Aid transparency as promulgated by the International Aid Transparency Initlative, including a comprehensive public statement of income, expendi- ture, outputs and outcomes. Recommendation 2: implementation plan Once the functions of PENDA have been agreed, the proposed partners in PENDA should undertake an indicative opera- tional planning and budgeting exercise, probably over 2-4 years. A. The plan should take into account both the existing commitments of its constituent Organizations, and the future strategy of PENDA. It will thus draw proposed activities from: ) Existing operational plans ofproposed partners in PENDA including APOC and partners supporting LF elimination. ) Activities proposed in the PENDA Strategic Plan. 16 Countriesadd theirown datatotheclobal Health Observatory and APOC/ PENDA could work with each country to identify/ support results data and get these on to the Observatory site. While there are limitations to the datâ on this site, they are at least country owned and reported. 3. B. Activities should be incorporated within the proposed operational plan, taking account of the agreed functions (recom- mendation 1). C. A preliminary budget and staffing requirement should be developed for the proposed operational plan. D. A provisional staffing structure may also be set out at this stage as a basis for discussion and consultation. Recommendation 3: staff workloads This report has notedwidespread concerns about current staffworkloads. These need to be properly identified by managers, and taken into account in planning and managing the last 18 months of APOC's existence. It is a core management task to ensure that staff and other resources are able to carry out the prescribed functions. Where this is not possible, resources either need to be increased, or tasks changed or reduced. In the present case, where funding for 2Ol4/75 remains incomplete, it is important to reduce or change tasks to fit capacity. It is recommended that this process be Ied by the managers in charge of each of the units and sub-units of the secre- tariat, under the overall coordination of the director. Many opportunities were identified to change or reduce the volume of tasks in the short to medium term, including: 1. Delegation of decision-making and budgetary authority, to reduce flows, particularly through the Office of the Director. 2. Increasing involvement of in-country officials and technical assistants in developing proposals, managing, moni- toring and review. 3. Piloting use of national or regional programmes to reduce number of project-related transactions and reports. Recommendation 4: transition timetable Finally we note that this review takes place in the context of a larger change process. The transition process should continue to be monitored closely by aII stakeholders in APOC and its partners. Set an overall timetable that allows for agreement of functions andthe imple- mentation plan to be agreed in time for othertargets required forthe establish- ment of PENDA, including establish- ment of committees, the development of the Investment Case for PENDA, resource mobilisation and staffing the new Organization. Support the TTF that is overseeing the transition to PENDA to report regularly on developments and to provide an updated timetable both to the expanded CSA and to all partners in the JAF with email updates and opportunities for partners and stakeholders to make comments or suggestions. Support the Govemance Working Group (GWG) in clariÿing institutional rela- tionships, roles and responsibilities of APOC, WHO AFRO and \MHO HQ in rela- tion to onchocerciasis and, in due course, LF andthe otherNTDsto be incolporated into PENDA by creating opportunities for discussion and consensus building at key partnership meetings. Ensure that those of APOC's functions that are still required are retained throughout the transition period, and after the establishment of PENDA, for as long as required for the completion of the APOC programme. F4 I

People interviewed and questions Interviewees Mrs Bineta Ba-Diagne, Health Economist, African Development Bank, Burkina Faso Mr Asmani Bizimana, Budget and Finance Officer,APOC Dr Donald A.P. Bundy, Lead Health and Education Specialist and APOC Coordinator, Africa Region Human Development Department, The World Bank Mr. Simon Bush, Director, Neglected Tropical Diseases (NTDs), Sightsavers Prof Yapo Felix Boa, Director General of Health, Ministry of Health, Cote d'Ivoire Prof Moses Bockarie, Director, Centre for Neglected Tropical Diseases, and Professor of Tropical Health Sciences, Liverpool School of Tropical Medicine Mr Xavier Daney, Senior Legal Officer, Office of the Legal Counsel, World Health Organization (WHO/HQ), Geneva Dr Grace Fobi, Chief, Sustainable Drug Distribution Unit John Gibb, Senior Programme Manager, DFID _ UK. Dr Caroline Harper, Chief Executive Officer, Sightsavers Dr Adrian Dennis Hopkins, Director, Mectizan@ Donation Program (MDP) Dr. Chris Lewis, Senior Health Adviser, DFID-UK Dr Georges Alain Etoundi Mballa, Director of Disease Control, Ministry of Public Health, Cameroon Dr Chris Ngenda Mwikisa, Country Analysis and Support, WHO AFRO Prof David Molyneux, Emeritus Professor and Senior Professorial Fellow, Liverpool School of Tropical Medicine Dr Diallo Nouhou, Technical Adviser, APOC, DRC Dr Frank Oliver Richards, MD, Director, River Blindness Elimination and LF Elimination Programs, The Carter Center Dr Neema Rusibamayila Kimambo, Acting Director, Preventive Services, Ministry of Health and Social Welfare, Tanzania Dr Maria Rebollo - Programme Director, Onchocerciasis Operational Research, the NTD Resource Center. Dr Jean-Baptiste Roungou, Director, APOC Prof Sidi Ely Ahmedou, Community Engagement, APOC Dr François Sobela, Health System Specialist Dr Daniel Luhata Shungu, Executive Director, United Front Against Riverblindness (UFAR) Dr Laurent Toe, Biologist, APOC Mr Ibrahim Toure, Administrative Officer, APOC Dr Tony Ukety, NGDO Liaison, WHO Geneva Ms Emily Wainwright, Senior Health Advisor, USAID Dr Laurent Yameogo, Coordinator of the Director's Office, APOC Mr Honorat Gustave Zoure, Biostatistics and Mapping Officer, APOC Inte rview que stionnair e Interviews drew on the following questions as appropriate to the context and person being interviewed: > The onchocerciasis programme has been running for some years now. What in your view are the most important achievements of the programme? > lÂIhat are the strengths of the programme? What are the main strengths of the APOC Secretariat? How could the programme be more efficient ? How could the programme better take advantage of its strengths? > Do you think the strategic plan to shift to elimination appropriate and achievable? What about incorporating tF? Or other NTDs? Is this a realistic plan? For you personally, what will you have to do differently? > And their areas for development? How do you think the APOC Secretariat could build its capacity further? Or become more efficient? Is mandate expansion (elimination, tF, 5 NTDs) something the Secretariat can lead given its current structure and level of performance/ skill? > What does APOC offer country programmes? (Drugs, training, data management, etc)? If the number of countries expands, what are the implications for you, for your role, for the way you interact with countries? What about if the number of diseases changes for prevention programmes (to include LF in some countries? Or, to include several diseases in some countries? > Can you talk about funding and other resource flows - are they adequate, secure, sequenced well? Are resources made available in a way that maximizes reach and efficiency? > What are the likely resource requirements for shifting to elimination or incorporating LF or other disease prevention programmes? Are there ways that data management, report,ing, monitoring could be done better? More efficiently? > How reliable is data from surveillance? What are the key outstanding technical needs that would really accelerate programme delivery and elimination of Oncho? Of tF? Others? Annex 2: Documents and papers consulted and reviewed APOC staff data including all CVs, post description, appointment information. APOC Revised Plan of Action and Budget 2014-2075. Audit letter 2013. Financial Report 2012,2013 World Bank Trust Fund. Report of the External mid-term Evaluation of the African Programme for Onchocerciasis Control, 2000. Report of the External mid-term Evaluation of the African Programme for Onchocerclasis Control, 2005. Report of the External mid-term Evaluation of the African Programme for Onchocerciasis Control, 2010. Concept Note: Transformlng APOC into a new regional entlty for Oncho and LF elimination and support to other PC/NTD, 2013. Programme forthe Elimination of Neglected Diseases in Africa (PENDA): Strategic Plan of Action, 2013. Overheads of NGDOs sponsoring/ supporting APOC CDTI projects: administrative and financial arrange- ments for reimbursement, 2012. Authorised Position List 1April 2014. Final Communiqué JAF 18/2072 and 79/2013. NGDO Coordinating Group for Oncho- cerciasis elimination, 43rd session 2074. Committee of Sponsoring Agencles 43rd Session 2013. APOC Organizational Chart, JAF 18 2012. A guide for reviewing letters of agreement for CDTI projects, APOC, undated. Memorandum for the Afrlcan Programme for Onchocerciasis Control,2011. Programme budget utilisation for 2072 and 2013 by Specific Objective. APOC 2010-2014 Expenditure. Performance of predictors: Evaluating sustainability in community-directed treatment projects of the Afrlcan programme for onchocerciasis control, Uche Amazigo, Joseph Okeibunor, Victoria Matovu, Honorat Zot:re, Jesse Bump, Azodoga Seketeli, World Health Organization (WHO), 01 BP 549, Ouagadougou, Burkina Faso. The MOPAN Common Approach: http: //www.mopanonline. or g / uplo ad / do cume nt s / MOPAN_2013_WHO_Vo1. Il.pdf Capacity Benchmarking Tool, August 2006, Branch Associates, Inc., Abt Associates, Inc. Prepared for: U.S. Department of Health and Human Services. http : / / one st arfoundatio n.or g / wp-content/themes/OneStar/docu- ments /Capacity_Benchmarking_Tool. pdf Annex 3: Terms of Reference (ToR) Review of APOC management structure Terms of reference The African Programme for Onchocer- ciasis Control (APOC) is one of the most successful public private partnerships for health in Africa.It was established in 1995 to assist African countries to control River Blindness, and is implemented by the AfricanRegional Office of WHO which established a Secretariat in Burkina Faso to lead the Programme. The World Bank acts as Fiscal Agent, transferring income to WHO from more than 30 donors, and all medicines used inthe Programme are donated by the Merck pharmaceutical company. The programme planned to sunset in 2015, but due to its success, as well the recognition that continuing a little longer could help ellminate River Blindness from most African countries, its governing body decided to expand the remit of the programto include other diseases, and to continue the program until 2025. To asslst planning for this new role, the APOC team is seeking a reüew of the effec- tiveness of the current management struc- ture, and advrce on how that structure might need to change in light of the new remit. APOC seeks a simple management review; it does not seek an audit or an evaluation of the program. 1. Context and justification The initial goal of the African Programme for Onchocerciasis Control (APOC)was to control onchocerciasis (River Blindness) as a public health problem in Africa.It did this through the establishment of effective and self-sustainable commu- nity-directed treatment (CDI) programs throughout the endemic areas within the geographrc scope of the programme - initially 19 countries, now expanded to 31. Based on this goal and strategy,lt was considered important to establlsh a Secre- tariat to manage the program and lead the overall APOC partnership in such a way that the programs were part of the national health system, and that countries would seamlessly continue the program afterthe planned sunset of APOC in 2015. The approved Organizational structure for the APOC Secretariat consisted of one Administrative Unit and two Technical Units, all three units based at the HO_ in Ouagadougou, Burkina Faso. The admims- trative unit is geared towards managerial and administrative issues necessary to support the efficient and effective delivery of technical activities. The unit deals with staffing, Organization of statutory meet- ings, advocacy and communications and financial matters. The first technical unit: Epidemiology and Vector Elimination (EVE) deals with disease mapping,long term impact evaluation of CDI lnterven- tions, disease transmission modelling, cross border issues, capacity building of countries, vector elimination, data management and analysis. The second technical unit: Sustainable Drug Distri- bution (SDD) deals with drug distribu- tion strategy, sustainablllty, partnership lssues, monitoring and evaluation and operatlonal research. As APOC has evolved, its role has become more complex. Due to the success achieved in the implementation of the initial strategy of sustalnable communlty directed treatment, countries added other health interventions to their Community Directed strategies: such as distribution and use of insecticide treated bed nets for malaria, home-based management of fever in children, vitamin A supplements. These add-ons were later institutional- izedby the governing structure of APOC - the annual board which is termed the Joint Action Forum (JAF) - as a strategy of co-implementation. Today, APOC supports the original Community-Directed Inter- ventions (CDI) strategy as well as such additional new areas as co-implementa- tion and health system strengthening at the community level. In addition to the above, recent evidence that ivermectin treatment alone can Iead to the elimination of onchocerciasis (Diawara et aI 2009, Tekle et al 2012) has guided a shift in paradigm in the goal of APOC from control to elimination of onchocerciasis from the African conti- nent. Since 2010 APOC had started addi- tional activities towards elimination of onchocerciasis. Notwithstanding this expanded scope of work, the original structure of the APOC management and the staffing has not altered, and the same structure and staffing levels have been maintained despite the additional responsibilities. In summary the APOC Partnership since its inception in i99 5 and over the years has performed very well to control onchocer- ciasis leading to the change in paradigm from control to elimination of onchocer- ciasis on the African Continent. Over the past eighteen years, the APOC Partnership has evolved (with new partners coming on board and others departing), strategies have changed and a new era of enthu- siasm for global efforts to control/elimi- nate a broad range of similar Neglected Tropical Diseases has emerged. There is now a consensus that APOC should restructure into a new entity that will l) seek the elimination of River Blindness, 2) combine this goal with the concomitant elimination of Elephantiasis, and 3)where appropriate, seek ways to assist WHO AFRO in implementing the regional plan for controlling other Neglected Tropical Diseases. Given these changes and evolution, and the development of plans for the future, a management review is timely. The review was recommended in the "APOC Donor Dialogue" report and was agreed on behalf of the JAF by the 141st meeting of the Committee of SponsoringAgencies (the CSA) in Hammamet, Tunisia in 2013. 2. Objectives 2.1. Main Objective Determine whether the current manage- ment structure and system, including the level of human resources is appropriate to perform the current duties ofAPOC at the Headquarters level in Ouagadougou and in the 31 countries served by the APOC, and to comment on the staffing and manage- ment implications of expandingthe scope of the program if it is restructured into a new entity beyond 2015. 2. 2. Specific objectives 1. Describe the current level of APOC staffing at the secretariat and country levels, their qualifications, their remunera- tion and their roles and responsibilities. . The qualifications of staff, their loca- tions, their support structures and their respective roles, duties and responsibilities. 2. Describe the management structure of APOC, especially the relationship, including roles and responsibilities of staff, between APOC and the national government programs: . The organogram and Organizational connections; . How funds flow, including the relation- ship with \MHO systems; . The accounting process, including the relationship with WHO systems; . The reporting system within and beyond the Secretariat; . How management meetings are organ- ized and their frequenry; . Any extemal audit queries and manage- ment response; . The management decision making processes; . The coordination mechanisms within the Secretariat; . The communication process and strate- gies within the Secretariat; . Government participation at the regional and country levels. 3. Comment on the implications for management and costs if the scope of the work is to be expanded to address 1) LF as well as River Blindness (country number increases from 31 to 34, two diseases to be addressed using common platforms) or 2) all 5 preventable NTDs ( country number increases from 31 to 47, more than one delivery platform required). . Assess workforce implication of para- digm change (control to elimination); . Identify changes needed to the manage- ment structure to meet the needs of elimination of onchocerciasis and LF; . Comment on the ability of the APOC secretariat to change in response to the expanded scope of work, and the processes required to facilitate this. 4. Benchmark these findings against appropriate management standards where possible. In particular to assess the extent to which the procedures follow appropriate standards and, if not, how this might be remedied. 5. Summarize the strengths and weak- nesses of the APOC Secretariat and provide comparison with established good practice. 3. Duration of contract The contract will have a duration of one month. The consultants should develop a work- plan which should include: . Howthis evaluationwillbe carried out, clearly indicating the steps involved; . The specific tasks to be performed and bywhom andwhen; . The specific data that will be needed, along with data sources; 4. Timeline The consultancy will start in May,2074 and the first draft should be ready by mid- June, 2014 for comments by CSA and TTF members in July, 2014. The Team leader ofthe Evaluation Team will be invited to present a second draft at the next session of CSA and get further inputs from CSA. One week after CSA, a final draft should be submitted to APOC Management for inclusion in the list of documents for the next CSA and JAF 20. 5. Skills of consultants Two consultants at P5 level with the following expertise are required to carry out this evaluation. . Masters in Business Administration with experience in management consulting and Organizational assess- ment.An experience in assessing health prograrns and institutions at the inter- national level would be an asset . Masters in Public Health with a major in Epidemiology and disease control would be an asset. Experience in assessing the performance of programme implemen- tation at intemational level is needed and past experience working for UN agencies especially \MHO would be an asset. 6. Financial arrangements Each consultant will receive honorarium at P5 level plus per diem when traveliing. The amount will be paid as follows: . 25% at start; . 25%" when first draft is delivered; . 25% whenthe second draft is submitted for CSA; . 25% when the final draft is submitted.