Sierra Leone - Technical Assistance Project

Do_mnt Of The World Bank FOR OFFICIAL USE ONLY Report No. 6489 PROJECT COPLETION REPORT SIERRA LEONE TECHNICAL ASSISTANCE PROJECT (CREDIT 970-SL) November 13, 1986 :.estern Africa Regional Office This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  LIST OF ABBREVIATIONS USED CPU Central Planning Unit (MDEP) CSO Central Statistics Office (MDEP) GOSL Govetnment of Sierra Leone IBRD International Bank for Reconstruction and Development IDA International Development Association MAP Ministry of Agriculture and Forestry MDI? Ministry of Development and Economic Planning MOE Ministry of Education MOF Ministry of Finance MOH Ministry of Health MOW Ministry of Works PCR Project Completion Report PEMSU Projects Evaluation, Monitoring and Services Unit (MAP) PIP Public Investment Program TOR Terms of Reference UNDP United Nations Development Program UNDTCD United Nations Department of Technical Cooperation for Development  FOR OFFJCIA USE ONLY THE WORLD SANK Washington. DC 20433 U S.A offi ce Doftwcttv.ce4aI Op.atumw EvekM lt" November 13, 1986 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Project Completion Report: Sierra Leone - Technical Assistance Project (Credit 970-SL) Attached, for information, is a copy of a report entitled "Project Completion Report: Sierra Leone - Technical Assistance Project (Credit 970-SL)" prepared by the Western Africa Regional Office. Under the modified system for project performance auditing, further evaluation of this project by the Operations Evaluation Department has not been made. Attachment This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  FOR OFFMCIAL USE ONLY PROJECT COMPLETION REPORT SIERRA LEONE - TECHNICAL ASSISTANCE PROJECT (CREDIT 970-SL) TABLE OF CONTENTS Page Preface .....,... ....... i Basic Data Sheet . . . ................ ii Highlights ,...,... v I. Project Background and Objectives ................................ 2 II. Project Management and Execution ................................... 3 .III. Major Issues. 5 IV. Conclusions ............... 8 ANNEXES 1 - Project Background and Preparation ....................... 10 2 - Project Management and Execution ......................... 12 3 - Project Supervision .................... 17 4 - Disbursements ............................................. 20 5 - Training ............................................,. 21 6 - Performance within Ministries ............................ 24 Appendix 1 - Value and Purpose of Consultant Contracts ......... ...... 28 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  PROJECT COMPLETION REPORT SIERRA LEONE - TECHNICAL ASSISTANCE PROJECT (CREDIT 970-SL) PREFACE The Credit providing technical assistance to the Government of Sierra Leone for the improvement of development planning in certain of its sectqral ministries was finally closed on December 31,. 1984, after two one year extensions. (Disbursements were allowed until June 30, 1985, for items agreed to prior to the Closing Date.) In December, 1985, it was agreed that the Project Completion Report (PCR) would be prepared by the Public Sector Management Division of the Western Africa Projects Department. It is based on a review of the President's Report (P-2665-SL) dated December 5, 1979, (no Staff Appraisal Report was thought to be necessary for this Project, a practice which remains fairly common for technical assistance projects) the Development Credit Agreement dated February 8, 1980, internal IDA reports and memoranda on project preparation and execution, progress reports prepared by various consultants. involved with the project, Project Supervision Reports prepared by IDA staff and other documents from the IDA's files as well as interviews with IDA staff involved with the design and implementation of the Project.Altnough it was originally intended for staff to visit Sierra Leone in order to interview officials of the Government who had been involved with the Project. it was eventually decided that the amount of time (almost three years) which had elapsed since the departure of the long term advisors, whose services comprised a very large part of the Project and whose performance remains one of the probable factors in the Project's lack of success, would make it unlikely that such interviews would produce new insights into the Project!s execution. This PCR will not touch on the performance of the two advisors under the UNDP-funded projects executed by the Bank nor of the other UNDP-funded advisors in MDEP since, although their work was closely related to that of the Project, they were not strictly part of it. The PCR is separated into two parts: the PCR itself; and a series of annexes which provide more detailed information on the design, preparation and execution of the Project. In accordance with the revised project performance reporting procedures, this report has been read in the Operations Evaluation Department (OED) but the Project was not audited by OED staff. The draft Completion Report was sent to the Borrower for comments; however, none were received.  -ii- PROJECT COMPLETION REPORT SIERRA LEONE - TECHNICAL ASSISTANCE PROJECT (CREDIT 970-SL) BASIC PROJECT DATA KEY PROJECT DATA Appraisal Actual Total Project Costs (US$ million) 2.80 2.20 Credit Amount (US$ million) 2.50 1.93 -Disbursed and outstanding - 1.93 Economic Rate of Return NA NA Financial Performance NA NA Institutional Performance Weak OTHER PROJECT DATA Original Plan Actual First mention in files - 12/04/78 Identification - 02/00/79 Appraisal July,1979 08/00/79 Decision M{emo September,1979 10/10/79 Negotiations November,1979 11/21/79 Board Approval December,1979 12/27/79 Credit Agreement Date January,1980 02/08/80 Effectiveness March,1980 06/30/80 Completion Date June,1982 06/30/85 Closing Date December,1982 12/31/84 Borrower Republic of Sierra Leone Executing Agency Ministry of Development and Economic Planning Credit Number 970-SL Follow-on Project None  MISSION DATA Item Month/Year No. of Days No. of Staff Report Date Identification February,1979 4 1 02/05/79 Preappraisa'. May,1979. 2 2 06/19/79 Appraisal August,1979 10 1 08/15/79 Supervision I November,1980- 4 1 11/26/80 Supervision II July,1981 4 1 08/14/81 Supervision III December,1081 4 1 01/20/82 Supervision IV November,1982 10 2 12/08/82 Supervision V April,1983 10 2 06/03/83 Supervision VI October,1983 4 3 12/08/83 Supervision VII February,1984 4 2 03/12/84 Supervision VIII July,1984 5 2 09/04/84 N.B. The mission data do not reflect the time spent supervising the various sectoral components in the course of other project supervision by those divisions responsible for those sectors.  -iv- DISBURSEMENTS BY FISCAL YEAR (US$ Million) FY80 FY81 FY82 FY83 FY84 FY85 ANNUAL Appraisal Estimates 0.48 1.57 0.45 - - - Actual- - 0.34 0.59 0.58 0.23 0.19 Actual as % Est mate 0 22 131 CUMULATIVE Appraisal Estimates 0.48 2.05 2.50 - - Actual 0 0.34 0.93 1.51 1.74 1.93 Actual as x Estimate 0 17 37 DISBURSEMENTS BY CATEGORY (US$ Million) Appraisal Estimates 'Actuals (1) Salaries and other emoluments 0.840 1.135 for long-term advisors (a) Salaries and other emoluments 0.750 0.535 for short-term consultants (3) Training 0.050 0.108 (4) Equipment and Vehiclvrs 0.075 0.071 (5) Project preparation studies 0.650 0.084 (6) Unallocated 0.135 Total 2.500 1.933 N.B. US$ 567,000 went unutilized and was cancelled. There remains some confusion about the dividing line between Categories 2-and 5..See Annex 4.  PROJECT COMPLETION REPORT SIERRA LEONE - TECHNICAL ASSISTANCE PROJECT (CREDIT 970-SLj HIGHLIGHTS The Credit to Sierra Leone for technical assistance to the planning sector was one of the first free-standing TA projects by the Bank Group in Africa. While it did achieve limited success in the areas of training (para. 10 and Annex 5) and project preparation (para. 9) it was not a fully successful project for a variety of reasons: (a) project design proved unable to cope with problems as they arose (paras. 4 and 12(a) and Annex 1); (b) the technical assistance being provided was badly coordinated by the donors (paras. 3, 7 and 12(b). and Annex 1); (c) Government commitment to project goals was less than it needed to be in order to achieve them (paras. 7, 9 and 12(c) and Annex 2, para.9); (d) there was initial confusion and eventual acrimony over the selection of the primary consultant. There is reason to believe that the IDA erred in allowing the appointment (paras. 5and 12(e) and Annex 2, para. 6); and, (e) supervision by IDA staff, although nominally adequate, may have been insufficient and wrongly directed (paras. 12(f) and Annex 3).  PROJECT COMPLETION REPORT SIERRA LEONE - TECHNICAL ASSISTANCE PROJECT (CREDIT 970-SL) SUMMARY The Technical Assistance Project (Credit 970-SL) was negotiated in November,.1979, approved by the IDA's Board-in December, 1979, signed in February, 1980, became effective in June, 1980,was Closed on December 31, 1984, after two one-year extensions, and disbursements ceased to be made on June 30, 1985, when the US$ 566,317.91 remaining undisbursed in the Credit Account (some 23X of the original amount) was cancelled. To date, there has not been a follow-up project although there were certain aspects of the TA Project meant to complement the proposed Structural Adjustment Credit (SAC) which were designed to build upon areas where some gains were made by the Project.. If one is to take literally the Project Objectives listed in the President's Report, which was the product of the Project negotiations and the vehicle for Project approval by the IDA's Board, the Project can, at best, be described as a very limited success. While it may not be fair to judge it an "almost total failure" as one consultant did, the Project certainly did not achieve the goals set out for it in the President's Report. Some achievements can indeed be laid to the 'Project's door, most notably in the agricultural sector, and in a more restricted sense, in training. Nonetheless, the Project was not a resounding triumph and this PCR will attempt to trace the causes of the successes and failures in achieving project aims and to try to identify lessons which can be learned from them.  -2- I. PROJECT BACKGROUND AND OBJECTIVES '. The Technical Assistance Project was imeant to provide support to the Government of Sierra Leone (GOSL) in the areas of development planning, the training of local staff, preparation of development projects and monitoring and evaluation of public investment programs. Prior to lending for this, the IBRD and IDA, which have been active in Sierra Leone since 1964, had been involved in the agriculture, power, education and highways sectors but the loans and credits were for more traditional investment projects. Moreover, due to the deteriorating economic situation in the country and to the Government's seeming inability to devise a rational plan to order its finances and public expenditure, the Bank Group had only approved one Credit (Third Power) since 1975. While this was the IDA's firs, such foray into planning sector in Sierra Leone, thz UNDP had been assisting the Government in the planning sector since the early 1970's with the UNDTCD as sxecuting agency. When the IDA was initially approa?ched by the Government in December, 1978, about the possibility of providing more assistance in this area, the GOSL had just concluded an - agreement with the IMF on a stabilization program and had also completed the preparation of the first three year Public Investment Program (PIP), something the Bank Group had been requesting action on for some time. Since these were the prerequisites for further lending, the IDA looked upon the Government's request favorably both as a means of resuming lending on a controlled scale and a way of assisting GOSL in improving its planning capabilities. Two avenues were initially considered as appropriate vehicles for this type of assistance: formulation of a project in which the IDA would be one of several donors; or, a continuation of the UNDP-funded assistance but with the Bank as the executing agency. 2. Under the original UNDP-financed project to provide planning support to GOSL, expatriate personnel assisted the.Government in the preparation of the First National Development Plan,which was designed to cover the years 1974/5- 1978/9. This Plan came to naught due to the unfortunate combination of economic trends completely beyond the Government's control (e.g.oil prices) and the serious deterioration in the internal management of the economy. However, the UNDP project was extended and the advisors continued to assist the Ministry of Development and Economic Planning (MDEP) in working toward a Second National Development Plan for the years 1981/2-1986/7 as well as an interim PIP covering only the three years between the Plans. It was partly because of some doubts as to the effectiveness of the UNDTCD team as well as the perception that the Government badly needed a stronger sector planning and project preparation capability, particularly in the agriculture and infrastructure investment areas, that led GOSL to approach the Bank with the request for additional assistance. 3. From the beginning, there was confusion about which agency would act to execute the planned second extension of the UNDP project, the IDA or UNDTCD. The Government, it seems, was already having discussions with the UNDTCD about its continuing as executing agency when it approached the Bank. Indeed, by May, 1979, when the Project was being preappraised, the UNDTCD had already prepared a preliminary project document for the provision of advisors and other support to MDEP. This UNDP-financed project was to last three years starting in January, 1980, and was budgeted at US$ 1,500,000. The assumption of UNDTCD was that the executing agency would be UNDTCD. During subsequent discussions UNDTCD, the UNDP and the Bank, it became clear that, despite certain concerns about UNDTCD's draft project document (concerns apparently based on the fact that this would be the third such project since 1970 and  -3- there appeared to be little difference in the objectives of the three or in the suggested means of achieving them nor had there been any discernible long term institution building effects resulting from either of the previous two projects), it appeared likely that GOSL would prefer to have UNDTCD as the executing agency rather than the Bank. While there remained some reservations about the efficacy of this approach, the IDA did not want to be perceived as trying to dominate the preparation of the next National Development Plan then urderway and agreed to appraise a TA Project directed at the sectoral Ministries (i.e. MAF; iMOE, MOH and MOW). Such a project would, it was reasoned, complement the UNDTCD's efforts in MDEP and contribute to the improvement of the planning capability of GOSL. In addition, the IDA agreed to be the executor for two one-man UNDP-financed projects: an infrastructure planning advisor for MDEP; and, a fiscal advisor for MOF. 4. The Project, as finally appraised in August, 1979, had the following objectives, which should be viewed within the "framework of a long-term policy aimed at gradual improvement in development planning and project preparation within the central and sectoral ministries": (a) to strengthen the Government's cap'ability in development planning with emphasis on the formulation of public sector investment programs, the establishment of development objectives and policy instruments, and identification of resource availability and constraints; (b) to assist Government in the preparation of priority development projects aimed at. diversifying and expanding the productive b3se of the economy, improving the cob. ry's economic and social infrastructure and promoting a broader distribution of the national income; (c) to train local staff in development planning and project preparation and implementation; (d) to establish a system for the monitoring and evaluation of public investment programs. The goals of objectives (b) and (c) were at least partially met in the case of the agricultural sector where some project preparation was completed and considerable training took place. In the other sectors it is difficult to identify an: real progress on these goals during the life of the Project. On the longer-term, specifically institution building aims of (a) and (d) very little, if any, success can be recorded in anv of the subject Ministries. II. PROJECT MANAGEMENT AND EXECUTION A. Project Costs 5. The designers of the Project envisaged the long-term advisors to be assigned to the sectoral ministries to be one of, if not the key, foundations upon which the success of the Ptoject would be built and, from the outset, the IDA advised the Government that hiring the advisors individually would make far more sense than contracting with a firm -o supply them from the standpoint  -4- of cost as well as performance. The Project costs were based on the assumption that individual advisors would be employed.However, the Government opted for a consulting firm to provide these services. The result was a cost uverrun of some 282 for the cost of the long-term advisors. The selected consultant was paid a total of $ 1,079,025 in addition to the Le.70,000 reallocated in 1983 for internal travel. The original estimate of the cost (and the amount in Category 1) was $ 840,000. This overrun was not apparently noticed by the IDA until later although the Bank approved the contract in April, 1981. 6. The costs in other areas rpflect to some extent the performance of the various ministries under the Project. Categories 2 and 5, for short-term consultants and project preparation and feasibility studies respectively, totalled some $ 1,400,000 but only $ 608,000 was disbursed. (Only MAP took full advantage-of the funds available,spending some $ 425,000 or 70% of the amount disbursed. MOW did employ a consultant to assist in the preparation of a highways project but neither HOE nor MOH used any of the funds in these Categories.) Category 3 (Training) was well utilized, being overrun by more than 100%. (This was largely spent on overseas courses, none of which were envisaged in the original project design.) Category 4 (Equipment and Vehicles) c.ame close to the original estimates for its utilization. v. Project Coordination 7. Beyond the question of cost, however, and equally important, was the issue of coordination of the work to be done by the advisors with that being done in MDEP by the UNDP-financed consultants and of the general coordination of inputs into the preparation of the Second National Development Plan. To ensure the overall coordination of planning activities between the central and sectoral ministries, the Bank and GOSL agreed to the reactivation of the National Planning Council and the Inter-Ministerial Planning Committee (both of which had been essentially moribund since their establishment in 1972) and the formation of a Technical Committee comprised-of representatives of the planning units withir. ministries (as and when these were created and staffed) and I"appropriate professional staff" from central and sectoral ministries. This Committee was to meet at least quarterly to "discuss activities and progress and monitor coordination of the Government's planning exercises." Neither the Planning Council nor the Planning Committee met during the life of the Project. The establishment and regular functioning of the Technical Committee was a condition of Credit Effectiveness and the Government duly informed the IDA of its founding and composition in May, 1979. However, while it was established, it seems that there was only one meeting held despite repeated assurances from GOSL over the life of the Project that a meeting was imminent although all of its members were theoretically directly involved with the execution of the Project. 8. Coordination suffered at yet another level when it was decided that it would be unnecessary to build any other device into the Project to ensure that the advisors in the different ministries funded by the various donors were all headed in the same direction and that their efforts were being channelled toward agreed goals. Prior to the appraisal of the ?roject, the IDA suggested the advisability of someone to be appointed to provide such coordination but others argued that this would be seen as a potential threat to the sectoral ministries involved and they were likely to refuse to relinquish any authority to such a person. In the face of that objection, the idea was not pursued. As a result, and in the absence of any overall coordination from the Technical Committee, two "project managers" eventually appeared on the scene; one for the UNDTCD team and another by the consultant to oversee the work of the  Project-funded long-term advisors. It is quite unlikely that either contributed greatly to the enhancement of the Government's planning capabilities. C. Project Implementation 9. The Project was to have contributed to the institutional capabilities of the Government through the provision of long and short-term consultants and funds for training seminars -nd project preparation and feasibility studies. While there were identifiable benefits from the Project in the preparation of the Second Highway Project (Cr. 1129-SL) and the Agricultural Sector Support Project (CR. 1501-SL) and, probably, the Third Education Project (Cr. 1353-SL), less progress can be claimed in either the direct production of feasibility studies or in preparing the staff of GOSL to produce these in the future. But, in addition to project-specific studies, there were several activities for which consultanc funds were to have been provided under the Project. Fxamples of these are: MOH was to have hired a consultant to reinforce the work of its planning unit (to heve been established under the Project); MOE was to have employed one to astist in the design and construction of school buildings and another to prepare and maintain school maps; the MOM was to employ a consultant to assist in the formulation of plans for the development of the mining sector; and MDEP was to have used funds to establish a central monitoring and evaluation system. None of these tasks were eventually carried out under the Project and the poor or inadequate use of consultancy funds available under IDA projects continues to be a problem area. 10. The funds provided for training were intended for the preparation of at least two in-country seminars dealing with sector and central planning, project preparation, implementation, monitoring and evaluation. Only one of these seminars was held and that was in November, 1983, several months after the departure of the long-term advisors. Of the total funds disbursed under this Category, some $ 84,000 (78%) were largely spent on overseas training for GOSL officials in a variety of Project-related areas. The majority of this money seems to have been appropriately spent to improve the capabilities of the staff of PEMSU and of the CSO. Nonetheless, some rather tenuous arguments were made for a few of the disbursements under this Category such as topping-up the overseas allowances of senior officials of MDEP while they were on courses in the United States and several requests from GOSL for funding other types of overseas training trips were rejected since they could not be justified as being relevant to Project objectives. 11. The selection and performance of the long-term advisors and the performance of the Government have been dealt with in some detail in Annex 2. Annex 3 presents an overview of the Bank's performance in superAtising the Project. Disbursements and Training are discussed in Annexes 4 and 5. III. MAJOR ISSUES 12. Several issues emerge from the preceding review of the Project. Among the most important of these concern: (a) project design. Frequently, projects such as this were seen as catchall vehicles for the provision of expatriate manpower in order to achieve many and often varied institution building goals. Thus, in this, as in many of the free-standing technical assistance projects, it was assumed that the long-  -6- term advisors could achieve all or most of the Project objectives by their very presence. The indications from the review of this and similar projects are that the advisors alone are not enough to assure success. The TOR for the long- term advisors were specific and well thought out but the other aspects of the Project were largely left to be designed as time went on. There was little thought given to the design of TOR for the counterpart staff who would be assisting, and learning from, the advisors. By implication, the major tangible "product" of the Project was to have been the successful production of a National Development Plan and yet there were no specific measurement criteria set for the evaluation of progress toward this goal, either for the country as a whole or.in specific Ministries. Admittedly, the Project was one of the first of the free- standing TA projects funded by the IDA in West Africa and the absence of stringent initial design was a conscious result of the planners' concern that the project be flexible. Nonetheless, white the lack of a confining framework was beneficial as far as the short-term studies were concerned,. overall it made meaningful evaluation of the Project's longer-term goals almost impossible. Thus it was not easy to anticipate needed changes on a rolling basis and, therefore, especially from 1983 on, each supervision mission was essentially required to re-appraise the Project and the long term goals of improving local capacity in the planning and project preparation areas often seemed to get lost in the effort to disburse the remaining funds under the Credit. To a' large extent, now that these potential problems have been identified, there is less danger of their being missed in the project design stage, but they remain an important issue; (b). coordination of donor technical assistance by the Government and the donors. While there was an appearance of cooperation and coordination of effort between the Government, the UNDP, UNDTCD and the IDA in this case, it does not seem to have translated to cooperation in the actual implementation of the Project. The Government established a Cabinet Sub-Committee for Aid Coordination in 1983 but it was short-lived and did not lead to any real reforms in this area; (c) Government commitment to meeting Project objectives. There is every indication that the IDA staff who identified and appraised the Project were reasonably convinced that there was support for the Project's goals when it was presented to the Board for approval. Nonetheless, there is evidence that any commitment which was there  -7- initially was not broad based and quickly dissipated with the departure of the Planning Secretary in 1980. Even his brief return sometime later did not seem to serve to revive any true support in MDEP. This lack of clear commitment (or perhaps simply the inability to act on it) indicated the general lack of priority seemingly accorded to planning as demonstrated by the continued failure to produce a plan document and must be considered one of the'chief reasons for the Project's limited achievements; (d) availability of properly qualified national staff. Projects such as this have, as their ultimate goal, institution building. This cannot be done if the raw material on which the institution is to be based - staff - is not available. The problem of attracting and retaining high-calibre staff is one *which still seems to defy solution. However, it must be taken into account in designing this type of Project. After all, if the ministries to be assisted lack the necessary staff, the technical assistants who should be overseeing.the training and growth of such staff are often required to attend to the line duties of the ministries instead. Thus, many TA projects are doomed from the start because they fail to address this issue either by providing funds for the salaries or salary supplements for local counterpart staff or in some other way. Only in the case of MAF (in particular PEMSU) was there any obvious (hopefully long-term) institutional building effect and that is almost certainly attributable more to the ongoing technical assistance which PEMSU was receiving through the existing agricultural projects than to this Project; (e) recruitment of staff. While the final choice and the difficulties surrounding the eventual decision concerning the provision of consultant services under the Project may be rare, it presents an important 4ssue. Such a matter is not a mere detail and must form a part of the project design, as it did in this case. From the beginning of the discussions concerning the Project between IDA and GOSL, IDA staff thought that there was agreement that individual consultants would be selected to provide the technical assistance to the ministries on the assumption that these would be cheaper and that it would be more likely to locate and recruit advisors with the needed skills on an individual bas. . It was with some surprise, therefore, that the IDA received the Government's decision to employ a firm, especially one with no demonstrable expertise in the areas of planning and project preparation. The delays which resulted in the finalization of the consultant's contract stemmed  -8- directly from the Government's decision to alter this basic supposition of the appraisal mission and the IDA's reluctance to accept the Government's choice of firms. Beyond the question of individual consultants versus a firm is another issue - whether it was responsible for the IDA to give in to the wishes of the Government, thus ignoring its stated reservations concerning the ability of the consultant to carry out this particular contract successfully. The results clearly indicate that it was not; (f) the purpose and role of IDA supervision of technical assistance projects. Only a total of some 53.3 staff weeks was recorded for the IDA's supervision of this Project. If the object of supervisioa is to check on progress and to assist when the project falls into difficulties, this is probably a reasonable figure, especially considering that there was quite certainly a large amount of unrecorded time spent on this Project by missions doing other things. But, if the IDA's role in such projects is to assure the adequacy of the institution building aspects of the project, it is probably not .enough. It would seem that the solution to this is either to make such projects far less complicated or to admit that they are going to absorb more time than other projects and budget for it. Certainly, projects such as this, which are directed at several ministries, require a great deal more time to effectively supervise. More thought needs to be given to the question of whether optimum results can be achieved using firms for this type of work. There would seem to be a strong case for employing individuals instead. Perhaps some consideration should be given to employing the type of consultant to provide technical supervision in such cases as described in Annex 3. IV. CONCLUSIONS 13. The general conclusion to be drawn from this PCR is that both parties to the Credit Agreement were at least partially to blame for the fact that the Project failed to achieve the objectives expected of it. On the IDA's side, the project designers were too sanguine about the extent of the Government's expressed commitment to the Project objectives and the supervision missions were late in noting the downward drift of the Project and, when they did, were not willing to propose the logical (albeit draconian) step of suspension of disburse.ients with a view towards eventual cancellation of the Credit in the face of repeated and seemingly obdurate non-compliance with the Credit conditions on the part of GOSL. The obvious benefits to such a course of action were tnat good money would no longer be thrown after bad and that it would probably-serve tG make the conditionality of future projects more believable. However, there were perceived costs as well.There was a desire to maintain IDA's dialogue with the Government and the perception that a cancellation would  9 do more harm than good in keeping lines of communication open and the supervision missions, especially after the consultant team's departure, felt that effective use could be made of the Credit funds. Therefore, rather than recommending a suspension and then the cancellation of the Credit, the supervision missions worked with the GOSL staff to make the Project into a sort oi rolling plan to try to achieve at least some of the Project's aims. This resulted in two fine-year extensions of the Credit Closing Date in order to allow the Government enough time to make use. of the Credit funds. However, while the motives of the missions may have been laudable, their efforts did not, in the end, contribute greatly toward achieving the original Project objectives. Considering the nature and complexity of the Project, the IDA should have been prepared not only to devote greater resources to its supervision but also should have ensured that proper coordination took place .between the various sectoral d'ivisions which were to have been supervising project components. On the Government's side, fault can be found with the apparent lack of commitment to the overall project goal of establishing a sound planning capability and the long-term, institution building benefits which that would have brought about. This lack of commitment was demonstrated in many ways, e.g. failure to appoint qualified counterpart staff or to make the necessary budgetary allocations to establish and properly fund the planning units in some of the sectoral ministries. Both GOSL and IDA are, to some extent, responsible for the poor p,erformance of the consultant, since it should have been clear'that it was unlikely that the firm had the potential to provide .adequately the services which would be required under the Project and this fact should have precluded the consultant's selection.  -10 ANNEX 1 Page 1 of 2 pages I. Project Background and Preparation A. Background 1. The UNDP started providing planning assistance to GOSL in 1970. The executing agency for both the first project and its successor in 1975 was the UN Department of Technical Cooperation for Development (UNDTCD). The First National Development Plan, meant to cover the years 1974/75 - 1978/79, was prepared with the assistance of the team of advisors fielded by UNDTCD. However, completion of the Plan coincided with the oil price rise in 1973 and, for this and a variety of other reasons, the Plan was never implemented. Preparation of a Secornd National Development Plan was not finally scheduled until 1980, this time to cover the years 1981/82 - 1984/85 and'it was not until 1978 that the Government was able to come up with an in.terim, medium-term Public Investment Program (PIP) as a stopgap measure. Indeed, the Second National Development Plan has yet to be done, despite the continued technical assistance received from the UNDP. The Government relies instead on rolling three year PIPs which have proved to be more flexible and manageable. B. Project Preparation 2. Following the Government's request in December, 1978, and the IDA's Agreement to consider planning assistance as a vehicle to recommence lending to GOSL, an identification mission was sent to Sierra Leone in January/February, 1979, and a follow-up preappraisal mission visited Freetown in May, 1979. Both were designed to identify possible project components and, through discussions with other donors, cofinanciers. Originally, it had been thought that the Bank Group would lend only $200,000/ $300,000 of the project costs with the remainder coming from other sources. This was being considered until just before the appraisal of the project in August, 1979, largely because, until that time, it was not clear whether the Bank would be the executing body for the continuation of the provision of the planning advisors.to the Ministry of Development and Economic Planning (MDEP) or whether this would again fall to UNDTCD. Eventually, it was decided that the IDA credit would fund long and short term consultants for four sectoral ministries (Agriculture and Forestry (MAF),Health (MOH),Works (MOW) and Education (MOE)), training, project feasibility studies and vehicles. The Bank also agreed to be executing agency for two one-man UNDP-funded projects:, an infrastructure advisor to assist MDEP; and, a fiscal advisor for the Ministry of Finance. Simultaneously, the UNDTCD was to ccntinue to execute the UNDP assistance to MDEP on an extended and somewhat broader scale. The reason for this compromise (about which there are indications that the UNDP in New York and the Bank were somewhat dubious) was that the UNDTCD had been?involved with planning assistance in Sierra Leone too long and the preparation of the extension of the existing UNDP project had gone too far at that point to allow the Bank to become the executing agent without causing some disruption. While some slight delay may have resulted, this decision must be considered an error for, although it is true that no one UN Agency nor any other donor should be seen or even be perceived to be attempting to dominate the aid process, each has a responsibility to assess each project proposed for its financing carefully and, should serious reservations exist about any facet of them (as they apparently did in this case) to address them. 3. Specifically, the Project was meant to increase Government's ability to carry out development planning, the preparation of development projects, the training of local staff and the monitoring and evaluation of investment programs. To accomplish this, the Project provided for the provision of funds for:  - 1 1 - ' ANNEX1 Page 2 of 2 pages (a) 10 man-years of advisors for the four sectoral ministries (5 advisors for two years each); (b) 75 man-months of short-term consultants; (c) training, with the emphasis on two in-country seminars but with provision for study tours abroad;and, (d) project preparation and feasibility studies.' It was assumed that this assistance would support the preparation of the 2nd National Development Plan by MDEP and eventually enable GOSL to re-shape its policies to develop a public investment program which would sustain a balance between social'and productive needs and remain within the country's resource capabilities. 4. Since there seemed to be real commitment, and the only important issue remaining was the Government's economic performance, which seemed improved, or at least intproving, the Project was submitted to Senior Management under cover of a Decision Memorandum dated October 10,1979, and following approval, negotiations were held in Freetown from November 19 to 21, 1979. During negotiations, it was agreed that two special conditions would apply to the Project: (a) contracts for advisors and consultants would be subject to the approval of IDA; and (b) the Government was to establish, as a condition of Effectiveness, a technical level working committee consisting of representatives of the planning units and appropriate professional staff of the sectoral and central ministries. Despite the Government's assurances',however,there remained some concern that there might not be complete commitment on the part of GOSL to its own development goals as indicated 'by the economy. Also, there was some concern that it might be difficult to locate and recruit the ri.ght type of advisors to successfully carry out.the Project aims. Nonetheless, the Project was presented to and approved by the Board on December 27, 1979, and the Credit was signed on February 8, 1980. After a delay of several months and continued prodding from the IDA the Government created the technical committee as required and, the other conditions having been met, the Credit was declared effective on June 30, 1980. The Closing Date was December 31, 1982.  - 12- ANNEX 2 Page 1 of 5 pages II. Project Management and Execution A. Project Management 1. Management was not really seen as an issue at the commencement of the Project for two reasons: (a) the Government had agreed at negotiations to resuscitate the National Planning Council and the Interministerial Committee of Permanent Secretaries to coordinate planning policy and to create a technical level working committee consisting of representatives of the planning units and appropriate professional staff of the sectoral and central ministries. The latter was a condition of Credit Effectiveness and the technical committee was indeed established but it is doubtful that it met more than once or twice over the entire life of the Project in spite of repeated urging on the part of the Bank; and, (b) it was originally envisaged that the long-term advisors would be employed as individuals and they would need little other coordination since they would be working almost solely with the staff of the Ministries to which they were assigned. The result of this was that very little cross sectoral management did take, place. The consulting firm with which GOSL eventually contracted for the provision of the long term advisors arranged for the contract to call for an "Overall Project Manager" with "responsibility for the preparation of the Project, overall coordination and monitoring." However, this person was to visit Sierra Leone only twice during the contract for one month at a time. It is unclear what specific management was provided under this clause of the contract. (The IDA was also guilty of failing to assure the provision of cross sectoral support between its own Divisions. See Annex 3.) B. Selection of the Consultants 2. The Project was originally designed on the basis that individuals would be employed by GOSL to carry out the duties of the five advisors in the sectoral Ministries and that the short term studies and consultancies would be awarded on a case by case basis lo the best qualified individuals or firms as necessary. The two prime reasons for this decision were that individuals were almost always substantially less expensive than consultants provided-through firms because a firm's overhead charges and profit would not be added to the costs and that individually recruited advisors' loyalties would be more obviously with the Government than those employed by a firm. The appraisal mission discussed this with officials of the Government, agreement was reached to follow this approach, and, as early as December, 1979, the Bank began submitting the curricula vitae of potential candidates to the Government for review. However, in May, 1980, a European consulting firm submitted a proposal to the Government to carry out the entire Project. This proposal was copied to the Bank, which informed GOSL in a letter to the Development Minister in June, 1979, that, while the advisors proposed by the firm should be considered along  -13- ANNEX 2 Page 2 of 5 pages with the many other c.v. which the Government were reviewing, the Bank still felt that individually recruited personnel would better suit the needs of the Project. 3. Despite a reiteration of this view and a reminder that the Bank must review the qualifications, experience and terms and conditions of all the advisors and consultants, the Development Minister telexed on 30 July, 1980, informing IDA that GOSL had accepted the firm's proposal to carry out.the Project. The award was to include not only the long term advisors to the sectoral Ministries but also the short term studies and consultancies which were a part of the Project. Although the consultant had been active in Sierra Leone and was then involved with the Mano River Union, it had no demonstrable experience in the development planning field. Furthermore, the candidates it had originally proposed for the long-term advisor positions were not considered suitable for the posts. Therefore, the IDA was, at first, not willing to agree to the selection of the consultant to carry out the Project. Finally, though, the IDA did give in to the wishes of GOSL in its selection despite the concerns about cost and over the objections of some'IDA staff who remained unconvinced of the propriety of the selection and despite the knowledge that at least one Ministry to benefit from the Project was not in favor of the award nor of the procedures used in the selection of the consultant by MDEP. However, IDA did insist that the short-term consultancies be contracted separately and be awarded to the best qualified person or firm. The consultant was to be eligible for consideration for such assignments as and when they arose but not to the exclusion of other, possibly better qualified, firms or individuals. (In the event, none of the short-term consultant assignments was awarded to the firm.) This compromise must be considered one of the initial failures by the IDA since it is clear that considerable reservations about the selected firm's suitability remained. 4. The contract between GOSL and the consultant was not finally signed until March, 1981, due to delays in providing the Bank with draft copies of the contract on a timely basis for review (the Bank's October, 1980, letter giving detailed comments on the first draft was not answered until mid-February, 1981) and because the consultant failed to propose candidates who were considered properly qualified for their intended assignments. Unfortunately, these delays led to some degree of friction between the consultants and GOSL on the one hand and the Bank on the other. The consultant felt that the Bank was intentionally slowing down the selection process because they were not aware that the Bank was sending its comments to the Government and the Government felt that the Bank was trying to exert too much control over the selection of staff, which GOSL thought should be a Government prerogative. However,the contract was signed in March, 1981,and the five advisors arrived in Sierra Leone between June and September of that year. The contract, as approved by the Bank, was for an amount not to exceed US$ 1.092,960. The Credit Agreement contained only $ 840,000 in Category 1, from which the consultant was to be paid.Thus, there was an initial and apparently unnoticed overrun of the Category by some $ 252,960. The first formal recognition of this was during the November, 1981, supervision. 5. Some seventeen (17) short term consultancies were eventually awarded under the'Project ranging from a review of the architectural plans for the new Ministry of Agriculture and Forestry building to the provision of two UN Volunteers for one year each to the Office of Statistics. However, with the exception of one contract awarded by MOW, one by MOP and two by MDEP, all of the short term consultancies were given through MAP. Indeed, out of some $ 620,000 disbursed from Categories 2 (short-term consultants) and S (project  -14- ANNEX 2 Page 3 of 5 pages feasibility studies) , more than eighty percent was spent through MAF. The contracts themselves were awarded to a variety of firms and individuals, both Sierra Leoneans and expatriates, over the life of the Project. C. Performance of the Consultants Long Term Consultants 6. The fact that the Government and IDA were at odds from the beginning concerning the award of the contract for long term advisors to the consultant quite probably served to sour the relationship between the consultant and Bank staff early on. Thus one has a tendency to view any negative criticism of the consultant's performance with a slightly jaundiced eye - at least initially. Nonetheless,it should be remembered that, while the performance of the individual long-term advisors under the contract was not entirely without merit, the overall performance of the consultant was not really up to the standard required. The five advisors were of uneven quality and none was ideal for his assignment. They all racked direct experience in the type of project preparation planning which was required under the Project and some were not even convinced of the relevance of overall plans for specific sectors. The consultant's backstopping and supervision of the individual experts was almost non-existent largely because the firm lacked any experience in macro/sector planning work. Further, the logistical/administrative support for the advisors from the consultant was also inadequate and resulted in frequent complaints fron. the advisors about prompt payment of salaries, housing, etc. Throughout the assignment, the consultant appeared to demonstrate little interest in project implementation or progress. True, in the "Final and Conclusive Progress Report" prepared by the consultant in October, 1983, the consultant stated that there were impediments to optimum working conditions during implementation and that these problems caused considerable delays. The statement is irrefutable, especially in the cases of MOW and MOE where the advisors lacked the minimum requirements of adequate counterpart staff and office facilities throughout. In addition to the lack of Government support, the advisors complained of the sometimes conflicting advice which they received from the Government, the consultant and IDA officials. Each of the advisors had to go through a process of acclimatization which included an acceptance of the seeming lack uf interest in and support of planning in general and an eventual re-definition of his job description to accept this :eality and still acconitish some diminished part of the original plan. The advisors achieved uneven results from this exercise, but the fault lies more in the project design, which assumed far more Government support than was there, and to the almost complete lack of technical back-up which they received from the consultant. Short Term Consultants 7. The performance of the short-term consultants was generally satisfactory. As noted in paragraph 5 above, more than 80% of these contracts by value were performed under the auspices of the Ministry of Agriculture and Forestry. The performance by the consultants under these contracts is discussed in Annex 6. Of the remainder, one contract was for the services of a long-term consultant to assist the Ministry of Works, another to employ two UN Volunteers to assist the Central Statistics Office, another to hire a consultant to assist in the review of the draft of the National Development Plan for MDEP and the last was was for the services of a Sierra Leonean economist to carry out a Public  ; - 15 - ANNEX 2 Page 4 of 5 pages Enterprises Study under the auspices of the Ministry of Finance. In all cases, the performance of the consultants was satisfactory and in line with thei.r TOR. Nonetheless, neither the Public Enterprises Study nor the review of the Development Plan was to receive the follow-up required. D. Counterpart Funding and Government Performance 8. As part of the Government contribution to the total Project costs, GOSL was to open a special project bank account with the sum of 70,000 Leones, which was to cover the running costs of the project vehicles and the expenses of the advisors when travelling out of Freetown. In addition, the Government was to reimburse the consultant at the rate of 10,000 Leones per year per advisor for housing expenses, a total of 100,000 Leones. GOSL was immediately and constantly in arrears on both of these amounts, which caused the consultant to take the extraordinary step of instructing the advisors to refrain from travelling up country. Finally, in order to at least make the advisors mobile, the October, 1982, supervision mission agreed to the reallocation of 70,000 Leones into the project account upon Government confirmation that the remaining 100,000 Leones (including some 32,000 which had already been paid) had indeed been deposited into the account. Until the Bank agreed to this reallocation the advisors (under instructicns from the consultant) were largely confined to Freetown. 9. Generally, the performance of the Government was disappointing. There are several factors which were partially responsible for this but, in the final analysis, one is forced to the conclusion that there was a fairly general lack of commitment to the goal of providing Sierra Leone with a rational planning mechanism for the medium term. Some of the secondary, but important, factors *which contributed to&the less than ideal performance on the part of GOSL were: (a) the replacement of the Development Secretary shortly after Board Presentation. Even had he been sincerely committed to planning, his replacement lacked any experience with the process by which the Project had been put together; (b) the decision to merge the Ministries of Development and Finance. This temporary (it lasted less than a year) merger diverted attention away from the crucial issues developing around the planning question at the time of Project start up; (c) the preoccupation with the parliamentary elections which lasted well into 1982 and caused attention to be fuither diverted from the planning process; (d) the failure to make MDEP the de facto as well as the de jure coordinating point for the several ongoing planning activities by not allowing it the necessary authority. Finally, the inability of the Government to retain the services of qualified economists and other professional staff in the CPU and in MDEP at large was probably a critical factor in the failure of the CPU to function properly. The question of salary and benefit leveis designed to attract and keep such staff remains unanswered despite a variety of suggested possible solutions. In  . 16- ANNEX 2 Page 5 of 5 pages general, the budgetary resources allocated to the Project were inadequate throughout and, at least to some extent, this was responsible for the Project's lack of success.  - 17- ANNEX 3 Page 1 of 3 pages III. Project Supervision 1.. The responsibility for overall project supervision of the Project ratnained somewhat blurred throughout. The International Relations Department iiad identified and appraised the Project in response to the Programs inspired idea and the first two supervision missions were carried out by IRD staff. In 1981, after the Regionalization of several IRD staff and responsibilities, responsibility was shifted to the Technical Assistance Unit of the Western Africa Projects Department. But, there has always been some confusipn about to whom to direct supervision reports. This situation, essentially harmless itself, was exacerbated, probably to the detriment of the Project, by the number and variety of informal, ancillary supervisory efforts occasioned by the number of different sectors involved and by staff changes within the Bank's divisions. This hodge-podge supervision caused concerns early in the Project since, whether correctly or not, the advisors felt that they were receiving frequently conflicting direction. And, on top of the Bank's own supervision missions (both formal and informal), the advisors were receiving instructions from the consultant's Overall Project Manager that purported to come from the Government. Concerns about these varying sources of direction for the advisors were expressed time and again in reports from returning missions and are cited as one of the excuses for any possible failure to achieve Project goals in the consultant's final report. 2. In view of the number of different sectors involved in the Project, the amount of staff time devoted to supervising it was not very large: Overall Supervision ( staffweeks ) FY80 FY81 FY82 FY83 FY84 FY85 TOTAL 6.5 1.1 10.5 20.3 9.3 5.6 53.3 Supervision by Sector ( staffweeks ) Tech. Assistance Programs Agriculture Transport Health Education Total