Papua New Guinea - Second and Third Highway Projects

Docrmrent of The World Bank FOR OFFICIAL USE ONLY PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGBLANDS ROADS IMPROVEMENT PROJECT (CREDIT 677-PNG) AND TBIRD HIGHWAY PROJECT (LOAN 1856- PNG AND CREDIT 1030-PNG) JUNE 30, 1993 Operations Evaluation Division This document has a restricted distribution and may be used by recipients only in the performance of their off~cialduties. Its contents may not otherwise be disclosed without World Bank authorization.  COUNTRY EXCHANGE RATES Name of Currency (Abbreviation): Papua New Guinea - Kina (K) Kina per USS1 Appraisal - Second Highlands Road Improvement Project Appraisal - Highway I11 Completion - Second Highlands Road Improvement Project Completion - Highway I11 Average since 1985 GLOSSARY OF ABBREVIATIONS ADT Average Daily Traffic (number of vehicles) DOT Department of Transport DOW Department of Works ERR Economic Rate of Return K Kina; PNG Currency Unit NPV Net Present Value OED Operations Evaluation Department OIDA Office of International Development Assistance PCR Project Completion Report PNG Papua New Guinea PAR Performance Audit Report RMCS Road Maintenance Cost Savings SAR Staff Appraisal Report VOCS Vehicle Operating Cost Savings FISCAL YEAR OF THE BORROWER January 1 - December 31  FOR OFFICIAL USE ONLY THE WORLD BANK Washington, D.C. 20433 U.S.A. Office of Director-General Operations Evaluation June 17, 1993 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Papua New Guinea Second Highlands Road Improvement P r o j e c t ( C r e d i t 677-PNG) and Third Highway P r o j e c t (Loan 1856-PNG and C r e d i t 1030-PNG) Attached i s t h e "Performance Audit Report on Papua New Guinea Second - Highlands Road Improvement P r o j e c t ( C r e d i t 677-PNG) and Third Highway P r o j e c t (Loan 1856-PNG and C r e d i t 1030-PNG)" prepared by OED. Both p r o j e c t s focussed on improving s e c t i o n s of t h e Highlands Highway. Their primary o b j e c t i v e was t o reduce t r a n s p o r t c o s t s and provide t h e C e n t r a l Highlands, where 40% of PNG's population l i v e s , w i t h a r e l i a b l e t r a n s p o r t connection. The Audit e s t i m a t e s t h e economic r a t e of r e t u r n of t h e Second P r o j e c t a t 4% mostly due t o poor economic a p p r a i s a l of t h e p r o j e c t and i n e f f e c t i v e road maintenance i n Simbu Province. The Audit e s t i m a t e s t h e r a t e of r e t u r n of t h e Third P r o j e c t a t 8% because of c o s t overruns i n c i v i l works and lower than expected t r a f f i c volumes -- by 1992 minus 392. Progress was made i n c e r t a i n p o l i c y a r e a s , such a s road u s e r charges and t r u c k i n g d e r e g u l a t i o n . L i t t l e was achieved i n t h e a r e a of t r a f f i c counting. Reversing t h e PCR-based e v a l u a t i o n s , t h e Audit r a t e s t h e two p r o j e c t s a s unsatisfactory overall, their i n s t i t u t i o n a l impact p a r t i a l and t h e i r sustainability unlikely. Attachment This docunent has a restricted distribution and may be used by recipients only in the their official duties. Its contents m y not otherwise be disclosed without World Bank authorization.  FOR OFFICIAL USE ONLY PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) TABLE OF CONTENTS Pane No . Preface .............................................................. i Basic Data Sheets ..................................................... iii Evaluation Summary ................................................... vii I. Background ....................................................... 1 A . Land Transportation ........................................ 1 B . Bank Assistance to Transport Sector ........................ 1 I1 . Preparation and Implementation .Second Highway Project ........... 2 A . Preparation ................................................ 2 B . Implementation ............................................. 3 111. Preparation and Implementation .Third Highway Project ........... 5 A . Preparation ................................................ 5 B . Implementation ............................................. 5 IV . Issues .......................................................... 7 A . Highway 11: Project AppraisalIEvaluation Methodology ...... 7 B . Economic Analysis and Documentation of SARs and PCRs ....... 9 C . Risk Analysis in SARs ...................................... 12 D . Physical Contingency ...................................... 13 E . Highway 11: Road Deterioration ........................... 13 V . Conclusions and Recammendations ................................. A . Conclusions ............................................... . 1 Overall Results ....................................... . 2 Bank Performance ...................................... B . Recommendations ........................................... Recommendation 1: On Rehabilitation and Maintenance of Chuave-Kundiawa-Garniger Road ........... Recommendation 2: On Risk Analysis in SARs ................ Recommendation 3: On Physical Contingency ................. Recommendation 4: On Completeness of Project File ......... This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization . Table of Contents (continued) Tables Civil Works Costs: Price Deflator Index. 1976-1986 ............. Second Highway Project: Actual Civil Works Costs 1977-1981 ..... Third Highway Project: Actual Civil Works Costs. 1980-1986 ..... Highlands Highway: Traffic Count Results for 1992 .............. Second Highway Project: Traffic and Traffic Composition. 1974-1992 Actual and 1999 Forecast .......................... Second Highway Project: Actual and Forecast Traffic. 1974-1999 . Third Highway Project: Traffic and Traffic Composition. 1979-1992 Actual and 2005 Forecast .......................... Third Highway Project: Actual and Forecast Traffic. 1979-2005 .. Highlands Highway. Vehicle Operating Costs and Savings .......... Second Highway Project: Reestimated ERRS and NPVs .............. Third Highway Project: Reestimated ERRS and NPVs .............. Second and Third Highway Project: Civil Works Cost Estimates. in Constant Prices. Ex Ante and Ex Post ..................... Second and Third Highway Project: Economic Results and Underlying Factors .......................................... Second Highway Project: Vehicle Operating Costs. Price Increase. 1974-1975 ........................................... - Vehicles and Spare Parts ................................ - Fuel .................................................... - PNG Inflation ........................................... - Vehicle Operating Costs ................................. Annex 1 Colmnents from the Region .................................... 34 Map IBRD 11428R2 (PCR)  PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT LOAN 1856-PNG AND CREDIT 1030-PNG) 1. This is the Performance Audit Report (PAR) of the Second Highlands Road Improvement Project, involving an IDA Credit of USS19.0 million, and of the Third Highway Project, involving a Bank Loan of USS17.0 million and an IDA Credit of USS13.0 million. The main objective of both projects was the same: to improve the Highlands Highway and provide the Central Highlands with a cheaper and more reliable transport connectionwith Lae. The Credit for the Second Highlands Road Improvement Project was approved January 28, 1977, and became effective on March 2, 1977. The Closing Date of June 30, 1981, was extended by one year. Final disbursement was made on June 10, 1982, and the balance of USS.5 million was canceled. The Loan and Credit for Highway I11 were approved May 27, 1980, and became effective November 6, 1980. The Closing Date, as scheduled, was December 31, 1984. Both Loan and Credit were fully disbursed and final disbursement was on November 7, 1984. 2. This PAR is based on the Project Completion Reports (PCRs) of the two projects prepared by the East Asia and Pacific Regional Off ice and issued in 1985 and 1986,' the President's Reports, the Credit and Loan documents, the transcripts of the Executive Directors' meetings at which the projects were considered, a study of the Bank's files, and discussions with Bank staff An OED mission visited PNG during the second half of November 1991 and discussed the . effectiveness of the Bank's assistance with OIDA, the Department of Works and the Department of Transport, all in Port Moresby ; with the Provincial Works Managers and staff in Western Highlands, Simbu, Eastern Highlands and Morobe province; and with several of the larger private road transport organizations and major transport users in the Highlands Highway area. The cooperation and assistance of PNG authorities and of the private sector in the preparation of this report are gratefully acknowledged. Subsequently, at the request of the Audit, the Department of Transport undertook, during 1992, a series of traffic counts on the Highlands Highway and made these available to the Audit. This effort, too, was very much appreciated indeed. ' Project Completion Report, Pa~uaNev Guinea, Second Highlands Road Im~rovement Prolect, (Credit 677-PNG), Report No. 5488, February 25, 1985, and Project Completion Report, Papua New GuineaL Third Rinhway Project (Loan 1856-PNG and Credit 1030-PNG), Report No. 6441, October 7, 1986. 3. The PCRs of the two projects discuss the project experience and the performance of the Bank and the Borrower. This audit report supplements the PCRs and highlights aspects such as the economic analysis and the risk analysis (ex- ante and ex-post) of the two projects and its documentation, the deterioration of two of the road sections improved under Highway 11, and the Bank's performance. Finally, the audit offers recommendations for consideration by the Government of Papua New Guinea and the Bank. 4. The East Asia and Pacific Regional Office of the Bank commented on the draft PAR. These comments, together with OED's responses, are attached (Annex 1). Following standard OED procedures, the final draft PAR was also sent to the Government for conrment but none were received.  PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) BASIC DATA SHEET KEY PROJECT DATA Appraisal Actual Expectation Estimate Total Project Costs (US$ million) Overrun (I) Credit Amount (US$ million) Disbursed Canceled Repaid Outstanding as of November 30, 1991 Date Physical Components Completed Economic Rate of Return (I) CUMULATIVE ESTIMATED AND ACTUAL DISBURSEMENTS Appraisal Estimate (US$ million) 1.7 6.6 12.0 16.7 19.0 - Actual (US$ million) 1.3 8.0 11.2 13.5 16.4 18.5 Actual as Z of Appraisal (I) 76 121 93 81 86 97 Date of Final Disbursement: June 10, 1982 PROJECT DATES Original Plan Actual Government Application Negotiations Completed Board Approval Credit Agreement Date Effectiveness Closing Date  STAFF INPUTS ( s t a f f weeks) FY73 - - FY75 - FY74 - FY76 - - FY79 FY77 FY78 - FY81 - FY8O -- - FY82 FY83 - - FY84 T otal Preappraisal 1.5 19.3 4.7 9.1 .8 35.4 Appraisal 18.9 3.9 6.8 29.5 Negotiation 6.5 6.5 Supervision 4.2 3.3 8.9 4.0 8.1 1.5 .1 3.3 33.5 Other 7.6 3.5 6.4 17.6 Total 1.5 19.3 31.2 16.5 24.6 3.3 8.9 4.0 8.1 1.5 .I 3.3 122.4 MISSION DATA Date No. of Man- Date of - Item (mo./yr.) Persons weekr, Report Identification 4/72 Preparation 5/73 Preappraisal 3/74 Appraisal 8/74 Total Supervision I Supervision 1 1 Supervision 1 11 Supervision IV Supervision V Supervision VI Supervision VII Completion Total OTHER PROJECT DATA Borrower: The Independent State of Papua New Guinea Executing Agency: Department of Transport, Works and Supply Follow-On Proiect: Name : Third Highway Project LoanlCredit Number: Loan 1856-PNG and Credit 1030-PNG Amount (US$ million): Total 30.0 (Loan 17.0; Credit 13.0) Approval Date : May 27, 1980  PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) BASIC DATA SHEET KEY PROJECT DATA Appraisal Actual Expectation Estimate Total Project Costs (US$ million) Overrun ( % ) Loan and Credit Amount (US$ million) Disbursed Repaid Outstanding as of November 30, 1991 Date Physical Components Completed Economic Rate of Return (I) CUMULATIVE ESTIMATED AND ACTUAL DISBURSEMENTS Appraisal Estimate (US$ million) 9.3 18.0 26.0 29.5 30.0 Actual (US$ million) 6.0 9.0 15.2 26.3 30.0 Actual as I of Appraisal ( X ) 65 50 58 89 100 Date of Final Disbursement: November 7, 1984 PROJECT DATES Original Plan Actual First Mention in Files 8/27/78 Negotiations Completed 41 2/80 41 2/80 Board Approval 5/27/80 5/27/80 LoanlCredit Agreement Date 9/16/80 9/16/80 Effectiveness 111 6/80 111 6/80 Closing Date 12/31/84 12/31/84 STAFF INPUTS rsrair weeks) Preappraiaal Appraieal Negotiation Su e m i s i o n orRer Total  MISSION DATA Date No. o f Man- Date of - Item (mo./vr.) Persons weeks Report Preparation 10/78 2 2.0 11/13/78 Preappraisal 5/79 2 4.0 5/31/79 Appraisal 10179 2 -6.0 11/15/79 Total 12.0 Supervision I Supervision 11 Supervision 111 Supervision IV Supervision V Supervision VI Supervision VII Supervision VIII Completion Total OTHER PROJECT DATA Borrower: The Independent State of Papua New Guinea Executing Agency: Department of Works and Services Follow-On Project: Name : Road Improvement Project Loan Number: 2265-PNG Amount (US$ million): US$31.0 Approval Date : April 19, 1983  PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) EVALUATION SUMMARY 1. Introduction 2. Obiective Since 1970, five years before PNG All four road projects focussed on became independent, the Bank has extending and improving the Highlands assisted the transport sector with Highway. Their primary objective was seven projects; four road projects, to reduce transport costs and provide two port projects and one transport the Central Highlands, where 40% of project. Three of the road projects PNG1s population lives, with a and the two port projects have been reliable transport connection with completed; one road project and the Lae (PAR, para. 4 ). transport project are nearing completion. No new transport 3. Hinhwav 11: Preparation, projects are under active Implementation. Results consideration at present by the Bank (PAR, para. 3). This project's appraisal was drawn out over a two-year period. In late PNG1s land transport system 1974, a Bank mission visited PNG to consists of a number of single roads appraise the upgrading of 12 and separate road networks. Roads sections, totalling 136 km, of the are primarily located in and near the Highlands Highway from gravel to main coastal towns and in the Central paved standards and for which the Highlands (PAR, para. 1). feasibility and detailed engineering had been prepared (PAR, paras. 8-12) . The longest road in PNG is the Highlands Highway. It was built By mid-1975, as the result of two gradually, starting from Lae, the successive rounds of bidding, project main port on the east coast of cost estimates had escalated more Guinea, and stretches 600 km west than 100% and the project's economic (see map). It traverses Morobe viability came into serious question province (with Lae as capital) and (PAR, paras. 13 to 16). the five provinces which make up the Central Highlands: Eastern Highlands, Simbu Western Highlands, Southern Highlands and Enga province (PAR, para. 2).  Subsequently, in late 1976, the negotiations in April 1980 (PAR, Government and the Bank agreed to para. 32) drastically reduce the project: upgrading of two sections of the The project consisted of Highlands Highway in Simbu province, improvement and paving of four road Chuave-Kundiawa (27 km) and Kundiawa- sections of the Highlands Highway: Garniger (23 km); and one section in Daulo Pass-Goroka (22 km); Goroka- Eastern Highlands, Asaro-Watabung ( 18 Henganofi (45.5 km); Henganofi- km) .The ERR for the entire project was expected to be 27.5I and for Kainantu (38 km); Kainantu-Kassam Pass (32.3 km); or in total 138 km. Chuave-Kundiawa, accounting for some 75% of project cost, 25% (PAR, paras. In addition, some technical 17 and 18). Other components of the assistance for road maintenance and project included studies and preparation of rural road projects technical assistance (PAR, para. 19) . was included, plus detailed engineering and some traffic count Project implementation was delayed equipment (PAR, para. 33). by about one year and the cost overrun was about 10% (PAR, para. The project was approved by the 21). Board on May 27, 1980, and became effective November 6, 1980 (PAR, The Audit inspected the three road para. 34). sections improved under the project. A large part of the two road sections As in the previous project, the in Simbu province has reverted to project was completed with a limited gravel standard or worse and requires delay of about one year. The cost rehabilitation. This condition overrun this time, however, was reportedly has existed since 1985 or considerably higher: 50% (including 1986, not long after completion of all contingencies) in terms of local the roads in 1981, and is primarily currency. This cost overrun due to ineffective road maintenance. reflected mostly considerable The third section, Asaro-Watabung, additional excavation (PAR, para. was in good condition, well 35). maintained (PAR, paras. 27 to 31). Construction progress was slwed The Audit has reestimated the ERR down by a number of factors such as: of Highway I1 at 4% (Table 10). heavy rainfall, land acquisition problems, contractor was denied The studies and technical access by the local population to the assistance components of the project work site and to borrow and gravel were by and large properly executed pits, unstable soil conditions albeit with some delay (PAR, paras. requiring major additional and 23 to 26). unforseen earth work, civil unrest and security problems and labor 4. Hiphwav 111: Preparation, strife. All this also explains the Implementation, Results considerable cost overrun (including supervision) of some 50%. The Audit While Highway I1 was being visited the four road sections and implemented, Highway I11 was being found them in good condition, well prepared. In the fall of 1979 the maintained (PAR, para. 37). project was appraised followed by  The Audit reestimated the (ERR = 4%) was the very costly and -- project's ERR at 8% (Table 11). in the opinion of this Audit -- wrong decision to build the Chuave-Kundiawa With some delays the other section for 75% on a new alignment components of the project were (i.e. from scratch) instead of implemented satisfactorily except for improving the existing road (PAR, the traffic counters which did not para. 69). function well (PAR, paras. 38 to 41). This decision, in turn, was Highway I11 also included a number supported by faulty economic analysis of conditions to be fulfilled by the (PAR, paras. 43 to 49) and Borrower. These included: considerably and unrealistically inflated benefits (PAR, paras. 50 to - establish a system of regular 58). traffic counting; this was not accomplished (see also previous More generally, the Audit paragraph) but efforts are attributes the poor results of being made by DOT to establish Highway I1 to poor and biased such a system; preparation and appraisal of the economics of the project (PAR, paras. - increase road user charges to 43 to 58). cover cost of road maintenance; this was accomplished; In addition, but secondarily, the ineffective road maintenance in Simbu - deregulate trucking industry; Province contributed to the project's this was accomplished "de poor results (PAR, paras. 27 to 30, facto", entry to the industry 67 and 76 to 79). and tariff setting ate free (PAR, para. 42). In the case of Highway 111, the two principal reasons for the 5. Outcome disappointing outcome (ERR = 8%) are the 33% cost overrun in civil works Given the reestimated rates of costs in real terms plus the lower return of the two projects, 4% and than expected traffic volumes on the 8%, (Tables 10 and 11) the project project roads: by 1992 minus 39% outcome cannot be considered (PAR, para. 70). satisfactory. Also, given these rates of return, the projects do not As to the policy and institution produce a satisfactory level of building impact of the projects, benefits and hence are not considered progress was made in certain areas. sustainable. The PCRs of the two For example, road user charges were projects indicated ERRS of 16% and considerably increased and now exceed 17% and based on the PCRs the road maintenance costs. Also, the projects were considered satisfactory trucking industry, de facto, was and sustainable (PAR, para. 80). deregulated and entry and rate setting are free. On the other hand, 6. Conclusions and Recommendations in the area of traffic counting progress has been limited (PAR, para. Overall Results 42). Overall, the institution building objectives were partially In the case of Highway 11, the achieved (PAR, para. 86). principal reason for this outcome Bank Performance The Bank's economic analysis in makers(the Borrower, the Bank's the appraisal of Highway 11 was Board, the Bank's Management) with a clearly inadequate. The methodology completely misplaced sense of was faulty (PAR, paras. 43 to 49), reassurance and comfort for which the SAR failed to established a clear there is no basis and no need. SARs linkage between the civil works cost should provide realistic estimates of estimate in the project cost table the project's downside propensity and the corresponding costs in the based on experience and keeping in economic analysis (PAR, paras. 51 and mind that overall 30 to 40% of Bank- 54). benefits in£lated (PAR, paras. financed projects do fail in practice 51-52 and Table 14), essential (see OED's Annual Reviews). The information (traffic counts) not Audit observes in this context that collected (PAR, para. 56), and the risk analysis in the two PNG obvious inconsistencies or concerns projects is not atypical (PAR, paras. simply ignored (PAR, paras. 50 and 72 to 74). 57). The Audit believes that a satisfactory economic analysis of the Similarly, but by no means as project would have affected the dramatically, the physical project design -- in particular for contingency allowed for in both Chuave-Kundiawa - - and might well projects (and in their predecessor, have resulted in a satisfactory Highway I), was clearly insufficient. overall project return (PAR, para. Here too a reality check is in order. 69). (PAR, para. 75). I n Highway 111 -- as in Highway I1 The bottom line is that both the -- the SAR failed to provide a link economic analysis and the risk between the financial and economic analysis, in both projects -- as costs of civil works; an appropriate reflected in SARs and PCRs -- comment at the time was ignored (PAR, systematically overestimated the para. 62). project outcome and systematically underestimated the downside risk. As to the economic analysis in the And this, in turn, harms the Bank's PCRs of the two projects, the Audit image of an impartial provider of has to reject and dismiss the quality professional advice (PAR, analysis of the Highway I1 PCR para. 93). because of obvious and glaring implausibilities (PAR, paras. 59 to Finally, the documentation in the 61). Bank' s proj ect files is inadequate : important information that should be Similarly, the Audit concludes there, is not (PAR, paras. 47, 53, 58 that the analysis of the Highway I11 and 60). PCRpresentedincorrect, inconsistent and inflated rates of return (PAR, Flowing from the project paras. 63 to 65). experience of Highways 11 and 111 this Audit offers four The risk analysis of both projects recommendations. provided biased, entirely unrealistic and misleading conclusions. The Recommendation 1: On Rehabilitation Audit believes that this type of risk and Maintenance of Chuave-Kundiawa- analysis provides the decision Garniger Road:  That the Government in its contingency normally used in road rehabilitation consider including project appraisal and, if found spot improvements where inadequate, provide appropriate economically justified and, to guidance to staff possibly help ensure the sustainability of differentiating between projects the rehabilitation, take steps to on the basis of different improve future maintenance, for geological and topographical example, through direct conditions encountered (PAR, involvement of local contractors, paras. 75, 92, 93 and 98). if necessary, assisted by the Government (PAR, paras. 76 to 79 Recommendation 4: On Completeness of and 96). Project File: Recommendation 2: On Risk Analysis That the Bank consider what steps in SARs : can be taken to help ensure that the Bank's project file on That the Bank make an effort to completion of a project is inject a sense of realism in its complete. One possible approach sensitivity analysis by requiring would be to entrust this SARs to report on previous responsibility to those relevant project experience in responsible for the preparation of quantitative terms as a basis for the project's PCR and have this a forward looking assessment of aspect reported on in the PCR the risks involved (PAR, paras. 72 (PAR, paras. 47, 53, 58, 60, 94 to 74, 91, 93 and 97). and 99). Recommendation 3: On Physical Contingency: That the Bank review the adequacy of the conventional 10% physical  PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) I. BACKGROUND A. Land Transportation 1. PNG's land transport system consists of a number of single roads and separate road networks. Roads are primarily located in and near the main coastal towns and in the Central Highlands. 2. The longest road in PNG is the Highlands Highway. It was built gradually, starting from Lae, the main port on the east coast of Guinea, and stretches 600 km west. It traverses Morobe province (with Lae as capital) and the five provinces which make up the Central Highlands: Eastern Highlands, Simbu, Western Highlands, Southern Highlands and Enga province (see map). B. Bank Assistance to Transport Sector 3. Since 1970, five years before PNG became independent, the Bank has assisted the transport sector with seven projects; four road projects, two port projects and one transport project. Three of the road projects and the two port projects have been completed; one road project and the transport project are nearing completion. No new transport projects are under active consideration at present by the Bank. 4. All four road projects focussed on extending and improving the Highlands Highway. Their primary objective was to reduce transport costs and provide the Central Highlands, where 40% of PNG'B population lives, with a reliable transport connection with Lae. 5. The First Highway Proiect (1970-1974) mainly covered construction and reconstruction of some 150 km of gravel road west of Kundiawa; see map). It was completed in 1974 and OED prepared an audit report in 1977'. 6. The final construction costs were 90% higher than the appraisal estimate. But because of traffic growing faster than anticipated and because of rising fuel cost the reevaluated economic rate of return was above the ERR estimated at appraisal. - - - - - - 2 , Papua New Guinea. First Highway Prolect, Report OED, Projecc performance Audit Report (PPAR) Number 1635,June 16, 1977. 7. , The Second Highway Prol ect ( 1977-1982) and its successor the Third Hiphway Project (1981-1984). are the subject of this Audit and discussed below. The fourth road project, Road Improvement Proiect (1983-19--), is still ongoing. 11. PREPARATION AND IMPLEMENTATION SECOND HIGHWAY PROJECT A. Preparation 8. While Highway I was being implemented, preparation of Highway I1 got under way in the early seventies. During preparation, in 1972, it was agreed that the project should focus on improving the Waterais-Kundiawa section of the Highlands Highway. (The Asian Development Bank had expressed an interest in financing improvements to the Lae-Waterais section.) 9. The Government then engaged the Australian consultant who before had done the feasibility study to carry out the detailed engineering. In addition, he was to include further study and detailed engineering for any additional improvement needed west of Kundiawa, direction Togoba. 10. In late 1974, upon completion of the consultant's work and after discussion between the Government and the Bank, bids were invited for improvement of 316 km of the Highlands Highway between Waterais and Togoba. 11. Also in late 1974 an appraisal mission visited PNG and produced in March 1975 a Yellow Cover Staff Appraisal Report (SAR). 12. The project was to cover 316 km of improvement and sealing of 12 road sections between Waterais and Togoba. Total costs was estimated at Kina 23 million (US$ 31 million) and the ERR at 1 4 2 . ~ 13. Shortly after the Yellow Cover SAR had been issued (para. 11) the construction bids came in. They were 80% above the estimates. 14. The government rejected these bids. Following discussions with the Bank eight kilometers was deleted from the project. Some relatively minor other changes were made and new bids invited in May 1975 in the hope that lower costs would result. 15. In July 1975 the Green Cover SAR was issued taking into account the bid results from April (i.e. the cost increase of 80%). The benefits were also revised and the ERR was estimated at 122.) 3 Excluding time savings of passengers. 16. Shortly after the Green Cover SAR had been issued the new bids came in. They were 13% higher than the previous (April 1975) bids and more than twice the original cost estimates. The economic viability of the project then was very seriously questioned in the ~ank.' 17. Subsequently, further discussions tookplace between the Government and the Bank and the scope of the project was drastically reduced. In these discussions too, at least initially, uneasinesewiththe project's economic justificationwas expressed. For example, in a meeting with a representative of the Government of PNG (September 2, 1975), Bank staff observed that "in view of the substantial jump in construction costs which took place over the last two years, we believe that the economic justification of the project based on vehicle operating cost savings alone may not be found to be satisfactory. "'During negotiations, in December 1976, it was agreed to include only three of the 12 road sections to be upgraded and sealed: As aro - Watabung (18 km); Chuave - Kundiawa (27 km); Kundiawa - Garniger (23 km); or a total of 68 km. 18. In addition it was agreed that the works would be mainly undertaken by contractors but that a part would be done by force account to reduce the costs. Total costs, excluding supervision and contingencies, on the basis of bids received (October 1976), was estimated at Kina 14.2 million (USS17.1 million) and the overall ERR at 27.5%. The ERR for the Chuave-Kundiawa section, accounting for some 70% of total cost, was estimated at 25%. A comparison between the bids received in late 1976 and those reflected in the Green Cover SAR indicates that the 1976 bids were about 10% lower in real terms (see project file, internal Bank memo, 9/27/78, page 2). 19. In addition to consultancy for construction supervision and management, it was agreed that the project would include (i) Rural Development Studies for Enga and Simbu province, (ii) some road feasibility and detailed engineering studies and (iii) technical assistance to local civil engineering contractors. These three items would require same nine man years of inputs and account for USS.7 million (excluding contingencies) or 3% of total project costs. 20. The Board approved the project on January 25, 1977 and it became effective on March 2, 1977. B. Implementation 21. The PCR describes the implementation phase of the project well. The project was completed with a limited delay of about one year and the cost overrun, overall, was limited to about 10% in terms of local currency. 4 S e e , f o r example, an i n t e r n a l Bank memo, July 15, 1975, i n t h e p r o j e c t f i l e . I n t e r n a l Bank memo, September 4 , 1975, i n t h e p r o j e c t f i l e . 22. The most important part of the project, the upgrading and sealing of the Chuave-Kundiawa road section -- accounting for 78% of total actual construction cost under the project -- encountered serious problems during construction. This was primarily because of unstable soils and steep slopes causing frequent slides or requiring costly additional earthworks to reduce slopes and hence the frequency of landslides. 23. As to the feasibilityldetailed engineering (Waterais-Togoba) retroactively financed under the project, the consultants had been allowed a short period of 11 months to do the detailed engineering. As a result they were not able to carry out an extensive investigation of the very difficult soil conditions. With more time available the detailed design could have been improved. However, the Audit does not feel that this was one of the more important shortcomings of the project. The difficult geological conditions were well known, the civil works were satisfactorily executed, and the cost overrun was not very high (some 30% in real terms; see Table 12). As to the difficult geological conditions, the PCR of the Second Highway Project gives a first rate and accurate description of the problems encountered and concludes that it was often virtually impossible to predict or locate these problems before roadway excavation took place. And this, in turn, has implications for determining appropriate physical contingencies at the project appraisal stage as discussed elsewhere (see paras. 75 and 98). 24. As to the feasibility study and detailed engineering for the Togoba- Wapenamanda Road, the feasibility study was somewhat delayed at the request of the government and the detailed engineering was included in the Third Highway Project (Loan 1856-PNG). The updating of the Waterais-Togoba Study of 1973 was also satisfactorily completed albeit with nearly two years delay. 25. As to the two rural development studies (for Enga and Simbu), they were properly executed and completed in 1982. The one for Enga formed a basis for the . Rural Provincial Development project (Loan 2125-PNG) There was no follow-up on the Simbu province study given the difficult conditions in that province. 26. Finally the Technical Assistance to civil engineering contractors covered a number of courses to some 20 equipment-for-hire contractors. The consultant, in addition, provided two manuals: one for civil engineering contractors and one for plant hire contractors. 27. The Construction Results. In October 1991, the Audit, in the company of an engineer, inspected in detail the road between Chuave and Kundiawa (28 km) and . between Kundiawa and Garniger (23 km) The condition of the two roads was very disappointing indeed. 28. The Chuave-Kundiawa section requires rehabilitation over 45% of its length: 29 subsections, totalling 8.9 km, have reverted to gravel standard while 19 subsections, totalling 3.7 km, show major failureldamage andlor manyllarge potholes. 29. The Kundiawa-Garniger section requires rehabilitation over 23% of its length: 22 subsections, totalling 4.7 km, have reverted to gravel standard while two subsections, totalling .6 km, show major pavement failure. 30. According to several users of the roads and road maintenance engineers familiar with the history of the roads, the above described road conditions have existed since 1985 or 1986, that is four to five years after completion of the roads. 31. The Asaro-Watabung road section, in Eastern Highlands province, was in good condition and well maintained. 111. PREPARATION AND IMPLEMENTATION THIRD HIGHWAY PROJECT A. Preparation 32. While the Second Highway Project was being implemented, the Third was being prepared. In the fall of 1979 the appraisal took place followed by negotiations in April the next year. 33. The agreed upon project consisted of: (i) improvement and paving of four road sections of the Highlands Highway : Daulo Pass - Goroka (22 km); Goroka -Henganofi (45.5 km); Henganofi - Kainantu (38 km); Kainantu - Kassam Pass (32.3 km); or in total 138 km; (ii) technical assistance for road maintenance; (iii) technical assistance for preparation of rural road projects; (iv) detaiied engineering Togoba-Wapenamanda Road; (v) some traffic count equipment. 34. The project was approved by the Board on May 27, 1980 and became effective November 6, 1980. 35. The PCR describes the project implementation well. As in the previous project (see para. 21 above) the project was completed with a limited delay of about one year. The cost overrun this time, however, was considerably higher: 48% (including all contingencies) in terms of local currency. This cost overrun reflected mostly considerable additional earthworks (excavation). Total actual project costs were Kina 30.2 million (U$44.7 million). 36. The most important element of the project, upgrading and sealing -ofthe 138 km Daulo Pass-Kassam Pass section of the Highlands Highway, accounted for 90% of total actual project costs. 37. Construction progress was slowed down by a number of factors such as: heavy rainfall, land acquisition problems, contractor was denied access by the local population to the work site and to borrow and gravel pits, unstable soil condit:ions requiring major additional and unforseen earth work, civil unrest and security problems and labor strife. All this also explains the considerable cost overrun (including supervision) of some 55%. The Audit visited the four road sections and found them in good condition, well maintained. 38. As to the technical assistance for the road maintenance, in 1981 Danish consultants were initially selected. But after lengthy bureaucratic wrangling about how the assistance was to be phased, a British firm was finally selected in early 1983. This firm first identified the road maintenance needs and formulated a series of recommendations on h w to improve the road maintenance. This was followed by the firm providing two maintenance experts to assist in setting up a maintenance which to improve the efficiency of road maintenance. This project component was completed in mid 1985 and probably has made a modest contribution to road maintenance in PNG. Some of the consultant's proposals are reflected in the ongoing Transport Improvement Project (Loan 2742-PNG). 39. As to the technical assistance for rural road transport, this started in 1981. A consultant team identified a number of rural roads. This was followed by more detailed engineering. Some of these roads were ultimately financed under the ongoing Road Improvement Project (Loan 2265-PNG); others were undertaken directly by the provinces themselves. 40. The detailed engineering of the Togoba-Wapenamanda Road was undertaken by the Department of Works itself and not financed by the Bank. 41. Some traffic count equipment was indeed procured under the project (Kina 30,000) but, according to the Department of Transport, did not perform satisfactorily. Other counters are n w being tested and a counting unit exists in DOT responsible, inter alia, for maintaining an inventory of traffic counts. 42. Highway 111 also included a number of conditions to be fulfilled by the Borrwer. These included: - establish a system of regular traffic counting; this was not accomplished (see previous paragraph) but efforts are being made by DOT to establish such a system; - increase road user charges to cover cost of road maintenance; this was accomplished; - continue training of DOW staff; this was done, formal and on-the job; - deregulate trucking industry; this was accomplished "de facto", entry to the industry and tariff setting are free; from discussions with government officials in Port Moresby and with transport companies and  transport users in the Central Highlands, the Audit got the firm impression that competitive conditions are prevailing in the trucking industry. More difficult to answer is the question to what extent the Bank's conditionality was a sine qua non for the changes that did take place. The Audit's impression is that the Bank's conditionality and ongoing dialogue with the Government were important - though not the only-contributing factors which led to the increase in road user charges and the deregulation of the trucking industry. IV. ISSUES A. Highwav 11: Project AppraisallEvaluation Methodology 43. During project preparation, in 1974, the question came up whether the Chuave-Kundiawa section should be realigned (i.e. reconstructed on a different alignment) or whether the existing road should be improved. 44. The consultant suggested the more costly realignment in the 1973 feasibility study. The rationale was that better horizontal alignment (reduced curvature) would enable the use of larger trucks, specifically semi-trailers, and hence the capture of the corresponding benefits of economies of scale. The benefits would not be limited to those generated on the Chuave-Kundiawa section alone; the other project sections as well would generate these benefits once the realignment of Chuave-Kundiawawould be completed enabling long distance through- traffic on all sections for the semi-trailers. Essentially the Chuave-Kundiawa section was considered the bottleneck for the use of semi-trailers for the whole project (on all sections). 45. The Bank, apparently not entirely convinced, asked the consultant for a separate report on the matter. This became available in May 1 9 7 4 . ~ 46. The report stated (on page 5) that: "A preliminary study of the 1973 and 1974 [traffic] count s h w s that notwithstanding the [recent] improvements of DPW (Department of Public Works) to Daulo Pass, the commercial traffic composition beyond Goroka is still predominantly light to medium trucks, with no semi-trailers. Until the [Chuave-Kundiawal link is upgraded the change in traffic composition will not take place. It follows therefore that any benefits accruing to traffic on [the other project links] because of the [Chuave-Kundiawa] link, should all be credited to [Chuave-Kundiawa]." [Underlining added.] Vallentine, Laurie and Davies, Highlands Highway Waterais-Togoba, Re-evaluation of Link No. 8 (Chuave-Kundiawa), May 1974. 47. Upon receipt of the consultant's report, the Bank wrote (July 16, 1974) the Department of Public Works as follows: "We have briefly reviewed the report, but we feel that we need more information before we can express a clear opinion. We suggest, therefore, that the [August 19741 Appraisal mission review the matter with you and the consultants" Terms of Reference for this mission are in the project file and the mission did take place. However, the project file does not include a Back-to-Of£ice report or any reference to the review referred to above. Apparently the mission did agree with the consultant's conclusion that Chuave- Kundiawa should be realigned and the issue is not further mentioned in the Issues paper of October 17, 1974. 48. The Audit, in this context, observes that the consultant, in the 1973 feasibility study, indicated for 1972 - based on his traffic count - one semi- trailer per day on both the Asaro-Watabung section (this includes the Daulo Pass) and the Chuave-Kundiawa section. The consultant's corresponding figure for 1974 -- based on his traffic count for that year -- was two semi-trailers.7 And for 1975, the Department of Transport undertook a traffic count and reported eight semi-trailers per day both on Chuave-Kundiawa and Kundiawa-Minj The foregoing .' clearly indicates that-contrary to what was stated in para 46 above (see underlined portion) - the Chuave-Kundiawa section could and did in fact handle (significant) semi-trailer traffic before any upgrading had taken place. This information, at the time, was available to the Bank but was ignored in the economic analysis of the project in the final (January 1977) SAR of Highway 11. 49. Given this evidence the Audit concludes that the Government's effort to increase the capacity of the Daulo Pass had been effective and that indeed the Daulo Pass and not the Chuave-Kundiawa section had constituted the obstacle to the use of semi-trailers. Hence, the Audit has to reject the methodology attributing a change in vehicle mix (a shift to larger vehicles, i.e. semi- trailers) to the project and ascribing the corresponding benefits (scale economies resulting from the use of larger vehicles, i.e. semi-trailers) of the various sections to the Chuave-Kundiawa section. In other words, the Audit has to reject the notion of two different traffic mixes or forecasts: one "with" and one "without" the project. This, of course, reduces both the overall rate of return of the project, and of the Chuave-Kundiawa section in particular, both ex- ante and ex-post. For example, the earlier consultants report referred to before (see para. 45) indicates that of the total annual benefits of the Chuave-Kundiawa section some 50% is attributable to the shift to larger trucks. Including these benefits gave an ERR of 13%; excluding them gave an ERR of 5.5%. Also, the Green Cover SAR (see para. 15) indicated for the Chuave-Kundiawa road section an ERR of 32 excluding the benefits from a shift to larger trucks. 7 V a l l e n t i n e , Laurie and Davies, Highlands Highway Waterais-Togoba, T r a f f i c Study, Hay 1974, Table 14. Department o f Transport, Highlands Hlghway, Chuave-Minj Upgrade, December 1975, page 5 6 , Table 6.3. B. Economic Analysis and Documentation of SARs and PCRs 50. Green Cover SAR of Hinhwav 11. The Green Cover S AR of Highway I1 ( J u l y 1975) -- which r e f l e c t e d t h e 802 i n c r e a s e i n p r o j e c t c o s t s ( s e e p a r a . 13 above) - - s t i l l managed t o show an o v e r a l l ERR of 12% ( s e e p a r a . 15 above) compared w i t h a 14%ERR i n t h e Yellow Cover S AR (March 1975) based on and r e f l e c t i n g t h e August 1974 c o n s u l t a n t ' s r e p o r t . 9 51. A s t o t h e c i v i l works c o s t s , on t h e b a s i s provided i n t h e SAR's p r o j e c t c o s t t a b l e , t h e "economic" c o s t s should have been K 40.6 m i l l i o n ( b a s e c o s t s p l u s 10% c o n t i n g e n c y ) . lo However, only K 38.4 m i l l i o n was used i n t h e ERR c a l c u l a t i o n . A p o s s i b l e e x p l a n a t i o n f o r t h e 6% discrepancy might be t h a t a t a x element was deducted b u t t h e S A R provides no i n f o r m a t i o n on t h e t a x element a p p l i c a b l e t o p r o j e c t c o s t s . A s t o t h e b e n e f i t s , t h e b e n e f i t s s t r e a m (mainly v e h i c l e o p e r a t i n g c o s t s a v i n g s ) from t h e Yellow Cover S A R were i n A p r i l 1974 p r i c e s and had t o be expressed (updated) i n February 1975 p r i c e s , i n l i n e w i t h t h e c i v i l works c o s t s . They were i n c r e a s e d by no l e s s t h a n 60.2%. 52. The Audit made a r e c a l c u l a t i o n of t h e ERR i n t h e Green Cover S A R taking i n t o account ( i )c i v i l works c o s t s K 38.4 m i l l i o n assuming a 6% t a x element was indeed deducted and ( i i ) a p r i c e adjustment f a c t o r f o r t h e b e n e f i t s of +22%, e s t i m a t e d by t h e Audit ( s e e Table 14) t o r e f l e c t a r e a l i s t i c p r i c e i n c r e a s e from A p r i l 1974 t o February 1975. The r e s u l t i n g ERR i s 8.3% and confirms t h e A u d i t ' s i n i t i a l r e a c t i o n t h a t t h e Green Cover SAR's ERR r e s u l t was t o o high. 53. The u n d e r l y i n g m a t e r i a l , by c o n s u l t a n t s , f o r t h e economic a n a l y s i s of t h e p r o j e c t -- though r e f e r r e d t o i n t h e S A R (Table 13) and i n t h e p r o j e c t f i l e " -- was not a v a i l a b l e i n t h e p r o j e c t f i l e . 54. F i n a l SAR of Hinhwav 11. The Audit was t r o u b l e d by a number of f a c t s . F i r s t , -- a s i n t h e Green Cover S AR -- t h e c i v i l works c o s t s i n t h e economic a n a l y s i s of t h e f i n a l SAR (January 1979) showed a discrepancy w i t h t h e b a s e c o s t s from t h e p r o j e c t c o s t t a b l e p l u s 10% p h y s i c a l contingency. T h i s time t h e d i f f e r e n c e was 10%. Again, t h e S A R was s i l e n t on t h e t a x a s p e c t s . 55. Second, t h e methodology of t h e Green Cover S AR ( p a r a . 49 above) was a l s o used i n c a l c u l a t i n g t h e b e n e f i t s i n t h e f i n a l SAR. This f a u l t y methodology, i n combination w i t h t h e u n d e r s t a t e d c i v i l works c o s t s r e s u l t e d i n a s y s t e m a t i c a l l y b i a s e d and i n f l a t e d r a t e of r e t u r n of t h e p r o j e c t . ' V a l l e n t i n e , Laurie and Davies. Highlands Highway, Waterais-Togoba, Project Evaluation, August 1974. 10 P h y s i c a l contingency allowances r e f l e c t expected i n c r e a s e s i n the base c o s t e s t i m a t e s o f a p r o j e c t due t o changes i n q u a n t i t i e s and methods o f implementation. Physical contingencies represent t h e estimated c o s t of the expected a d d i t i o n a l r e a l resources required and therefore should be included i n the economic a n a l y s i s o f a l l p r o j e c t s . See Central P r o j e c t s Note 3 . 1 1 , February 25. 1982, paras. 8 and 2 6 . " Telex from t h e Bank t o the Department o f Public Works, PNG, o f June 23, 1975, i n t h e p r o j e c t file . 56. Third, in spite of the crucial role of traffic counts in the appraisal of road projects, the traffic counts used in the justification of the project were nearly three years out of date. See also para. 48 above illustrating the importance of up-to-date traffic counts particularly in the context of Highway 11. 57. Fourth, the SAR did not attempt to explain or reconcile in some detail why (how) its ERR estimates were consistent with those of the Green Cover SAR for each of the road sections. This in particular, because at the Green Cover SAR stage the economics of the project had been seriously questioned (para. 16 above) while the final SAR's overall ERR of 282, on the other hand, suggested a quite attractive economic proposition. 58. And fifth, the consultant's Economic Evaluation Report of November 1976, which underlies the economic analysis of the final SAR of January 1977 -- though referred to in the project file (two copies were sent to and acknowledged by the Bank in November 1976) -- was not in the project file. 59. PCR of Highway 11. The PCR estimates the project's ERR at 15.9%. The Audit was unable to reconcile the PCR's ERR cost and benefit streams (PCR, Tables 6, 7 and 8) with the PCR's underlying civil works costs (PCR, Table 2) and vehicle operating costs (PCR, Table 3) in combination with the PCR's traffic count information (PCR, Tables 4 and 5 ) . One example, for the cost element, may illustrate the extent of the discrepancy involved: Asaro-Watabunq ERR Construction "Costs" Costs (PCR Table 6 ) (PCR Table 2 1 (K 000) 1976 13,410 --- 1977 6,128 937 1978 4,200 803 1979 5,325 227 1980 7,537 --- 60. Also, the PCR's benefit streams are out of line with the SAR's. For example, for 1985, the PCR's benefits for Kundiawa-Garniger (PCR, Table 8) are seven times those in the SAR; for Chuave-Kundiawa they are less than half of the SAR' s (PCR, Table 7). As to documentation, the Audit was unable to find any information in the project file underlying the PCR's ERR calculation. Nor was the Audit able to identify who had undertaken this calculation. 61. In the Audit's opinion, the PCR of Highway 11 presents an ERR calculation which lacks credibility and must be dismissed. 62. SAR of Hiphwav 111. As in the SAR of Highway 11 (para. 54 above) -- no link was established between the civil works costs in the project cost table and those in the economic analysis. Again, the SAR was silent on the tax aspect and 12 this time in spite of a reminder at the time. 63. PCR of Hinhwav 111. The Audit was troubled by the fact that the PCR -- despite a very substantial civil works cost overrun, some 50% -- presented an overall ERR of 17% compared with an ERR of 18% in the SAR. 64. The Audit could retrace the cost side of the ERR calculation. The costs are, however, incorrectly in current rather than in constant prices in the PCR's ERR calculation. As to the vehicle operating cost savings (VOCS) the VOCS , are the same as those used in the SAR and prepared by consultants who had reviewed previous unit VOCS estimates for the Highlands Highway. Total VOCS as per the PCR, however, are a multiple (about twice for 1986) of those calculated by the Audit based on the PCR's own underlying data (traffic volume, unit VOCS, length of road sections, etc.) supplemented by information from the project file on traffic composition not presented in the PCR. Similarly, the PCR's total VOCS (PCR, Table 7) are also a multiple (about twice for 1986) of and inconsistent with those of the SAR (SAR, Table 17). 65. The Audit has to conclude that the PCR of Highway I11 presented incorrect and inflated ERRS, inconsistent with the SAR, and overstating considerably the economic results of the project. 66. Audit's ERR/ NPV Calculation. It is against this background (paras. 43-65) that the Audit decided to calculate for both projects the ERRS and NPVs for each of the seven road sections and for each project in total. The detailed underlying data are reflected in Tables 1 to 9; the ERR/NPV calculation is reflected in Tables 10 (Highway 11) and 11 (Highway 111); and Table 13 provides a summary of the results, a comparison with the SAR and PCR results, and some analysis of the principal factors accounting for the Audit's results. 67. Results. Please see Table 13. Highway 11 has an ERR of 4% essentially because of the negative ERR ( - . 4 % ) for the Chuave-Kundiawa section. And this, in turn, is mainly because of the very high civil works costs per km, between five and six times the corresponding costs of the two other road sections of Highway 11. Excluding the impact of the deterioration of the road over 45% of its length, the "ex-post" ERR for Chuave-Kundiawa is 1.3% (Table LO, footnote 2 ). 68. This calculation fully confirms the Audit's initial impression, backed up by subsequent more detailed scrutiny of the Green Cover and final SAR, as well as the PCR, that the SAR and PCR assessment of the project was both unrealistic and misleading and of unsatisfactory quality (paras. 50-61 above). 69. In the case of the SAR, the Audit believes that a satisfactory economic analysis of the Chuave-Kundiawa section would have affected the design decision for that section and would have resulted in improving the existing road rather than -- as was done -- realign this road and incurring costs which were five to 12 The Bank's Loan Cornittee, i n comnenting on the draft SAR, "pointed out that the cost estimates i n the SAR do not state s p e c i f i c a l l y whether taxes have been excludedw (see Project F i l e s . internal memo, 10129/76). six times the costlkm of improving the neighboring sections. In that case, the overall return of the project might well have been satisfactory. 70. Highway I11 has an ERR of 82 compared with an ERR at appraisal of 182. Two factors mainly account for this. The 33% cost overrun in real terms combined with a 39% shortfall in traffic by 1992. The Kassam Pass-Kainantu section has an ERR of below 4%: it had the highest civil works cost overrun and the highest civil works costslkm in combination with the lowest traffic volume and lowest VOCS benefitslkm compared with the other three sections. 71. Again, this calculation confirms the Audit's initial impression, backed up by closer scrutiny of the PCR, that the PCR's economic assessment was incorrect and overstated considerably the economic result of the project (paras. 63-65 above). 13 C. Risk Analysis in SARs 72. SAR of Highway 11. It devotes one paragraph (para. 5. lo), of two sentences, to the sensitivity analysis of the ERR. It states that "a 20% cost increase or a similar reduction in benefit would lower the rate of return to about 24%" (from 27.5%). The Audit's reestimated ERR for the project is 4%. 73. SAR of Hinhwav 111. It devotes one paragraph (para. 4.12) and one table (Table 17) to the sensitivity analysis. It states that "a 20% increase in costs decreased the ERR to 16%" and that "elimination of maintenance cost savings decreased the ERR to 15%" (from 18%). It concludes that the "analysis indicates that the risk is minimal in that neither overall cost increases nor benefit decreases have an impact on the ERR that would put into question the proposed improvements to the Highway". The Audit ' s reestimated ERR for the project is 8%. 74. The foregoing two examples are not atypical and quite representative for the way risk analysis is routinely dealtwith in SARs Bank-wide.14 However, the Audit believes that this type of risk analysis provides the decision makers (the Borrower, the Bank's Board, the Bank's Management) with a completely misplaced sense of reassurance and comfort for which there is no basis and no need. Staff Appraisal Reports should provide the decision-makers with realistic estimates of the project's downside propensity based on previous experience in the country or region and in the sector or subsector concerned and keeping in mind that overall l3 The Audit has p r e f e r r e d t o s t a y a s c l o s e l y a s p o s s i b l e t o t h e g e n e r a l approach and d a t a adopted i n the S A R and t h e P C R f o r t h e sake of comparability and t o reduce a s much ae p o s s i b l e t h e i n t r o d u c t i o n of a r b i t r a r y judgements. For t h e maintenance c o s t savings t h e Audit has r e l i e d on t h e e s t i m a t e s provided i n t h e P CR ( s e e Tables 10 and 11, s o u r c e ) . The v e h i c l e o p e r a t i n g c o s t (VOC) a r e i n 1979 p r i c e s (Table 9 ) . Hence t h e o i l c r i s i s of t h e e a r l y 1970s d i d not modify assumptione during p r o j e c t p r e p a r a t i o n . Since 1979, t h e (economic) p r i c e of automotive f u e l has come down r e l a t i v e t o t h e o v e r a l l p r i c e l e v e l i n PNG. A s a r e s u l t , t h e b e n e f i t 8 i n term8 of VOC savings during t h e 1980s a r e somewhat overestimated i n t h e ERR c a l c u l a t i o n . l4 A review of r e c e n t l y (Novernber/December 1992) i s s u e d SARs confirmed t h a t t h e same ( p r o forma) approach towards r i s k a n a l y s i s i s s t i l l being p r a c t i c e d a t p r e s e n t . 30 t o 40% of Bank-financed p r o j e c t s do f a i l i n p r a c t i c e ( s e e OED's Annual Reviews 15 . D. Physical Contin~encv 75. S i m i l a r l y , t h e SARs of both Highway p r o j e c t s i n c l u d e r o u t i n e l y - a s o t h e r s do - a s t a n d a r d 10% p h y s i c a l contingency. However, t o cover t h e a c t u a l c o s t overrun i n r e a l terms of Highway I1 i t should have been 41% and i n t h e c a s e of Highway 1 11 i t should have been 4 3 % . A s t o t h e c o s t overrun ( i n c u r r e n t p r i c e s ) f o r Highway I , t h i s was 90% ( s e e p a r a . 6 above). Again, t h i s r a i s e s t h e i s s u e of t h e l a c k of r e a l i s m of widely accepted conventions and h e r e t o o a r e a l i t y check i s i n o r d e r . E. Highway 11: Road D e t e r i o r a t i o n 76. S i n c e completion of t h e c i v i l works, t h e two improved road s e c t i o n s i n Simbu p r o v i n c e have r a p i d l y d e t e r i o r a t e d ( s e e p a r a s 27 t o 31 above). T h i s r a i s e s t h e q u e s t i o n s why, and what can be done about i t i n a l a s t i n g manner. 77. The Audit b e l i e v e s t h a t t h r e e f a c t o r s have c o n t r i b u t e d t o t h i s d e t e r i o r a t i o n . F i r s t , a s i s wellknown, t e r r a i n and geology i n t h e Highlands a r e d i f f i c u l t , c r e a t i n g an environment prone t o l a n d s l i d e s . Second, road s t a n d a r d s -- i n p a r t i c u l a r r e l a t i v e l y s t e e p s l o p e s and l a c k of r e t a i n i n g s t r u c t u r e s , e t c . , f o r economic r e a s o n s ( i . e . t o avoid p r o h i b i t i v e c o n s t r u c t i o n c o s t s ) -- a r e such t h a t l a n d s l i d e s blocking t h e roads and i n c a p a c i t a t i n g d r a i n a g e f a c i l i t i e s should be expected and accepted t o occur n o t i n f r e q u e n t l y . And t h i r d , road maintenance i n Simbu p r o v i n c e , which c o u l d / s h o u l d "compensate" f o r t h e f o r e g o i n g two f a c t o r s , i s n o t e f f e c t i v e mainly because of o b s t r u c t i v e behavior of t h e l o c a l p o p u l a t i o n . Hence l a n d s l i d e s a r e n o t promptly removed and proper d r a i n a g e i s n o t promptly restored. A s a r e s u l t , t h e roads d e t e r i o r a t e c o n s i d e r a b l y and q u i c k l y a s d e t a i l e d before. 78. To remedy t h i s s i t u a t i o n road maintenance e f f e c t i v e n e s s i n Simbu province has t o be i n c r e a s e d s o t h a t l a n d s l i d e s a r e promptly removed and damage caused by d r a i n a g e o b s t r u c t i o n prevented o r a t l e a s t reduced t o an a b s o l u t e minimum. One o p t i o n t h e government might l i k e t o c o n s i d e r i n t h i s c o n t e x t i s t o co-opt t h e r u r a l p o p u l a t i o n i n t h i s maintenance endeavor by i n v o l v i n g and, i f n e c e s s a r y , a c t i v e l y a s s i s t i n g l o c a l c o n t r a c t o r s ( i . e . from Simbu p r o v i n c e ) t o do most of t h e maintenance work. I f p r o p e r l y organized, t h e s t e a d y income t h e l o c a l p o p u l a t i o n would d e r i v e from i t s p a r t i c i p a t i o n i n t h e road maintenance e f f o r t might w e l l 15 OED's Annual Reviews of P r o j e c t Performance R e s u l t s f o r 1986 and 1987 comnented on t h e Bank's r i s k a n a l y s i s . The 1986 Review c a l l e d f o r a much b r o a d e r r i s k a n a l y s i s of p r o j e c t performance a t appraisal. The 1987 Review s u g g e s t e d a more e l a b o r a t e and c a n d i d assessment o f r i s k a t a p p r a i s a l and recomnended t h a t t h e Bank re-examine i t s approach t o r i s k a n a l y s i s . I n p r e p a r i n g t h e Annual Review f o r 1989 t h e m a t t e r was a g a i n a d d r e s s e d (Annual Review, P r o j e c t Performance R e s u l t s f o r 1989, T r a n s p o r t , OED Working Paper t 6 , May 23, 1990; t h i s p a p e r was w i d e l y c i r c u l a t e d i n t h e Bank and d i s c u s s e d w i t h t h e Regions). S p e c i f i c s u g g e s t i o n s were made on how t o improve t h e Bank's r i s k a n a l y s i s and i t was recomnended t h a t t h e Bank develop g u i d e l i n e s t o make t h e r i s k assessment i n f u t u r e a p p r a i s a l r e p o r t s more r e a l i s t i c . A number of i n d i v i d u a l OE D A u d i t s have e i m i l a r l y addressed t h e r i s k i s s u e before. result in a fundamental change in its negative attitude and behavior vis-a-vis road maintenance and thus remove the principal obstacle to proper maintenance.16 79. Second, the deteriorated road sections should not only be rehabilitated but the design standards (slopes, retaining structures) should be reviewed and,where economically warranted, spot improvements made. 17 V. CONCLUSIONS AND RECOMMENDATIONS A. Conclusions 1. Overall Results 80. Though both projects did contribute to the main objective of improving the Highlands Highway (para. 4 above), with rates of return of 4% and 8% (Tables 10 and ll), the overall project outcome has to be rated unsatisfactory and both projects unsustainable. The PCRs of the two projects indicated ERRS of 16% and 17% and based o n the PCRs the projects were considered satisfactory and sustainable. 81. I n the case of Highway 11, the principal reason for this outcome (ERR = 4%) was the very costly and -- in the opinion of this Audit wrong decision to -- build the Chuave-Kundiawa section for 75% o n a new alignment (i.e. from scratch) instead of improving the existing road (para. 6 9 above). 82. This decision, in turn, was supported by faulty economic analysis (paras. 43 t o 49 above) in combination with considerably and unrealistically inflated benefits (paras. 50 to 58 above). 83. More generally, the Audit attributes the poor results of Highway I1 to poor and biased preparation and appraisal of the economics of the project (paras. 43 t o 5 8 above). 84. I n addition, but secondarily, the ineffective road maintenance in Simbu province contributed to the project's poor results (paras. 27 to 30, 6 7 and 76 t o 79 above). 85. - I n the case of Highway 111, the two principal reasons for the disappointing outcome (ERR 8%) are the 33% cost overrun in civil works costs in real terms 16 The Region emphasizes that: "indigenous people obstruction t o and rent seeking from contractors- - ... has been a main cause for delayed clearance of landslides i n the Simbu province, rather than poor maintenance practices per s e . " 17 A s t o the question whether rehabilitation of Chuave-Kundiawa-Gamiger i s economically j u s t i f i e d , the Audit has l i t t l e doubt. Rehabilitation c o s t s per kilometer are l i k e l y t o be small, a s the road i s not t o be realigned any more, while the t r a f f i c volume since the 1970s has increased substantially. plus the lower than expected traffic volumes on the project roads: by 1992 minus 39% (para. 70 above). 86. As to the policy and institution building impact of the projects, progress was made in certain areas. For example, road user charges were considerably increased and now exceed road maintenance costs. Also, the trucking industry, de facto, was deregulated and entry and rate setting are free. On the other hand, in the area of traffic counting progress has been limited (para. 42 above). Overall, the institution building objectives were partially achieved. 2. Bank ~erformance'~ 87. The Bank's economic analysis in the appraisal of Highway I1 was clearly inadequate. The methodology was faulty (paras. 43 to 49 above), the SAR failed to established a clear linkage between the civil works cost estimate in the project cost table and the corresponding costs in the economic analysis (PAR, paras. 51 and 5 4 ) , benefits inflated (paras. 51, 52 and Table 14), essential information (traffic counts) not collected (para. 56), and obvious inconsistencies or concerns simply ignored (paras. 50 and 57). The Audit believes that a satisfactory economic analysis of the project would have affected the project design -- in particular for Chuave-Kundiawa -- and might well have resulted in a satisfactory overall project return (para. 69 above). 88. In Highway I11 - as in Highway 11 -- the SAR failed to provide a link between the financial and economic costs of civil works; an appropriate comment at the time was ignored (para. 62 above). 89. As to the economic analysis in the PCRs of the two projects, the Audit has to reject and dismiss the analysis of the Highway I1 PCR because of obvious and glaring implausibilities (paras. 59 to 61 above). 90. Similarly, the Audit concludes that the analysis of the Highway 111 PCR presented incorrect, inconsistent and inflated rates of return (paras. 63 to 65 above). 91. The risk analysis of both projects provided biased, entirely unrealistic and misleading conclusions. The Audit observes, however, that the two projects are not atypical in this respect (paras. 72 to 74 above) .I9 18 I n a memo d a t e d June 18, 1993. t o Region s t a t e s : "... w h i l e we a g r e e , a l b e i t t o a d i f f e r e n t d e g r e e , t h a t t h e p r o j e c t s and t h e PCRs ( i s s u e d i n 1985 and 1986) f e l l s h o r t o f t h e i r o b j e c t i v e s , we do b e l i e v e t h a t t h e r e p o r t could p r e s e n t a more balanced view o f t h e p r o j e c t a reviewed and of t h e c o n c l u s i o n s which can be d e r i v e d from them. W e s t i l l q u e s t i o n some o f t h e PAR'S c o n t e n t i o n s r e g a r d i n g p r o j e c t p r e p a r a t i o n and a p p r a i s a l ( e . g . c h o i c e o f a l i g n m e n t , inadequate physical contingencies) The a p p r a i s a l o f t h e s e p r o j e c t s f o l l o w e d s t a n d a r d p r a c t i c e f o r s e n s i t i v i t y a n a l y s i s i n t h e Bank i n t h e 1970's and t h e r e i s l i t t l e b a a i s f o r t h e r e p o r t ' s c l a i m t h a t i t l e d t o b i a s e d and u n r e a l i s t i c c o n c l u s i o n s . Improved methodologies f o r c a r r y i n g o u t r i s k a n a l y s i s have s i n c e been d i s c u s s e d i n t h e Bank b u t t h i s would n o t a p p e a r t o be t h e a p p r o p r i a t e forum f o r d o i n g i t . Also of c o n c e r n a r e t h e c l a i m s of f a u l t y ERR computations a t a p p r a i s a l and i n t h e PCR. W e f e e l t h a t t h e i n f o r m a t i o n t h e n a v a i l a b l e was used i n a n a c c e p t a b l e manner - q u e s t i o n s remain about V OC i n c r e a s e s i n r e a l t e r m s used a c r o a s t h e v a r i o u s r e p o r t s b e t v e e n 1973 a n 1979." Annex 1 shows t h e d e t a i l e d c o n n e n t s o f t h e Region and t h e manner i n which t h e s e were d e a l t w i t h ( w i t h e x c e p t i o n o f e d i t o r i a l s u g g e s t i o n s ) . 19 The O p e r a t i o n s P o l i c y Department of t h e Bank h a s had under p r e p a r a t i o n f o r some time a D i r e c t i v e (10.40-Economic A n a l y s i s of P r o d u c t s ) which i s t o i n c l u d e an annex on r i s k and s e n s i t i v i t y analysis. 92. Similarly, but by no means as dramatically, the physical contingency allowed for in both projects (and in their predecessor, Highway I), was clearly insufficient (para. 75 above). 93. The bottom line is that both the economic analysis and the risk analysis, in both projects -- as reflected in SARs and PCRs -- systematically overestimated the project outcome and systematically underestimated the downside risk. And this, in turn, harms the Bank's image of an impartial provider of quality professional advice. 94. Finally, the documentation in the Bank's project files is inadequate: important information that should be there, is not (paras. 47, 53, 58 and 60 above). B. Recommendations 95. Flowing from the project experience of Highways I1 and 111 this Audit offers four recommendations. 96. Recommendation 1: On Rehabilitation and Maintenance of Chuave-Kundiawa- Garniger Road. That the Government in its rehabilitation consider including spot improvements where economically justified and, to help ensure the sustainability of the rehabilitation, take steps to improve future maintenance, for example, through direct involvement of local contractors, if necessary, assisted by the Government (see paras. 76 to 79 above). 97. Recommendation 2: On Risk Analysis in SARs. That the Bank make an effort to inject a sense of realism in its sensitivity analysis by requiring SARs to report on previous relevant project experience in quantitative terms as a basis for a forward looking assessment of the risks involved (see paras. 72 to 74, 91 and 93 above). 98. Recommendation 3: On Physical Contingency That the Bank review the adequacy of the conventional 10% physical contingency normally used in road project appraisal and, if found inadequate, provide appropriate guidance to staff possibly differentiating between projects on the basis of different geological and topographical conditions encountered (see paras 75, 92 and 93 above). 99. Recommendation 4: On Completeness of Project File. That the Bank consider what steps to take to help ensure that the Bank's project file on completion of a project is complete. One possible approach would be to entrust this responsibility to those responsible for the preparation of the project's PCR and have this aspect reported on in the PCR (see paras. 47, 53, 58, 60 and 94 above).  Table 1 PERFORMANCE AUDIT REPORT PAF' A llEW CUINIU U SECOND E I G ~ R O S AD IMPROVENEXT PROJECT (CREDIT 677-PNC) AND THIRD EICBWAY PROJECT (LOU 1856-PNC MID CREDIT 1030-PNC) C i v i l Worke Costs: P r i c e D e f l a t o r Index, 1976-1986 (1979~100) Exchange Rate Combined Ditto, !!I!!! US CNP PN C Consumer Deflator - Year KinalUS$ - Index 1ndex1 Deflator P r i c e Index P r i c e Index (in Kina) Sourcesr (i) For exchange r a t e s and P N G deflator. 6 . 0 I n t e r n a t i o n a l h e t o r y Fund, International Financial S t a t i s t i c s (ii) For HUV i n d e x , s e e World Bank, I n t e r n a t i o n a l Econanics Department, October 1990 (iii) For P N C consumer p r i c e index, s e e I n t e r n a t i o n a l W e t a r y Fund, International Financial S t a t i s t i c s (iv) The ncanbinedn p r i c e d e f l a t o r is a simple average of ( a ) t h e ttW index m u l t i p l i e d w i t h t h e KinaIUSS exchange r a t e index, ( b ) t h e U S G HP index m u l t i p l i e d w i t h t h e KinalUSS exchange r a t e index, and (c) t h e P NC consumer p r i c e index. The f i r s t two r e f l e c t i n t e r n a t i o n a l c o n d i t i o n s while t h e l a s t one r e f l e c t . l o c a l c o n d i t i o n s roughly in l i n e w i t h t h e two t o one foreign/local cost relationship. - Note: The above 'combined p r i c e d e f l a t o r p r i c e i n d e x n , i n t h e o p i n i o n of t h e Audit, provides a f a i r b a s i s f o r a d j u s t i n g t h e a c t u a l c i v i l worlrrr c o s t s of t h e two p r o j e c t s from c u r r e n t i n t o c o n s t a n t p r i c e s . S i m i l a r c a l c u l a t i o n s have been made f o r o t h e r p r o j e c t s . A s t o t h e Region's s u g g e s t i o n t o c o n v e r t t h e c i v i l works w i t h t h e " s p e c i f i e d e s c a l a t i o n c l a u s e n f o r t h e c i v i l w o r h c o n t r a c t s , f o r two reasons t h i s was n o t attempted. F i r s t , t h e r e is no s i n g l e e s c a l a t i o n c l a u s e . I n t h e c a s e of P NC t h e r e were sane 15 p r i c e s , ranging f r a n timber t o d i e s e l t o s t e e l p i p e s t o g a s o l i n e t o s k i l l e d workers t o cement, e t c . . Even i f t h e Audit would have been a b l e t o s e c u r e t h i s information on a m u n t h - b y - m t h b a s i s , f o r each of t h e c i v i l w r k s c o n t r a c t s , both f o r t h e d e t a i l e d c o s t s and f o r t h e corresponding p r i c e s , i t would have gone beyond OED's l i m i t e d Audit r e s o u r c e s t o do t h e necessary c o l l e c t i n g and processing. But much more important, f o r t h e purpose of a r r i v i n g a t r e a s o n a b l e e s t i m a t e s of t h e c i v i l works c o s t s i n c o n s t a n t p r i c e s , t h e Audit feel. t h a t t h e above t a b l e s e r v e s t h a t purpose w e l l . I MIV Index - Manufactures Unit Value Index  Table 2 PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Second Hiphway Project: Actual Civil Works Costs, 1977-1981 (in current and in constant 1979 prices, K million) Total Asaro-Watabunq Chuave-Kundiawa Kundiawa-Garniner A. In Current Prices 1977 7 -99 1.04 6.95 1978 3.71 .89 2.41 1979 2.74 .25 1.82 1980 3.71 2.75 1981 5.08 4.13 Total 23.23 2.18 18.06 B. In Constant (1979) Prices 1977 9.18 1.20 7.99 1978 4.03 .97 2.62 .45 1979 2.74 .25 1.82 .67 1980 3.53 2.62 .91 1981 4.20 3.41 .79 2.42 18.46 2.82 23.68 - Note: Civil works costs include both construction and supervision costs Sources : (i) For costs in current prices, see PCR Highway 11, Table 2; to arrive at the civil works costs the construction costs have been increased by 10.9% to reflect the corresponding supervision costs. (ii) For costs in constant (1979) prices, see this PAR, Table 1, providing price index.  Table 3 PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Third Highwav Project: Actual Civil Works Costs, 1980-1986 (in current and in constant 1979 prices, K million) Daulo Pass- Goroka- Henganofi- Kainantu- - Total Goroka Henganofi Kainantu Kassam Pass A. In Current Prices 1980 5.24 1981 3.27 1982 6.16 3.96 1983 8.64 2.28 6.36 1984 15.88 5.14 9.24 1.50 1985 2.70 .90 -90 .90 1986 .23 .07 .08 .08 Total 42.12 6.11 12.50 12.80 B. In Constant (1979) Prices 1980 4.99 1981 2.70 1982 4.63 2.98 1983 5.97 1.57 4.39 1984 10.11 3.27 5.89 .96 1985 1.58 .53 .53 .53 1986 .12 .04 .04 .04 Total 30.10 3.84 8.03 8.90 9.34 Note: Civil works costs include both construction and supervision costs. Sources : (i) For costs in current prices, see PCR Highway 111, Tables 2 and 7. (ii) For costs in constant (1919) prices, see this PAR, Table 1, providing price index.  Table 4 PERFORMANCE AUDIT REPORT E PAPUA NW GUINEA SECOND HIGHLANDS R OAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Hinhlands Highway: T r a f f i c Count R e s u l t s f o r 1992 (ADT) Road S e c t i o n s Location Cars Conmercial Vehicles of Counting- Stations: Chainage - Small Medium Heavy (km) A. Highway 111: Kassam Pass-Kainantu 176 56 230 85 56 427 196 75 464 127 57 723 Average 66 347 106 57 576 Kainantu-Henganofi 219 91 490 157 106 844 Henganofi-Goroka 252 226 300 148 69 743 274 196 794 317 81 1388 Average 211 547 2 33 75 1066 Goroka-Daulo Pass 307 133 788 246 61 1228 313 90 397 26 1 109 857 Average 112 593 254 85 1044 B. Highway 11: Asaro-Watabung Chuave-Kundiawa Kundiawa-Garniger Note 1: T r a f f i c c o u n t s were undertaken on t h e Highlands Highway (Route N-7) d u r i n g seven c o n s e c u t i v e days. 12 hours p e r day, i n May 1992. To a r r i v e a t 24 hour r e s u l t s t h e 12-hour c o u n t s were m u l t i p l i e d by 1 . 3 . The Audit has followed t h e same approach a s t h e two SARs and PCR which t a k e s t h i s d a t a a s r e p r e s e n t a t i v e of t h e y e a r ' s average. Note 2: A s t h e Audit noted i n p a r a . 8 6 , p r o g r e s s i n t h e a r e a of t r a f f i c c o u n t i n g h a s been l i m i t e d . I n t h i s c o n t e x t , a r e c e n t s t u d y by t h e Asia Development Bank, PNG, Road T r a n s p o r t P r o f i l e . March 1991. o b s e r v e s : " I n a d d i t i o n t o t h e r e l a t i v e s h o r t a g e of t r a f f i c count d a t a , t h e r e i s no system of r e g u l a r c o u n t s and a s a consequence i n f o r m a t i o n is patchy. D O T has c a r r i e d o u t an average of a b o u t 170 manual c l a s s i f i e d c o u n t s o v e r t h e p a s t f i v e y e a r s . Almost a l l t h e c o u n t s a r e over a f u l l week, a p p a r e n t l y a l l o v e r 12 h o u r s . U n f o r t u n a t e l y t h e r e a r e v e r y few p o i n t s a t which t h e r e have been r e p e a t e d c o u n t s . The d a t a a r e t h e r e f o r e of almost no use i n judging t r a f f i c growth o v e r t h e y e a r s and of no use i n a s s e s s i n g s e a s o n a l v a r i a t i o n s . " Source: Department of T r a n s p o r t , PNG  Table 5 PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Sacond Hiahway Prolact: Traffic and Traffic CmDoaition. 1974 - 1992 Actual and 1999 Porocaar Amaro-Yatabung Chuava-Kundiawa Car -1974 23 - 1983 30 1992 90 1999 111 - 1974 20 - 1983 38 1992 177 Small C m m . 98 220 397 488 Medium C m . 35 72 261 321 Heavy C m m . Total -49 205 -50 372 109 857 - 134 LO54 Note 1: Small Comnercial Vehicles - 2 a x l e , 4 t i r e s Medium C o m e r c i a 1 Vehicles - 2 a x l e , 6 t i r e s Heavy Comnercial Vehicles - 3 o r more a x l e s Data f o r 1974, 1983 and 1992 r e f l e c t a c t u a l t r a f f i c count r e s u l t s ; t h e 1999 d a t a a r e f o r e c a s t s . Note 2: P r o j e c t e d annual t r a f f i c growth is 3% f o r 1992 t o 1999; r a i s i n g t h i s r a t e t o 10% i n c r e a s e s t h e p r o j e c t ' s r a t e of r e t u r n from 4.2% ( s e e Table 10) t o 5.0% and the p r o j e c t ' s n e t p r e s e n t v a l u e from 4 8 . 8 m i l l i o n t o -K8.1 m i l l i o n . - Source : For 1974, V a l l e n t i n e , Laurie and Davies, T r a f f i c Study Highlands Highway, Hay 1974, Page 9 and Table 14 SAR Second Highlands Highway P r o j e c t , January 11, 1977, Table 12 For 1983, P CR Second Highlands Highway P r o j e c t , Table 4 For 1992, t h i s P A R Table 4 For 1999, based on a c t u a l 1992 t r a f f i c p l u s p r o j e c t e d 3% annual growth r a t e .  Table 6 PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Second Highway Prolect: Actual and Forecast Traffic. 1974-1999 SAR (1977) PCR (1985) PAR (1992) Actual Forecast Actual Forecast Actual Forecast A. Asaro-Watabung B. Chuave-Kundiawa 1974 - 295 1983 578 697 1992 1008 991 - 1089 1999 1055 1217 C. Kundiawa-Garniger 1974 - 438 1983 844 - 890 1992 1438 1226 1437 1999 1806 1384 1768 Sources: (i) SAR Highway 11, Table 12 (ii) PCR Highway 11, Table 5 (iii) This PAR, Table 4 for actual traffic volumes; traffic forecasts from 1992 to 1999 assume traffic to grow at 3% a year.  Table 7 PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Third Hinhway Project: Traffic and Traffic Composition, 1979-1992 Actual and 2005 Forecast Doulo Pass-Goroka Goroka-Henuanof i Henganofi-Kainantu Kainantu-Kassam P a s s - 1979198519922005197919851992200519791985199220051979198519922005 Car 497 203 112 121 Small 195 623 593 641 Corn. Medium 136 224 254 275 Corn. Heavy - 137 - 58 - 85 - 92 Comn. Total 965 1107 1044 1129 Sources: For 1979, SAR T h i r d Highway P r o j e c t For 1985, PCR T h i r d Highway P r o j e c t supplemented w i t h a c t u a l t r a f f i c count i n f o r m a t i o n i n t h e p r o j e c t f i l e on t h e t r a f f i c composition For 1992, t h i s PAR Table 4 For 2005, t r a f f i c i s assumed t o grow from 1992 t o 2005 a t same r a t e a s from 1979 t o 1992 f o r t h e f i r s t two road s e c t i o n s and a t 2% a y e a r f o r t h e l a s t two s e c t i o n s - Notes : Small Comnercial V e h i c l e s - Medium Comnercial V e h i c l e s - 2 axles, 4 t i r e s 2 axles, 6 t i r e s Heavy Comnercial V e h i c l e s - 3 o r more a x l e s Data f o r 1979, 1985 and 1992 r e f l e c t a c t u a l t r a f f i c count r e s u l t s ; t h e 2005 d a t a a r e f o r e c a s t s .  Table 8 PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEXENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (MAN 1856-PNG AND CREDIT 1030-PNG) Third Highway Project: Actual and Forecast Traffic. 1979-2005 SAR (1979) PCR (1986) Audit (1992) Actual Forecast Actual Forecast Actual Forecast A. Daulo Pass - Goroka - 965 1517 - 1108 21 14 1729 - 1044 4165 3687 B. Goroka-Henganofi - 885 1390 - 1131 2593 1765 - 1066 3820 3764 C. Henganofi-Kainantu - 460 716 - 766 1045 1173 - 844 1967 2503 D. Kainantu-Kassam Pass 1979 - 350 1985 55 1 - 5 75 1992 804 88 1 - 576 2005 1513 1878 745 Sources : (1) SAR Highway 111 Table 13 and para. 4.09. (ii) PCR Highway 111, Table 5. (iii) This PAR, Table 4 for actual 1992 traffic volumes; traffic forecasts from 1992 to 2005 assume traffic to grow at same rate as between 1979 to 1992 for the first two road sections (A + B ) and at 2% a year for the laat two sections (C + D) .  Table 9 PERFORMANCE AUDIT REPORT PAPU E A NW GUINEA SECOND HIGHLANDS ROAD IWPROVEHENT PROJECT (CREDIT 677-PNG) AND TEIRD HIGHWAY PROJECT (WAN 1856-PNG AHD CREDIT 1030-PNG) Highlands Blahway. Vehicle m e r a t i n g Costs and Savings ( i n 1979 Kinalthousand v e h i c l e km) C m e r c i a l Vehicles Operating Costs: - Cars - Small Medium Heavy -Without Project 104 211 430 820 -With Project 79 160 347 696 Cost Savings : -With Project - Source: SAR and PCR Highway 111  T a b l e 11 PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) T h i r d Highway P r o j e c t : R e e s t i m a t e d ERRS a n d NPV's ( K m i l l i o n , i n constant 1 9 7 9 p r i c e s ) Total Project, All Pour 5ectlona ( ~ f i l A. Daulo Paas-Goroka I2zx.) Civil Civil Nat Civil Nat - VOCS Hiiifita - WaG - VOCS Giafita 4.99 2.70 4.63 .38 .I0 5.97 .39 .I1 10.1 L .40 .12 1.58 1.01 .27 .12 2.68 .43 2.68 .45 2.70 .47 2.70 .51 2.72 .55 2.72 .57 2.72 .59 2005 3.29 .59 3.88 1.24 .13 1.37 .95 .22 1.17 .54 .I8 .I2 ERR ( 2 ) 7.7 12.0 7.5 3.6 -R 4.4 m i l l i o n -R 1.7 m i l l i o n -K 4.1 m i l l i o n - NPV (101) R 1.2 million - Not., VCCS - Vehicla Operacing Cost Ssvingsj RIlCS Road Maintenance Coot Savlngs Sourcas~ (i) For C i v l l Work8 Coat. aea t h l e PAR. Table 3 P a r t B (iia) For t unit v e h i c l e o p a r a t i n g c ~ s 6aving6, oee SAR Highway 111. Tables 14 and I5 and a l a o P CR Highway 111, Table 6. f o r e x a c t l y the *am. infornatlonr see t h i s PAR Tabla 9 for tha data. (iib) For a c t u a l t r a f f i c v o l u w r 1977, 1985, 1992 and f o r e c a e t f o r 2005. a r e t h i s PAR, Table 1. (iii) For road maintenanca coot aavinge, eea P CR Highuag 111. Tabla 7.  PEBFOIIIlWCO AUDIT PO IT PAPU REV G U I I I U SECOWD HIGHLANDS ROAD II(PR0- PROJECT (CREDIT 61 1-PWG) AND THIRD IIGHUAY PROJECT (WAN 1156-PUG AND CREDIT 1030-RIC) S.eond and Third Righway Prolectl Eeonmie Rawulta and in~ U n d ~ r l ~ Iretor8 Economic R e e u l t e Some F a c t o r s U n d e r l y i n ~Audit R e s u l t s ERR T r a f f i c Volume -1992 PAR N PV ( 1 0 2 ) i n A c t u a l C i v i l Worke Coet Overrun - SAR - PCR - PAR 1979 p r i c e s Coete, i n 1979 C i v i l Works VOC (K m i l l i o n ) r i c e Coete i n SAR Benef i t e / R e a l Terme F o r e c a s t PAR A c t u a l PAR:SAR km 2 KO00 i n 1983 Second Highway: - Total -P e r Ian (K m i l l i o n ) - Aearo-Watabung ( 1 8 h) 2.42 .13 - Chuave-Kundiawa (27 hn) 18.46 .68 - Kundiawa C a r n i g e r ( 2 3 h) 2.82 .12 T o t a l (68 km) 23.68 .35 T h i r d Highway: - Kaeeam Paee-Kainantu (32.3 km) - Kainantu-Henganofi (38 km) - Henganofi-Coroka ( 45.5 laB) - Goroka-Daulo P a s s ( 2 2 km) T o t a l (138 km) Grand T o t a l (206 h) 2 Simple a v e r a g e o f t h e two t o t a l s . 3 Simple average of t h e two t o t a l s . 4 Simple average o f t h e two t o t a l s .  T a b l e 14 Pane 1 o f 4 P E R F O W C E AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMPROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Vehicle Operating Costs: P r i c e I n c r e a s e 1974-1975 I. Vehicles and Spare P a r t s A. Exchange R a t e s . 1974-1975 - Date Exchange R a t e s (USSIKlna) [KinalUSS) End o f March, 1974 1.4880 .672 End o f F e b r u a r y , 1975 1.3666 .732 I n c r e a s e i n KinaIUSS +8.9% B. MW I n d e x . 1974-1975 - 1975 Increase M o l e Year 21.8% 9 months 16.4% - 1975 Whole Year 2 months A p r i l 1974-February 1975 C. Exchange R a t e P l u s MUV I n c r e a s e , 1974-1975 A p r i l 1974-February 8.9% + 18.3% = 27.2% 1975 Sources: F o r exchange r a t e s , IMF, I n t e r n a t i o n a l F i n a n c i a l S t a t i s t i c s , O c t o b e r 1975. For MUV Index, World Bank, I n t e r n a t i o n a l Economics Department, O c t o b e r 2 6 , 1990.  T a b l e 14 Page 2 o f 4 PERFORHANCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS R OAD MPROVmNT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Vehicle Operating Costs: P r i c e I n c r e a s e 1974-1975 A. F u e l P r i c e s . Bulk, 1974-1975 P u l a u Bukom, S i n g a p o r e , S h e l l E a s t e r n Petroleum P t c . L t d . S i n g a p o r e Regular Gasoline Diesel Date o f P r i c e Changes (85 o c t . ) (50 CET) I n c r e a s e S i n c e A p r i l 1. 1974 (US$cts/gallon) Gasoline Diesel 1974: February 7 April 1 December 31 February 7.8% 4.7% March 20 35.5 - O c t o b e r 14 40.0 36.6 B. Exchange R a t e P l u s F u e l P r i c e I n c r e a s e , 1974 and 1975 A p r i l 1974 - February 8.9+7.8 = 16.7% 8 . 9 + 4 . 7 = 13.6% 1975 Average F u e l P r i c e I n c c r e a s e : 15.2% Sources : F o r f u e l p r i c e s , P l a t t ' s o i l P r i c e Handbook and Oilmanac, 1974 - 1975 P r i c e s , S l s t and 52nd E d i t i o n , New York. F o r exchange r a t e s , s e e P a r t I of t h i s t a b l e .  Table 14 Pane 3 of 4 PERFORMkNCE AUDIT REPORT PAPUA NEW GUINEA SECOND HIGHLANDS ROAD IMF'ROVEMENT PROJECT (CREDIT 677-PNG) AND THIRD HIGHWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Vehicle Operating Costs: Price Increase 1974-1975 111. PNG Inflation PNG Consumer Price Index. 1974-1975 (price increase) - 1974 Whole Year April - December - 1975 Whole Year January - February April 1974 - February 1975 17.2% Source: W ,lnternational Financial Statistics, Yearbook, 1991, page 116.  Table 14 Page 4 of 4 PERFORMANCE AUDIT REPORT PAPUA NEW GUINEA SECON D HIGHLANDS ROAD IWROVEMENT PROJECT (CREDIT 677-PNG) AND TBIRD HIGRWAY PROJECT (LOAN 1856-PNG AND CREDIT 1030-PNG) Vehicle Operating Costs: P r i c e Increase 1974-1975 IV. Vehicle Operating Costs A. Weights, by Cost Category, 1975 Individual Cost Weight % Fue 1 Tires O i l and Lubrication Repair and Maintenance Depreciation and I n t e r e s t Overhead T o t a l Costs Aggregated Cost Weights X Fue 1 ( I n c l . Fuel and O i l and Lubrication) Imported Vehicles and Spare ( I n c l . T i r e s ; 50% of Repair and Parts Maintenance; Depreciation and Interest) Local Cost ( I n c l . 50% of Repair and Maintenance; and Overhead) Total B. Vehicle Operating Cost P r i c e Increase VOC Price P r i c e Increase Cost Increase by Cost Weights X Fue 1 26 Imported Vehicles and Spare P a r t s 52 Local Costs - 22 T o t a l Vehicle Operating Costs 100 Sources : For i n d i v i d u a l weights, see Valentine Laurie and Davies, Highlands Highway Road P r o j e c t , Vehicle Operating Costs, August 1975, Tables 11 t o 16. The i n d i v i d u a l weights - - i n terms of v e h i c l e operating c o s t s a r e t h e average of t h e weights f o r f i v e v e h i c l e types ranging from c a r s t o s e m i - t r a i l e r s .  ANNEX 1 COMMENTS FROM THE REGION In the following are presented the comments, dated February 1, 1993, from the East Asia and Pacific Regional Off ice of the Bank, on OED's draft final Audit report, dated January 15, 1993. In addition, the following presents OED's responses to these comments with, where appropriate, references to the text of the final report. OED's Responses Risk analysis To say that a faulty economic analysis caused the selection of the wrong (and shorter) alignment for Chuave-Kundiawa, there should have been a basis to know better. However, it is not clear that there was a basis The basis to "know better" is in to "know better". The problem seems the Audit: paras 43 to 49 and 56. to have been insufficient engineering studies -- these did not undertake the necessary soil investigations to know that the proposed works would be stable or not. However, landslides and earthquake tremors were being experienced on the old alignment and it is not clear that the new alignment has had worse landslides than the old. Para. 23 says #'the detailed engineering could have been improved", but as this was a main deficiency, the The Audit does not feel that this report should be stronger on this was a main deficiency; see expanded point. para. 23. The broad audit conclusion against what was perhaps a standard incomplete risk analysis at the time, no doubt comes from the Wapenhans and other report; it can hardly come from only the two projects considered, and it should be framed in the overall context (e.g.page x, last para. column one; recommendation 2 on page xi). See para. 74 with footnotes. Likewise, "the Bank is less than fully See (revised) para. 93. committed to provide objective professional . advice.. " (page x) serves no purpose at this time and should be deleted. The report mentions lightly the indigenous people obstruction to and rent seeking from contractors -- but that has been a main cause for delayed clearance of landslides in the Simbu province, rather than poor maintenance per se. Contractors See parae 77, 78, and 96. include high premia to work in PNG. The fact that PNG gained independence in 1975 no doubt contributed to the second project processing delays and cost increases; this could be mentioned. Economic Analysis Para. 49 on savings by using larger trucks. What percentage of the voc savings in the SAR was due to the foreseen shift to larger trucks? What percentage did the assumed shift contribute to the ERR? This information is needed to substantiate the importance the audit assigns to the assumption not holding. Percentage did the assumed shift contribute to the ERR? This information is needed to substantiate the importance the audit assigns to the See expanded para. 49. assumption not holding. Para. 51 of the audit critiques the SAR for "understating the civil works costs in the ERR calculation by 6%". Normally the cost table includes taxes, and the economic evaluation excludes them, making the two costs different. There is no indication the project was tax-exempt; what was the tax incidence? (this affects other paras as well, such as para 54 where the difference See revised para. 51, 52, 54, is stated as 10% and speculated to be due to 62 and 88. In tho Road physical contingencies left out). In the Improvement Project, approved Transport Improvement Project approved in in 1983, tho tax component war 1986, the tax component in civil works was zero percent. 13%.  Table 1 , d e f l a t o r s . On what b a s i s was a simple average of t h e MW index, t h e US GNP d e f l a t o r and t h e PNG consumer price index used for d e f l a t o r ? The c i v i l works c o n t r a c t s should be converted i n t o " r e a l " terms by d e f l a t i n g them w i t h t h e i r s p e c i f i e d e s c a l a t i o n c l a u s e (what was i t ? ) . I f i t i s based on l o c a l PNG p r i c e s as could be expected ( t h e Kina was f u l l y c o n v e r t i b l e ) , then t h e c o s t increase i n r e a l terms was q u i t e lower t h a n t h e a u d i t assumes, a s p r i c e s i n c r e a s e d more i n Kina t h a n i n d o l l a r s . The See Table 1 , Note. The Audit a g r e e s a u d i t may f i n d t h a t t h e e s c a l a t i o n w i t h t h e Region's b a s i c comment and c l a u s e may have been i n a p p r o p r i a t e , has revised t h e c a l c u l a t i o n s b u t t h i s would need t o be e s t a b l i s h e d accordingly. and i s a n o t h e r p o i n t . T r a f f i c ; Table 4: Does t h e a u d i t know t h e s e a s o n a l i t y of t r a f f i c w e l l enough t o know t h a t counts one week i n May a r e r e p r e s e n t a t i v e enough of t h e See Table 4 , Notes 1 and 2 . y e a r ' s average? Table 6: t h e a u d i t f a i l s t o mention t h a t t h e 2nd p r o j e c t t r a f f i c forecasts a r e very close t o t h e a u d i t ' s " a c t u a l s " f o r 1992 and mostly below the audit's 1999 f o r e c a s t s . Why does t h e a u d i t assume only a 3% t r a f f i c growth f o r t h e future? With o i l e x p l o r a t i o n and other a c t i v i t i e s booming i n the Highlands? EERs: The Table 10, Chuave- Kundiawa s e c t i o n , reduces a l l b e n e f i t s by 45% to account for the d e t e r i o r a t i o n of t h e new road -- however, t h e 45% should apply only t o voc s a v i n g s due t o t h e s u r f a c e type, not t o savings f o r t h e s h o r t e r d i s t a n c e ( t h e avoided d i s t a n c e does The Audit a g r e e s and h a s r e v i s e d t h e not change) nor to (actual) c a l c u l a t i o n s accordingly. maintenance costs. With this c o r r e c t i o n b e n e f i t s a r e h i g h e r , e.g. 33% h i g h e r from 1986 onwards, and t h e ERR q u i t e h i g h e r t h a n t h e a u d i t ' s a l t h o u g h s t i l l low due t o t h e h i g h c o n s t r u c t i o n c o s t (almost K700,000 p e r h i n the audit, but l e s s with a l o c a l d e f l a t o r ) . More g e n e r a l l y , a r e p a s t maintenance cost savings shown close to actuals? How did the audit deal with the real fuel price increases (oil crises) of the early 1970s and early 1980s, in the voc benefit estimation? Was the high adjustment between yellow and green cover SAR not See para. 71, footnote 13. due to the first oil shock? Voc unit savings used seem low compared to other projects /countries; are unit voc close to current estinates with the HDM (in 1979 prices)? Correcting the deflator and the It does change the ERRS but not by benefit estimates will improve the much. ERRs. "the audit has little doubt" The Audit's judgment that the project that rehabilitating the sections is not sustainable reflects the (totalling 12.6 of 27 km) that unsatisfactory economic outcome of reverted to gravel standards is the project (para 80). As to the economically justified (page 13). It rehabilitation now, with previously also says that the project is not incurred costs "sunk", the Audit has sustainable(?) indeed little doubt that this would be economically justified. Both statements are fully compatible.  -- 1 4 ~ " \ ,LC 45 1 ,ae0 I&~ PAPUA NEW GUINEA 10C KILOMETERS )" I ! j \ THIRD HIGHWAY PROJECT