Turkey - Industrial Technology Project

 ICRR 12576 Report Number : ICRR12576 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/19/2007 PROJ ID : P009073 Appraisal Actual Project Name : Industrial Technology Project Costs (US$M): US$M ): 387.0 380.0 Project Country : Turkey Loan/ Loan /Credit (US$M ): US$M): 155.0 152.0 Sector Board : PSD US$M ): Cofinancing (US$M): Sector (s): General industry and trade sector (89%) Law and justice (11%) Theme (s): Small and medium enterprise support (23% - P) Other financial and private sector development (22% - P) Export development and competitiveness (22% - P) Personal and property rights (22% - P) Technology diffusion (11% - S) L/C Number : L4495 Board Approval Date : 06/17/1999 Partners involved : Closing Date : 12/31/2003 04/30/2006 Evaluator : Panel Reviewer : Group Manager : Group : James Sackey Roy Gilbert Ali Khadr IEGCR 2. Project Objectives and Components: a. Objectives: The objectives of the project were to : (i) assist in the harmonization of Turkish technology infrastructure with European Custom Union (ECU) standards; and (ii) assist firms in upgrading their technological capabilities in order to improve the competitiveness of Turkish industry, both in domestic as well as foreign markets . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project had four components : Component A; Strengthening Industrial Property Rights and Services (Appraisal Total Cost: US$ 19.3M; Actual Total Cost: US$ 19.63M; Bank Contribution: US$ 15M at appraisal; US$ 14.63M Actual Disbursement). The component supported: (i) the improvement of patent filing and search examination procedures; (ii) the establishment of dedicated information centers for documentation and dissemination of information; and (iii) the enforcement of industrial property rights . Component B : Upgrading Metrology Services (Appraisal Total Cost: US$ 42.5M; Actual Total Cost: US$ 58.56M; Bank Contribution: US$ 33M at appraisal; US$ 44.56M Actual Disbursement). . The component supported (i) the upgrading of existing measurement services; and (ii) the extension of measurement services into new areas. Component C : Restructuring of Research and Development (R&D) R&D ) institutions (Appraisal Total Cost: US$ 42.3M; Actual Total Cost: US$ 41.97M; Bank Contribution: US$ 33M a appraisal; US$ 32.27M Actual Disbursement). The component supported the restructuring of the network of research institutes associated with the Marmara Research Center (MAM)) through the upgrading of management practices, the development of human resources and the establishment of profit centers . Component D : Supporting Technology Upgrading by Firms (Total Cost US$ 265.4M; Actual Total Cost: US$ 258.75M; Bank Contribution: US$ 60M at appraisal; US$ 58.95M Actual Disbursement). The component supported: (i) the establishment of an independent National Accreditation Council (NAC); (ii) the development of a pilot program to address Y 2K issues; (iii) the provision of technology support services to private enterprises; and (iv) the establishment of technoparks and venture capital funds (VCF). The Appraisal Total Cost included US$ 17.5M unallocated contingency and Actual Total Cost included US$ 1.55M for IBRD Front-end Fee. Bank contribution at appraisal included US$ 12.45M unallocated contingency and US$ 1.55M for IBRD Front-end fee. Actual Bank disbursement included US$ 1.55M for IBRD Front-end fee. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The Bank contributed about 40 percent of the total appraisal cost, with the private sector 40 percent, and government and its agencies, the remainder . These ratios did not change at project completion . Bank disbursements amounted to 97 percent of total loan allocation . The closing date of the project was extended twice; first, for two years up to December 31, 2005 and later, up to April 30, 2006 to allow for the completion of implementation which was affected by the availability of counterpart funding on the part of Government and its agencies during the financial crisis of 2001. 3. Relevance of Objectives & Design: Relevance of Objectives The objectives of the project were substantially relevant. They responded to the key factors affecting industrial competitiveness of Turkey, in the context of its possible membership in ECU . Membership of ECU offered Turkey a unique opportunity for accelerated growth and development through freer and better access to markets . But Turkish industries needed to upgrade from low -quality, labor-intensive products with limited growth prospects, to the production of higher value-added goods and services . The objectives of the project were clearly defined to meet this need, consistent with the goal of the CAS for Turkey to upgrade its infrastructure for standards, testing and certification in line with ECU accession agreement . Relevance of Design . The design of the project was modestly relevant. The components listed in 2(c) were relevant to the project's objectives of dealing with the four main constraints facing Turkish industry : low product quality, complex industrial property rights regime, low rate of technological development, and weak linkages between R&D institutions and industry. The design of the project was relevant in that each of the four components dealt with one of the four issues identified and permitted the selection of the four key institutions in the sector as implementing agencies . This made it possible for the Turkish Treasury to oversee overall implementation, while the activities planned under the project were managed and implemented by the various institutions concerned . The results framework had negligible relevance. It defined inputs and outputs by components but without specific indicators and time-bound targets. Development impact indicators were also specified but they were generic, not time -bound, and had no benchmarks . They were also not directly linked to the two objectives of the project . 4. Achievement of Objectives (Efficacy): Because the results framework defined for the project did not specify outcome indicators with which to assess the achievement of the objectives of the project, this assessment could only be done through inference from output information. (i) Harmonization :Modest. To assess improvement in the competitiveness of Turkish industry through the assistance provided by the project, three indicators of competitiveness are analyzed : cost reduction, increase in productivity, and export intensity. The ICR reports on a simulation analysis based on survey data gathered at the close of the project. The output data indicated that the assistance provided to the National Metrology Institute (UME) had led to increase coverage. At the end of the 1990s, UME was capable of meeting about 30 percent of the industry's need for metrology; by the end of the project it had reached the 85 percent level. UME has expanded the number of services offered to industry, with a reduction in costs and time for these services . The processing time for Industrial Property Rights (IPR) applications was reduced from 19.8 months to 10 months. Prior to the project, most metrology work had to be done outside Turkey, which was slow and expensive . The ICR also reports on a simulation exercise based on the assumption that clients who benefitted from the project should be expected to be more productive than nonparticipating firms . In order to test this hypothesis, average effects of participation in the project were estimated for low -technology manufacturing, medium and high -technology manufacturing, services and the software sectors . The result indicated that there was significant difference between participant and nonparticipant firms in terms of labor productivity (measured as output per employee ), especially in manufacturing. The evidence for services and software was not conclusive . Finally, the average effect of participation in the project on export intensity (defined as the share of exports in total sales) was estimated on the assumption that firms that were able to penetrate export markets were likely to be more competitive. A simple comparison between the average export intensity of project participating firms and nonparticipants showed that in 12 out of 16 comparisons, project participants had higher average export intensity than nonparticipants in control groups . The three outcomes (cost reduction, increased productivity, and export intensity) together support the inference that the project assisted in improving competitiveness of Turkish industries . Furthermore, the Growth Competitiveness index (GCI) and the Business Competitiveness Index (BCI) showed improved ranking for Turkey on several innovation and technology development indicators for 2004 over 2003. (ii) Technology Upgrading : Substantial. The outcomes of two components of the project (restructuring of R&D institutions, and technology upgrading of firms ) support the rating. The M&E Report in 2006 reported that the Technology Development Foundation of Turkey (TTGV), one of the four project implementing agencies, had become an important agency for R&D financing in Turkey by supporting 380 R&D projects through matching foreign exchange loans. The report estimated that close to 90 percent of the R&D projects supported by TTGV were being put into commercial production. TTGV also participated in two venture capital funds (VCFs) to facilitate the development of the venture capital industry in Turkey . Similarly, the restructuring of the Marmara Research Center (MAM), a network of R&D institutions, led to their reorientation towards more contract research . The output indicators collected by the project showed that MAM had significantly expanded its client base from 87 in 1999 to 348 in 2005. In addition, the project had equipped MAM and all its laboratories to obtain ISO 9000 certification. Many of its laboratory analyses were internationally accredited and others were in the process of obtaining such accreditation . 5. Efficiency (not applicable to DPLs): NA ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Moderately satisfactory . Assistance provided by the project helped to upgrade the technological capabilities of Turkish firms, reduced the cost of metrology services, improved productivity and increased exports of participating firms and are likely to lead to improved competitiveness in the medium -term. But project implementation was subject to long delays and the results framework had a negligible relevance . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The move away from a wholly government funded R&D program to that of self -financing and privately supported programs presents a risk to sustaining the development outcome of the project because of the possibility of economic crisis. Since an overly disruptive economic crisis (of the type in 2001) is a very remote possibility for Turkey in the medium term, this risk is rated moderate . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at Entry : The Bank built a good team that produced a relevant project design that linked each project component to an implementing agency. The project preparatory work took 16 months and involved close collaboration between the Bank team and Government in building upon the lessons learnt from the previous Bank interventions in the sector and in identifying the key issues to be addressed . The appraisal team was adequately staffed with personnel specializing in industrial property rights, R&D institutions, technology finance, industrial technology, and the usual fiduciary personnel. The assistance provided to the client in assessing the feasibility of proposed program and preparing project implementation plans was timely , although sometimes (as in the case of the reforms of property rights) proposed implementation schedules were ambitious . Supervision : Bank performance during supervision was satisfactory . There were 18 supervision visits (averaging 2-3 per year), which included specialists as appropriate for each of the targeted reform during a specific mission . Although the M&E framework built into the supervision process provided information for assessing progress in project implementation and served that purpose adequately, weaknesses in the design made evaluation of the achievement of the expected outcomes difficult (see section (10) below). at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: Government Performance Government (represented by the Treasury ) exhibited throughout the preparation and execution of the project its commitment to continuing the technology reforms and strengthening the process started under the first Bank supported Technology Development Project . But it failed to put corrective measures in place to deal with the financial crisis in 2001, which resulted in counterpart funding problems and implementation delays . Implementing Agencies Performance Because the agencies had to depend on the Treasury for counterpart funding, the pace of project implementation was substantially slower than anticipated at project launch as a result of the financial crisis of 2001. Some implementing agencies also faced management and staffing problems . Despite the Bank's best effort with the Treasury and the implementing agencies, the funding and staffing issues were not resolved on a timely basis thereby resulting in implementation delays . Despite this, commitment to the reforms by the implementing agencies were sustained and a major surge in activities took place in the last two years of the project, negating the slow start-up. a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: M&E Design The M&E system was designed to provide information on project implementation . The design had a good feature in that the input and output indicators were aligned to the implementing agencies, while the development impact indicators were cross-cutting. Unfortunately, while base line values were estimated at the start of project implementation, no target values to achieve were indicated, making it very difficult to evaluate performance . Furthermore, the design did not provide a framework for monitoring and evaluating the expected outcome of the project in improving the competitiveness of Turkish industry through interventions to upgrade technology services . M&E Implementation As part of the process of project execution, the implementing agencies regularly collected data on the input and output indicators and a final M&E report was prepared in May 2006. The exercise was executed on target and provided valuable data for calibrating progress in project implementation . M&E Utilization The input and output indicators collected by the implementing agencies were used effectively during site visits by the Bank team in assessing progress in project implementation and in pursuing policy dialogue with the PIUs on issues effecting progress of project execution . The first of the three development impact surveys was used to benchmark firm implementation performance, while the second survey was used for monitoring and generating feedback on project impact. The final M&E report provided the data for assessing the possible sustainability of the development impact of the project. Unfortunately, in the absence of outcome indicators, only simulation analyses could be undertaken to evaluate expected outcomes . a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): None 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Highly Satisfactory Moderately Delays during project implementation Satisfactory could affect the efficiency of the operation's achievement of its objectives. Design weaknesses of M&E made it difficult to assess outcome . Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Moderately Quality at entry and effectiveness of Satisfactory supervision were affected by design weaknesses of the M&E system . Borrower Performance : Satisfactory Moderately Substantial delays encountered in Satisfactory project implementation due to Government bureaucracy and weaknesses in some implementing agencies that necessitated a 28 months extension of loan closing . Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Policy interventions to promote technological innovation are likely to succeed if conducted in an environment undergoing market liberalization aimed at improving the incentive framework . Frequent changes in project management and staffing, as with the Turkish Patent Institute (TPE), could lead to long delays in project implementation as it takes new management and staff time to understand the objectives and rationale for the project. In a difficult and uncertain economic environment, building f lexibility in project design through the provision for contingency, makes it possible, during implementation, to streamline components to achieve project DO without necessarily adjusting other components .  In projects involving complex and innovative components, a well -defined set of outcome indicators which are consistent with project objectives, are necessary for monitoring achievement of objectives . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: Although the ICR made an effort to deal with the failure of the PAD to specify outcome benchmarks for the M&E system, the quality of the resulting analysis and assessment of outcome was only fair . The lessons derived from the implementation of the project were unclear because the structure of the monitoring system was too disaggregated and difficult to interpret. While the ICR touched on the right issues highlighted by the project, it was not concise, and had missing information (such as project cost and financing data ) and paragraphs. a.Quality of ICR Rating : Unsatisfactory