Official Documents- Reimbursable Advisory Services Agreement with the Ministry of Economic Development of Polad, P159728

OFFICIAL DOCUMENTS PROJECT NUMBER __79 REIMBURSABLE ADVISORY SERVICES AGREEMENT between MINISTRY OF ECONOMIC DEVELOPMENT OF THE REPUBLIC OF POLAND and the INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT Dated ___ ff_ _, 2016 AGREEMENT FOR REIMBURSABLE ADVISORY SERVICES AGREEMENT dated the .93 day of ffoV 2016, between the MINISTRY OF ECONOMIC DEVELOPMENT of the Republic of Poland (the "Client") and the INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT (the "Bank") (jointly referred to as "the Parties"). WHEREAS, the Client has requested the Bank to provide to the Client reimbursable advisory services (the "Reimbursable Advisory Services" or "RAS") described in the Schedule to this Agreement to fulfill two complementary objectives to: (i) equip Voivodship and Poviat labor offices staff with skills and tools to design and oversee impact evaluations for their employment promotion projects through impact evaluation capacity building series; and (ii) support the carrying out of pilot evaluations through advisory services as a follow-up to the evaluation training. NOW, therefore, the Parties hereto agree as follows: 1. Reimbursable Advisory Services. The Bank shall provide to the Client the services ("Reimbursable Advisory Services" or "RAS") described in the Schedule to this Agreement, on the terms and conditions set out in this Agreement, including the Annex hereto, which constitutes an integral part hereof. 2. Client Contacts. In carrying out the Reimbursable Advisory Services, the Bank will work closely with the designated officials of the Client. The Client will provide the Bank with the names and contact information for said designated officials. 3. Timing. While the Bank undertakes to mobilize all reasonable means available to it to carry out the Reimbursable Advisory Services in a timely manner, the work program and timetable set out in the Schedule to this Agreement have been prepared in good faith based in light of information currently available to the Bank, and are given on an indicative basis assuming that: (i) the Client and its personnel will carry out their respective duties in a satisfactory and timely manner; and (ii) the Client will at all times act in a timely manner in providing information, making decisions and providing necessary support as provided herein and as requested from time to time by the Bank. -2- 4. Payment (a) The Client shall pay the Bank a fixed fee of one hundred ninety seven thousand and nine hundred and forty eight Euro (EUR 197,948) in accordance with the following schedule of payment: Amount of Installment Event upon which Payment is Due 1. EUR 21,452 until 15 January 2017. 2. EUR 71,507 upon the delivery of Workshop I for Activity 1 in the Schedule to this Agreement. 3. EUR 50,055 upon the delivery of Workshop 2 for Activity I in the Schedule to this Agreement. 4. EUR 54,934 upon delivery of the advisory services described under Activity 2 in the Schedule to this Agreement. (b) All payments to the Bank hereunder shall be made in full when due, upon submission of an invoice by the Bank. Payment shall be made, in EURO, in immediately available funds, without any deductions whatsoever for taxes, duties, charges or other withholdings, and notwithstanding any pending dispute between the Parties, to such account as the Bank may from time to time designate in writing. 5. Effectiveness. This Agreement shall become effective as of the day and year first above written, once it has been duly signed by both Parties. 6. Expiration. This Agreement shall expire on December 31, 2017 unless it shall earlier be renewed with the mutual agreement of the Client and the Bank. 7. Termination. Either the Client or the Bank may terminate this Agreement prior to its expiration upon ninety (90) days' written notice to the other. Upon receipt of such notice, the Parties shall take all appropriate steps to terminate in an orderly manner the activities then ongoing under the Reimbursable Advisory Services and to settle promptly all outstanding matters. 8. Cooperation. (a) The Client shall at all times provide the Bank in a timely manner with any and all information that may affect the performance of the Reimbursable Advisory Services, inform the Bank of any developments relating to the Reimbursable Advisory Services, and do all things necessary to enable Bank Personnel to carry out the Reimbursable Advisory Services hereunder. (b) It is expressly agreed and understood that the Bank shall bear no responsibility for delay in performance occasioned by the Client's failure to provide its contribution as set forth in Section D of the Schedule or otherwise cooperate as set forth in paragraph (a) above. -3- 9. Notices and Addresses. (a) All notices required or permitted to be given pursuant to this agreement shall be in writing and shall be deemed to have been duly given or made when delivered by hand or by mail, or facsimile to the signatories to this Agreement at their addresses as set out below or such other addresses as may be notified by either party from time to time. Notices forwarded by registered mail shall be deemed to be delivered upon delivery. Notices made by facsimile transmission shall also be confirmed by mail, with their effective date being the date of initial transmission. (b) The following addresses are specified for the purposes of this Agreement: For the Client: Ministry of Economic Development Department of European Social Fund Poland Phone: +48 22 273 80 50 Fax: +48 22 273 89 19 For the Bank: The World Bank 1818 H Street, NW Washington, DC 20433 USA Phone: (202) 477 -1234 Fax: (202) 477 - 6391 -4- IN WITNESS WHEREOF, the Parties hereto, acting through their duly authorized representatives, have caused this Agreement to be signed in their respective names as of the day and year first above written. MINISTRY OF ECO OMIC INTERNATIONAL BANK FOR DEVELOPMENT E RECONSTRUCTION AND REPUBLIC OF P D DEVELOPMENT By: By: A orized Represetattive Authorized Representative Name: Anna Mickiewicz Name: Emilia Skrok Title: Deputy Director, Title: Senior Economist, Acting Country Department of European Social Fund, Manager for Poland Ministry of Economic Development -5- SCHEDULE Description of the Reimbursable Advisory Services A. Reimbursable Advisory Services. Except as the Client and the Bank may otherwise agree, the Reimbursable Advisory Services shall include the following activities and outputs: Activity 1: Impact evaluation capacity building The Bank shall carry out a series of Impact evaluation capacity building workshops targeted to staff in Voivodship and Poviat labor offices, particularly those engaged in planning, program design, program evaluation, and those in charge of administrative data collection and analysis. Obiective(s): The workshops will provide a rationale for: (i) why evaluation with counterfactual (e.g. "impact evaluation") is important for evidence-based decision making, (ii) the need to think about evaluation and program design jointly, and (iii) an overview of impact evaluation methods and explain how these methods can be applied to program design and policy-making. In particular, the focus of the workshops is to equip participants with the knowledge and skills to understand the impact evaluation design and process and be able to knowledgably contract out the evaluation work. The workshop will also provide an overview on how to "operationalize" an impact evaluation and the steps required to implement it]. Methodology: The training material is based on the World Bank impact evaluation in practice handbook, its updated 2016 edition, and the multi-year World Bank teaching experience of the Strategic Impact Evaluation Fund (SIEF). The approach is intuitive, covers the newest techniques for evaluating programs and includes state-of-the-art implementation advice. The content of the workshop and the evidence presented will be adapted to be relevant for the Polish context and youth employment programs. The government will provide a list of participants to be trained. Workshops will be preceded by a survey aimed at taking stock of the pre-workshop knowledge of participants about impact evaluations, as well as statistics and econometrics knowledge to allow for calibrating more precisely the content of the sessions to the audience. The workshops will provide with a few demonstrative group sessions some of the key steps to derive impact results from a pre-clean dataset with stata software. However the workshops do not envisage in-depth training on statistical analysis. The data analysis of collected data will be carried out by a third party institution to be contracted by the Regional Labor Office and in cooperation with local office. The focus of the workshops is rather on equipping participants with the knowledge and skills to understand the impact evaluation design and process and thus be able to knowledgably contract out the evaluation work. -6- A total of 60 participants per workshop (120 in total for this RAS agreement) will be trained through the mix of plenary lectures, smaller group discussions and seminars, and hands-on application of impact evaluation tools to participants' own projects. The workshops are structured as a combination of general plenary sessions and two parallel tracks, a technical and a policy-maker track, in order to adjust level of technicality to the content to the two different profiles of participants. Technical workshops are expected to last 5 days (3 days preceding the policy maker track, and 2 days after the policy maker track), and the policy maker-track workshops are expected to last 2 days. The first workshop will be treated as a pilot in terms of the contextualization to youth employment programs and Poland context. To assess the fit and usefulness of the pilot format, the first workshop will be followed by a focus group-like discussion with participants and written evaluation questionnaire. The feedback obtained will allow customizing the format of the subsequent workshops. Workshop Structure Plenary session This session gives a joint introduction on the principles of impact evaluation to both participants of the technical and the policy-maker-track sessions. It explains why it is important to make correct inference and uses accessible examples to illustrate it. Technical track workshop * The Technical-track sessions are designed for researchers and professionals who want to acquire knowledge of quantitative methods of impact evaluation (IE) and are responsible for implementation and logistics of impact evaluations at the local level. The audience will include Head of Evaluation units at the regional Labor Offices and in the Poviat Public Employment Services. While this staff is not expected to implement IE directly, this is in charge of writing IE Terms of Reference, monitoring their implementation and making sure that the design of the intervention allows for evaluation with counterfactual to begin with. These sessions provide a summary of the latest evaluation methodologies. In addition, they are concentrated on how to design and conduct impact evaluations, using econometric and statistical tools as well as case studies that illustrate the clear link between theory and practice. Participants will receive guidance on how to supervise and administer evaluation projects at the local level. * The 5-day workshop would unfold as follows: the first morning is dedicated to the plenary session and introduction of the two tracks. On the afternoon of day 1, evaluation methodologies will be reviewed in detail (experimental, quasi-experimental, etc.). On days 2 and 3, participants will work together -7- as project teams with expert moderators on the "hands-on" design of a rigorous impact evaluations and implementation plan for their projects. Day 4 focuses on costing and implementation of the evaluation process. On the final day of the workshop, the teams present their proposed evaluation plans to the whole group and receive guidance from each other and the course moderators. Policy track workshop The Policy-maker-track sessions are designed for Directors responsible for the overall design of programs and policies. These sessions provide the theoretical background to appreciate the value of estimating causal impact and reviews the main evaluation methodologies and the appropriateness of each technique to different contexts. The policy-maker sessions also cover practical aspects of the requirements to include high quality impact evaluations into the new programs, including sample power issues, roll-out strategies, and other operational tools. Understanding of these issues is key for policymaker to effectively budget for review and foresee IE implementation timelines. Key themes (i) Technical-track: theory, methods, and designing impact evaluations 1. Causal Inference 2. Experimental designs: Randomization 3. Regression discontinuity 4. Introduction to other non-experimental techniques 5. Designing and planning an evaluation 6. Dealing with operational constraints 7. Case studies from practitioners and policy makers 8. Linking monitoring systems / data and impact evaluation 9. Communicating the impact evaluation results to the general public (ii) Policy-maker-track: managing evaluation studies and using results for more effective programs 1. What is impact evaluation and what are the advantages over traditional ex-post evaluation 2. What it takes to implement an IE and when it is appropriate: understanding program assignment (treatment and control group), baseline survey, follow-up survey 3. Interpreting results from impact evaluations and how it feeds into program cost-effectiveness analysis 4. Monitoring and mainstreaming impact evaluation. 5. Discussing current and planned interventions that could be evaluated 6. Communicating the impact evaluation results to the general public -8- Activity 2: Advisory service for implementation of Impact Evaluations The Bank will provide hands-on support in order to facilitate the realization of the first round of IE of European Structural Funds (ESF) projects. The IE processes will include a review of the proposed design of the IE for a specific program, a review of the terms of reference to contract out the evaluation, a joint review of the proposals from applicant vendors, a review at different stages of the IE implementation plan of the IE, and just-in-time support from the client labor office or public administration to the client, and quality control and review of the IE report once this is produced. Methodology: The advisory services will be provided in a modular way in up to three locations in order to assist in the implementation of up to three lEs. The extent of the advisory services to be provided by the Bank will depend on the actual demand from the regions and on the timing of the programs that they want to evaluate. B. Timetable. Except as the Client and the Bank may otherwise agree, the Bank shall endeavor to perform the Reimbursable Advisory Services in accordance with the following tentative timetable: Timeline: * Capacity Building workshop 1, Evaluation of February 2017 the workshop and adjustments * Advisory Service November 2016 - December 2017 * Capacity Building workshop 2 June 2017 C. Bank Personnel. The following is an indicative list of the categories of Bank Personnel to be involved in carrying out the Reimbursable Advisory Services: Senior Economists and Senior Education Specialists. D. Counterparts and Facilities. The Client shall carry out the following activities in support of the Reimbursable Advisory Services: (a) The Client shall provide the Bank Personnel with necessary administrative and organizational support for the Reimbursable Advisory Services. The Client will also cover the accommodation costs for participants who cannot commute daily to the training venue and cost of venue and catering during workshops. -9- E. Records. The Bank shall keep appropriate records of the Reimbursable Advisory Services in accordance with its normal record-keeping practices. - 10 - ANNEX STANDARD TERMS AND CONDITIONS 1. Performance Standard; Non-Exclusivity. The Bank hereby shall carry out the Reimbursable Advisory Services with the same care and diligence as it uses in its other analytical and advisory activities. The engagement of the Bank as adviser to the Client hereunder is non-exclusive and shall not restrict the Client from engaging other advisers on the same or related issues. 2. Bank Personnel. The Bank shall, in its sole discretion, determine the composition of the staff (including staff holding consultant appointments) ("Bank Personnel") assigned to perform the Reimbursable Advisory Services. The Schedule to this Agreement contains an indicative list of Bank Personnel likely to be involved in carrying out the Reimbursable Advisory Services; provided, however, that the Bank retains at all times the right to assign such other person or persons in addition to, or substitution for, any of the persons appearing on that list as the Bank may deem necessary or appropriate in the performance of its duties hereunder. The Client may, if it has reasonable cause for dissatisfaction with the performance of any of the Bank Personnel, request the Bank to replace such person(s). For avoidance of doubt, it is agreed and understood that this Agreement does not create any employment or other contractual relationship between the Client and Bank Personnel. 3. Client Contributions. The Client shall carry out the activities and provide the facilities and other arrangements set out in the Schedule to this Agreement. 4. Bank Operational Policies. The Bank shall provide its advice in a manner consistent with its relevant environmental and social safeguards policies. 5. Confidentiality. The Parties agree that this Agreement and the outputs set out in the Schedule to this Agreement shall only be made available to the public after the Client has given its written consent to such disclosure. For this purpose, the Client hereby authorizes the Bank to publicly disclose this Agreement and the outputs set out in the Schedule to this Agreement in their final form. With respect to underlying information provided by the Client in support of the Reimbursable Advisory Services, the Client reserves the right to designate said information as confidential. The Bank may publicly disclose such information only after the Client has given its prior consent. With respect to underlying information (other than the final outputs) provided by the Bank in support of the Reimbursable Advisory Services, the Bank reserves the right to designate said information as confidential. The Client may publicly disclose such information only after the Bank has given its prior consent. 6. Intellectual Property. The intellectual property rights of the Parties in any of its pre-existing data or documents used by the Bank in connection with the Reimbursable Advisory Services shall remain with that Party. The intellectual property rights in new materials prepared by the Bank in connection with the Reimbursable - 11 - Advisory Services shall belong to the Client; provided, however, that the Bank shall have the global, non-exclusive, perpetual (for the duration of the copyright) , fully sub- licensable and royalty-free right to use, copy, display, distribute, publish and create derivative works of all or part of these materials and incorporate the information therein in its research, papers, publications, web sites, and other media without the consent of the Client, subject to the limitations on disclosure of confidential information and any third party rights, as indicated in paragraph 5, Confidentiality, of this Annex. 7. Representation of the Bank's views and use of the Bank's name, marks and logo. (a) The Client agrees that it shall not represent, or permit the representation of, the Bank's views without the prior written consent of the Bank. (b) The Client further agrees that it shall not use, or permit the use of the Bank's name, marks or logos in any advertisements, promotional literature or information without the prior written consent of the Bank, and that if such consent is provided that it shall use the name, marks and logos strictly in accordance with the permission provided and with the insertion of the Bank's usual disclaimers. (c) Both Parties shall include the appropriate attribution and disclaimers in new materials prepared in connection with the Reimbursable Advisory Services. 8. Disclaimers and Liabilities. (a) While the Bank will make diligent efforts in its performance of the Reimbursable Advisory Services, the Bank makes no express or implied representation or warranty as to the extent of success that may be achieved in the implementation of any recommendation contained in any work product prepared by or with the assistance of the Bank or Bank Personnel. (b) Without limitation to the immunities and privileges of the Bank under its Articles of Agreement and other applicable rules of law, the Bank shall not be liable to the Client or other third party for any loss, cost, damage or liability that the Client shall incur as a result of the Reimbursable Advisory Services, unless those resulting from the gross negligence or willful misconduct of the Bank or Bank Personnel. Notwithstanding anything herein, the Bank's liability, if any, to the Client hereunder shall not extend to any indirect, punitive or consequential damage, loss of profit or loss of opportunity, nor shall it exceed the amount of the professional fees received by the Bank for its account under this Agreement. (c) The Parties acknowledge and agree that it is not the purpose of this Agreement to create a partnership, joint venture or similar arrangement whereby the Parties could be held jointly liable vis-a-vis third Parties or for any other purposes. Nothing herein shall constitute a commitment by the Bank to provide financing to the Client in respect of the Project or otherwise. - 12 - 9. Governing Law. This Agreement is governed by, and shall be construed in accordance with, the laws of England. 10. Settlement of Disputes. (a) The arties hereto will endeavor in good faith to resolve any differences and disputes under, or in connection with, this Agreement by amicable settlement. Any dispute arising out of or in connection with this Agreement which is not settled by agreement of the Parties shall be finally settled by arbitration in accordance with the UNCITRAL Arbitration Rules in force on the date of this Agreement. In the event of a conflict between the UNCITRAL Arbitration Rules and the terms of this Agreement, the terms of this Agreement shall govern. The place of arbitration shall be Brussels, and the language of arbitration shall be English. (b) Neither the Client nor the Bank shall be entitled in any proceeding under paragraph (a) of this Section to assert any claim that any provision of these Standard Conditions or of the RAS Agreement is invalid or unenforceable because of any provision of the Bank's Articles of Agreement. I1. Privileges and Immunities; Tax Immunity. The Client recognizes and shall take all reasonable steps to give effect to the status, immunities and privileges of the Bank and its Personnel set forth in the Bank's Articles of Agreement and other applicable rules of law. The Parties acknowledge and agree that no provision of this Agreement, nor the submission to arbitration by the Bank, in any way constitutes or implies a waiver, renunciation, termination, or modification by the Bank of any privilege, immunity or exemption of the Bank granted in the Bank's Articles of Agreement and other applicable rules of law. This includes, inter alia, the immunity of the Bank, its assets, income and its operations and transactions, from all taxation and customs duties. 12. Amendments. Any amendment or waiver of, or any consent given under, any provision of this Agreement shall be in writing and, in the case of an amendment, signed by the Parties. 13. Saving of Rights. No course of dealing and no failure or delay by any party hereto in exercising any power, remedy, discretion, authority or other right under this Agreement shall impair, or be construed to be a waiver of or an acquiescence in, that or any other power, remedy, discretion, authority or right under this Agreement, or in any manner preclude its additional or future exercise. 14. Successors and Assignees; No Assignment without Consent. This Agreement binds and benefits the respective successors and assignees of the Parties, provided that none of them may assign this Agreement in whole or in part without the prior consent of the other. 15. Entire Agreement and Counterparts. (a) This Agreement, together with its Schedule, constitutes the entire agreement among the Parties hereto and supersedes - 13 - any and all prior agreements, understandings and arrangements, oral or written, between the Parties with respect to the subject matter hereof. (b) This Agreement may be executed in several counterparts, each of which is an original, but all of which constitute the same agreement. (c) This Agreement has also been prepared in the Polish language at the request of the Client, however, only the English version of the Agreement shall be legally binding. 16. Termination. Notwithstanding termination or expiration of this Agreement, the provisions of this Agreement relating to (i) obligation of confidentiality under Section 5 of this Annex, (ii) the obligations spelled out in Sections 6 on Intellectual Property and 11 on Privileges and Immunities of this Annex; and (iii) the obligation of the Client to pay to the Bank remuneration for the Reimbursable Advisory Services performed prior to the date of termination or expiration of the Agreement, as well as reimbursement of any reasonable costs related to the termination of the Agreement by the Client, shall continue in full force and effect.