Mozambique - Estimating the impact of lowering the cost of primary education and baseline analysis of the poverty and social impact of general secondary school fees poverty and social impact analysis survey (PSIA) 2008 survey

ABCD 69681 The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees PSIA 2008 Survey Final Report March 2009 This report contains 28 pages The World Bank-PSIA Survey Report Final reviewed © 2009 KPMG is a Swiss cooperative of which all KPMG firms are members. KPMG International provides no services to clients. Each member firm is a separate and independent legal entity and each describes itself as such (e.g KPMG Auditores e Consultores, SARL, a Mozambique limited liability company, is a member of KPMG International). All rights reserved. ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 Contents List of Tables ii Executive Summary iii Acknowledgement v 1 Introduction 1 1.1 Context 1 1.2 General Objectives 1 1.3 Specific Objectives 2 2 Research Design and Methodology 3 2.1 Research Design 3 2.2 Research Methodology 3 2.3 Sampling Size 4 2.4 Data Collection Methods and Instruments 5 2.4.1 Data Collection Methods 5 2.4.2 Questionnaire Structure, Length and Content 5 2.4.3 Questionnaire Development and Changes 7 2.4.4 Database Development and Changes 7 2.4.5 Training materials 7 3 Fieldwork Targets and Outcomes 9 3.1 Field Staff Structure and Composition 9 3.2 Training 10 3.3 The Pilot Exercise 11 3.3.1 The Need for Pilot II 11 3.3.2 Critical Issues 11 3.4 The National Phase 12 3.4.1 Sample Performance 12 3.4.2 Field Work Performance 15 4 Lessons Learned and Challenges 21 The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 i © 2009 KPMG Auditores e Consultores, SARL. All rights reserved. ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 List of Tables Table 1: Overall sample of EAs and Households within each province ..................... 4 Table 2: PSIA 2008 Overall Attrition Rate and by Province, ..................................................... 13 Table 3: Response Rate and Key Reasons for Non-response, PSIA 2008.................................. 16 Table 4: Other Reasons for no Interview .................................................................................... 17 Table 5: Summary Display of Field level indicators for PSIA 2008 .......................................... 17 Table 6: Field performance indicators for PSIA 2008 ................................................................ 18 The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 ii © 2009 KPMG Auditores e Consultores, SARL. All rights reserved. ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 Executive Summary The PSIA 2008 survey general objective is to analyze the barriers to school enrolment and retention and particularly the effect that fees and other schooling costs paid at the household level, particularly households with the poorest and most vulnerable children, including girls and orphans. Of interest, is the impact that such costs have on: (a) household decisions with respect to child enrolment and dropout /repetition/completion in primary and secondary school education; (b) school level decisions on budget and expenditures, and (c) perceptions of school quality and educational outcomes at the school and community levels. The PSIA 2008 was a complex exercise, not only on coverage, but particularly on architecture content. The questionnaire was subtle and complex and underwent many changes from its initial formulation. The manual is very long and detailed. The sample was drawn from a data set not meant for a panel study and this introduced a number of complexities and challenges that influenced heavily its implementation in the field. In assessing performance of the PSIA 2008 survey implementation it is shown that the overall attrition rate is 25%. At the provincial level, the highest attrition rates were found in Maputo Province (46%), Sofala (38%), Manica (35%), Maputo City (34%), Manica (30%) and Niassa (24%). The lowest attrition rates were found in Zambézia and Inhambane (12%, each) provinces. In general, it has been very difficult working in urban areas, particularly in Maputo and Sofala, due to mobility, but also incomplete addresses references from the INE database. In some sites it was necessary to replace non-located households when no single one was found after series of attempts at getting hold of the households. Thus the teams visited nationwide a total of 2,234 households. Of those visited only 1,954 households were eligible. Among the eligible households, the teams were able to interview 1,478 households. Note that among those not interviewed, the attrition rate only counts eligible households that were not found after tracking efforts have been exhausted with local leaders, and other key informants. These households could neither be located nor was relevant information for locating them found. In sum, the overall Completion Rate1 is 100%. The success2 and the interview3 rates are 78% and 75% respectively. The response rate for the whole PSIA 2008 sample is 73.6%. Among the non-responses, about 60% the main reason was that the households could not be identified either by the local authorities or any other key informant in the site; 11% could not be located in the enumeration area either because the address did not help to find them or the local authority/key informant could not find them in their area of jurisdiction or had no specific information on where they were at the time of interview. Among the 16.3% of households not interviewed for other reasons were those that moved outside the district – most of which moved for work related reasons (mainly transfers), and; among the death were mostly where either both spouses passed away 1 The completion rate is the ratio of the total number of households visited (confirmed as non-eligible; households interviewed and did not generate tracks; households interviewed and generated tracks, and those not interviewed for a number of reasons) over the total number of households expected for interview. 2 The success rate is the ratio of the number of households actually interviewed over those eligible (both original and generated tracks). 3 The interview rate is the ratio of total number of households interviewed over the total number of sampled households. The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 iii © 2009 KPMG Auditores e Consultores, SARL. All rights reserved. ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 and the remaining members moved outside the district or the household member that made the household eligible passed away. The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 iv © 2009 KPMG Auditores e Consultores, SARL. All rights reserved. ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 Acknowledgement PSIA 2008 survey is a consortium led effort between KPMG Mozambique and Manitou Inc. of the USA. KPMG and Manitou, Inc. formed an alliance in order to assist the World Bank with the collection and preparation of a database with the intent to estimate the impact of lowering the costs of primary education and baseline analysis of the poverty and social impact of general secondary schools fees. We recognize the strategic importance of this project to the World Bank and to the Government of Mozambique. Thus, the strategic alliance formed between KPMG and Manitou, Inc is a blend of complementary skills and resources that are collectively provided to the World Bank with a leveraged solution to the expressed concern of developing a sound Policy for Education Sector and Economic Growth in Mozambique. Under Mozambique´s current circumstances and characteristics we have made all the effort to provide an experienced team led by our best people. During all stages of the project implementation we have benefited from World Bank staff inputs, both from Mozambique and Washington DC. We are thus grateful for all the discussions around questionnaire content and design, training, piloting and monitoring of the actual implementation. In that regard we would like to thank Rui Benfica, Melissa Wells and Louise Fox for their endless patience. KPMG top management would like to extend thanks to its technical team that include Paulo Mole, Binit Varajidas, Marcelino Botao, Momed Jamu, Ricardo Adamo, and all other internal staff, enumerators, field supervisors and provincial coordinators that provided their inputs during project implementation. From Manitou Inc. thanks extend to Tristi Nichols and Joseph Cadwell for their inputs during survey instrument and database design processes. KPMG subcontractors also contributed to the project, and we would like to extend gratitude to Chris Hill, Luis Seródio, Olivia Sigauque, Raul Pitoro, and Fausto Maquina, for the provincial coordination of the fieldwork. We were pleased that the World Bank had selected us on the basis of required skills and capacity to deliver a high quality service that met the required standards. We look forward to being your continuous strategic business partner that offers you the highest level of service and accessibility. The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 v © 2009 KPMG Auditores e Consultores, SARL. All rights reserved.  1 Introduction 1.1 Context On behalf of the Ministry of Education and Culture (MEC) of Mozambique, in 2004 the World Bank and a number of donors commissioned a team of consultants to analyze schools costs for the primary education in Mozambique. The study is known as the Poverty and Social Impact Analysis (PSIA) including both quantitative and qualitative analyses. This study led to policy recommendations to the Ministries of Education and Planning and Finance. One such policy decision was the September 2004 Ministerial Diploma that abolished school fees such as the ASE (social action fee) in all public primary schools (from 1st to 7th grade), effective as of school year 2005. Time has elapsed since the decision to drop out the school fees from households´ education budget lines was made. Thus in the course of 2007, MEC with the support of the World Bank and the group of donors involved in the education sector decided to implement a follow up study to the PSIA of 2004 with the aim of estimating the impact of lowering further the costs of primary education as well as to undertaking a baseline study of the cost impact of school fees on the general secondary education. It is in that context that the World Bank and the group of donors commissioned KPMG to implement in 2008 the survey to gather the necessary data for the analysis. This report is a general documentation of the PSIA 2008 survey process, the main survey objectives, approach and methodology, the data collection methods and instruments, the fieldwork targets and outcomes, as well as lessons learned and challenges. 1.2 General Objectives The PSIA 2008 general objective is to analyze the barriers to school enrolment and retention and particularly the effect that fees and other schooling costs paid at the household level, particularly households with the poorest and most vulnerable children, in particular girls and orphans. Of interest, is the impact that such costs have on: (a) household decisions with respect to child enrolment and dropout /repetition/completion in primary and secondary school education; (b) school level decisions on budget and expenditures, and (c) perceptions of school quality and educational outcomes at the school and community levels. The above objective would be met by analyzing the gathered data at two education levels: primary and secondary levels. In primary education PSIA would analyze the role of recent reforms on achieving PARPA education goals, namely: (a) increasing net schooling rate at EP1+2 school level; (b) increasing the first grade girls enrollment level; (c) decreasing dropout rates and repetition levels; (d) increasing graduation rates, and (e) decreasing student-teacher ratios. In the secondary education level, the objective is to collect baseline data to inform policy makers on measures that could support the achievement of the PEEC secondary education goals,  ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 particularly with respect to education fees. At the level of the school and the community, the objective is to collect initial data on (a) school budgets and the role of fees and (b) perceptions of barriers to the achievement of the strategic goals in the PEEC. 1.3 Specific Objectives The specific objective of the PSIA 2008 survey was, as commissioned by the World Bank, for KPMG to design, and pilot test, review the survey instruments and fully implement the survey nationwide, in order to produce a high quality data set for its panel analysis. The survey would cover a sample of approximately 2,000 households that were part of the IAF 2002/03 from the Instituto Nacional de Estatística (INE). KPMG should match school data to household’s data by identifying which schools the children in the household attended. On the basis of the MEC´s school database, a household would be linked to a specific school, and provide a data set that would include school characteristics for each of the households. Note that the initial draft of the questionnaire was provided by The World Bank, the basis from which KPMG adapted the instrument to Mozambique’s conditions. The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 2 © 2009 KPMG Auditores e Consultores, SARL. All rights reserved. ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 2 Research Design and Methodology The following sections present the research design and methodology followed throughout in collecting, recording and archiving the PSIA 2008 data into a database for future analysis. The raw data in CSPRO, as well as conversions to SPSS and STATA are also provided to the World Bank. 2.1 Research Design Further to the general objective, data gathered under the PSIA survey result from one common sample of Households that provide a substantial sample for children attending primary schools, especially in rural areas, and a limited sample of secondary school students. For purposes of measuring the impact of lowering the costs of primary education, the World Bank decided to use a panel survey design on households drawn from the National Household Budget Survey (Inquérito aos Agregados Familiares sobre o Orçamento Familiar 2002/03, IAF) conducted by the Instituto Nacional de Estatística (INE). Data collected in 2007/8 would then be compared with same household data from the IAF 2002/3 survey. Furthermore, it is important to note that the World Bank has used the approach for the qualitative part, PSIA 2004, and therefore would also like compared it with that from the PSIA 2008, in order to picture the resulting differences about relationships among (1) household expenditures, (2) educational enrollment/retention, and (3) education quality. As the issues of interest are which variables inhibit and support the achievement of the PEEC education goals, together with the likelihood that benefits of decisions taken over the education sector take time to be observed, a longitudinal design is appropriate for examining how households respond to subtle impacts over time. 2.2 Research Methodology As indicated above, PSIA 2008 is follows a panel research design. As such, all members of the original household (as interviewed in IAF 2002/2003) were re-interview together with any new members. Survey instruments were designed in a way that an important task during its implementation required the enumerators to first identify positively all household members interviewed in 2002/2003, before the current household members were all listed. Furthermore, all individuals meeting the eligibility criteria in the household in 2002/3, but were no longer resident in the household in 2008 were followed/tracked for interview, if they were still resident in the district they have been interviewed in before. In addition to household survey data, PSIA 2008 survey used a database from the Ministry of Education and Culture for the identification of schools attended by household members. Where The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 3 © 2009 KPMG Auditores e Consultores, SARL. All rights reserved. ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 the MEC database did not have information on a specific school, survey teams went to the schools and collected the necessary data. 2.3 Sampling Size The sample size for the PSIA 2008 was set to 2,230 selected from the 8,700 households interviewed in 2002/3 for the National Household Budget Survey, (IAF 2002/03). INE sources indicated that IAF´s sample coverage was 99.7% of which 91.8% were included in the original sample and 7.6% were replacement households. The 2002/3 sample was stratified by urban and rural areas in each province, with approximately equal samples taken from each stratum. The primary sampling units (PSUs) were enumeration areas from the 1997 census. In each primary sampling area a fixed sample of 15 households was taken in urban areas and 12 in rural areas. In all, there were approximately 660 primary sample units. The households for PSIA 2008 were selected on the basis of the following criterion: a household interviewed in 2002/3 should have children aged 17 years or younger in 2002/2003. Households meeting the criteria should amount to between 80% and 90% of all households in the sample. This meant that an average of approximately 12 urban and 10 rural households per primary sampling unit included children aged 17 years or younger in 2002/3. These parameters set limits on the distribution of the PSIA 2008 sample. At one extreme the sample could include all the 660 PSUs with approximately 3 households each. At the other extreme the sample could cover about 220 PSUs or less with an average of 9 households each. Table 1 shows the overall sample size per province, and Enumeration Areas (EAs) both Urban and Rural, as well as the number of households in each of the selected EAs. Table 1: Overall sample of EAs and Households within each province Urban Rural Total Provinces Enumeration Enumeration Enumeration Areas Households in Areas Households Areas Households in the sample the Sample in the sample in the Sample in the sample in the Sample Niassa 11 132 10 90 21 222 Cabo Delgado 2 24 17 152 19 176 Nampula 4 48 17 152 21 200 Zambézia 3 36 17 152 20 188 Tete 6 72 15 135 21 207 Manica 11 132 10 90 21 222 Sofala 10 119 10 90 20 209 Inhambane 1 12 20 179 21 191 Gaza 7 84 11 99 18 183 Maputo Província 10 120 8 72 18 192 Maputo Cidade 21 244 0 0 21 244 Moçambique 86 1,023 135 1,211 221 2,234 The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 4 © 2009 KPMG Auditores e Consultores, SARL. All rights reserved.  ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 Note that in addition to the original target, eligible households members were to be tracked as long as they have moved out from the original households, and remained within the geographical limits of the districts where they have been interviewed in 2002/2003. 2.4 Data Collection Methods and Instruments This section provides an overview of the data collection methods and instruments developed and used for PSIA 2008 survey. 2.4.1 Data Collection Methods PSIA 2008 was carried out using the traditional pen-and-paper interviewing at the household level through a formal questionnaire. Collected data was computer entered, initially in the field and later re-entered centrally for quality consistency check. The survey used GPSs for the geo-location of households and schools. Data entry was undertaken with resort to CSPro. CSPro is a software system specifically designed for survey data entry developed by the US Bureau of the Census for use in developing countries. CSPro has been used over a number of years both by INE and for the Agriculture Surveys (TIA) in the Ministry of Agriculture. The TIA surveys used CSPro for direct data entry in the field. Members of our team have worked with this application. 2.4.2 Questionnaire Structure, Length and Content The PSIA 2008 questionnaire is complex. It covers a diversity of topics within one interview. Some sections involve collection of large amount of detail. Other sections attempt to collect with precision information that is not precise or varies greatly in structure from person to person or household to household. The majority of the questions are at the household member level with pre-coded responses. However, there are sections with questions referring to occupation by household member, goods, expenditure categories, food consumption, and expenditures on semi- and non-durable goods per household. The questionnaire consists of two main parts: (i) Pre-filled information from the 2002/2003 IAF survey; and (ii) sections to be filled during the 2008 interviews. The two parts make a distinction between an original household to that required for a member to be tracked. Overall, the questionnaire contains 12 sections. Below is a brief description of the content of each of the questionnaire’s sections: Section A – The Questionnaire Cover, which consist of two components: (i) pre-filled component with information from 2003; and (ii) a component to be filled by the enumerator. The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 5 © 2009 KPMG Auditores e Consultores, SARL. All rights reserved. ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 • The pre-filled component identifies the household, and include geographical information, the name of the household head, the home address where available, and the household eligibility criteria – that is, if the household had either children 17 years old or younger, or people studying full time during the year 2003; and • The component to be filled by the enumerator with 2008 geographical information (if different from that of 2003), as well as the 2008 household eligibility criteria, household location, confirmation of whether the household was or not interviewed in 2008. Furthermore, the enumerator should indicate if the household has moved out of the district, the current location, the name of the enumerator filling the questionnaire, the data and time of interview and the geographical coordinates of the household house. Section B – This section shows the household composition in 2003 and 2008. It includes basic bio-data for each of the household member. The section helps in identifying positively the household members from 2003 whose information is required in other sections. Section C – Questions included in this section explore the maximum education level attained by each of the household members, and register the reasons members may have not go to school, transport means used to attend school [Sub-section C0]; education profile for the 1998-2008 time period [Sub-section C1]; and education expenses, both in cash and in kind for the last school year [Sub-section C2]. Section D – This section determines whether the household members that 5 years or older are involved in any economic and other activities. It includes four different employment activities: (i) domestic activities – economic and others [Sub-section D0 Part B], (ii) wage employment [Sub-section D1], (iii) self-employment [Sub-section D2], and (iv) agricultural activities – agricultural and livestock [Sub-sections D0 Part C and D3]. Section E – This included a single page with questions related to housing characteristics and transport means used to access services. Sections F and G – These sections contain questions related to household consumption. There are five parts: (i) durable assets ownership [Sub-section F], (ii) household food, beverage and tobacco for the last 7 days [Sub-section GA], (iii) verification of food purchases for the past 30 days [Sub-section GB], (iv) more frequent non-durable goods and services purchased in the last 30 days [Sub-section GC], and (v) Durables, Semi-Durables and services paid in the 12 months [Sub-section GD]. Section H – This section refer-se to respondent’s perspectives and perceptions about the education system, the quality of the education provided, and the level of involvement of the civil society in school decision making. The Final World Bank-PSIA 2008 Survey Report - 9 April 2009 6 © 2009 KPMG Auditores e Consultores, SARL. All rights reserved.  ABCD The World Bank Estimating the Impact of Lowering the Cost of Primary Education and Baseline Analysis of the Poverty and Social Impact of General Secondary School Fees March 2009 Section I – The questions in this section record transfer received and sent in the last 12 months, its frequency and sources. It registers transfers from both public and private sources. Section J – This is the last section of the questionnaire. It contains only one question on the household poverty status. It asks if the household holds any Poverty Certificate. 2.4.3 Questionnaire Development and Changes The time required to perfect survey instruments to get to an agreeable form and state was huge. The observations made above on changes made during training on both pilots are an example of the complexity of the PSIA questionnaire. This had an obvious impact on the process of developing complementary instruments for the training and testing. Every change in the questionnaire introduced other changes including: The possibility of needing to make other changes within the questionnaire; The retraining of field staff, in particular field staff needed to “unlearn