Colombia's targeted education voucher program : features, coverage, and participation

17813 Working Paper Series on Impact Evaluation ofEducation Reforms Paper No. 3 Colombia's Targeted Education Voucher Program: Features, Coverage, and Participation Elizabeth Kinga Laura Rawlingsa Marybell Gutierrezb Carlos Pardob Carlos Torresb September 1997 Development Economics Research Group The World Bank aThe World Bank; bGovernment of Colombia This paper is a product of the research project, "Impact Evaluation of Education Projects Involving Decentralization and Privatization" (RPO 679-18) being carried out under the guidance of Elizabeth King by the Development Economics Research Group of the World Bank. The findings, interpretations, and conclusions are the authors' own and should not be attributed to the World Bank, its Board of Directors, or any of its member countries. Comments are welcome and should be sent directly to the author(s). For copies, please send request to Selina Khan at SKhan8@Worldbank.org Colombia's Targeted Education Voucher Program: Features, Coverage, and Participation Elizabeth M. King and Laura Rawlings The World Bank Marybell Gutierrez, Carlos Pardo, and Carlos Torres Government of Colombia September 1997 ABSTRACT -- This report describes and reviews Colombia's secondary school voucher program. The program which is targeted at poor students has reached more than 100,000 students since it began in 1992. The cost of the vouchers is shared by central and local governments, with the central government financing 80 percent and participating municipalities financing the rest. Not all municipalities have joined the program, but participation is more common among those that have a greater demand for secondary education, where the public school capacity to meet this demand is limited, and where there is a larger private sector presence. As intended by the designers of the program, the private schools that have elected to participate in the program tend to be those who serve poorer families. Yet, these schools appear to offer an education that is at least of comparable quality to that offered in public schools -- at a lower cost to the government than public schools. *** We thank Berk _zler and Dan Levy for their assistance in organizing and analyzing the data; Michael Potashnik, Martha Laverde, Claudia Uribe, Lilian Rosero, Gabriel Martinez Morales and the staff of the ICETEX regional offices for all of their contributions. We are also grateful for comments from the Projects Coordination Unit at the Ministry of Education and from National Planning Department staff who attended a seminar on this report. This study has been funded by the Policy Research Department of the World Bank and its Research Support Budget under RPO 679-18, and by the Government of Colombia. However, the findings, interpretations, and conclusions are the authors' own and should not be attributed to the Government of Colombia, the World Bank, its Board of Directors, or any of the World Bank's member countries. Comments are welcome and should be sent directly to the authors. For copies, please contact Selina Khan at Skhan8@worldbank.org. Table of Contents I. IN TR O D U C TIO N ..............................................................................1 II. VOUCHERS AS A POLICY TOOL.......................................................3 Pros and Cons Evidence from Other Countries II. FEATURES OF THE COLOMBIAN VOUCHER PROGRAM......................7 The Central-Local Government Partnership Qualifying for the Voucher Program: Students and Schools Program Administration, Cost and Finance Program Size and Coverage IV. DETERMINANTS OF MUNICIPAL AND SCHOOL PARTICIPATION.......21 Data Sources Municipal Participation School Participation V. QUALITY IN VOUCHER AND NON-VOUCHER SCHOOLS...................31 VI. CONCLUSIONS AND FURTHER QUESTIONS ....................................37 REFERENCES 2 I. INTRODUCTION In 1992 the Government of Colombia initiated an educational voucher program for secondary education. Its aim is to increase the transition rate from primary to secondary education by addressing one of the constraints faced by poorer students -- the shortage of space in public schools, especially in large cities. The program does this by tapping into the capacity of private schools which were already enrolling 40 percent of students at that time.' Now five years after it started, the program has involved more than one-fifth of the municipalities in the country and close to 2,000 private schools, and has supported more than 100,000 secondary students. It is undoubtedly one of the largest programs of its kind in the world. This report is the first phase of an evaluation of the program. The central government is currently examining its support for this type of public subsidy for secondary education. But whether it continues or not in Colombia, there are lessons to be gained from the design and implementation of such a major program for the future. In this report we first review the program features in detail. We then examine its coverage and the participation of municipalities and schools, using data we have compiled from various administrative sources. Finally, we discuss the determinants of program participation in order to assess its success in reaching its intended beneficiaries, using the econometric results obtained by a background study (King, Orazem, and Wohigemuth 1997). An important feature of the program has been the exercise of choice by municipalities and schools, as well as students, in participation. Municipalities can choose to join the program or not, and those that do also agree to financing 20 percent of the cost of the voucher, a partnership that is consistent with the country's ongoing policy of decentralization. Within participating municipalities, private schools must elect to join the program or not, and eligible students must apply to qualify for the voucher. The report is not able to address other questions, such as those about the accuracy of the targeting of the vouchers at the student level, or the program's net impact on the enrollment, due to lack of data 3 during the preparation of this report. These issues are discussed briefly at the conclusion of the report. In summary, the principal findings of this report are: Which municipalities are participating in the voucher program? A total of 216 municipalities have participated in the program. The decision of municipalities to participate has depended on their relative supply of public and private secondary school places as well as on the number of-students enrolled at the primary level which measure the potential demand for secondary education and the need to augment capacity at the secondary level. In general, large urban municipalities with greater numbers of primary school students, more limited public school capacity, and larger existing private school capacity are the most likely to participate in the program. Which private schools are part of the program? In 1996, more than 1,700 private schools were participating in the voucher program. The statistical analysis reveals that non- profit schools and vocational and technical schools tend to have a higher propensity to participate than do other schools. Size of enrollment has a positive effect on this propensity but it is not statistically significant. Schools that charge higher tuition per student, up to the maximum value of the voucher, tend more to participate in the program, but those that charge fees higher than this maximum value are usually not in the program. On average, fees are 40 percent less in participating, than in non-participating, private schools. Schools that charge very low fees do not participate since these schools are probably affordable to the poor even without the program and, to the extent that fees are correlated with school quality, students may prefer to use their vouchers in higher quality schools. What is the quality of education offered by participating private schools? This is a question which has been frequently raised by policymakers since some anecdotal evidence suggest that the program is dominated by low-quality schools. Two pieces of information from schools which were in existence in 1993 strongly suggest otherwise. First, multivariate results In 1992, 51 percent of 13-19 year olds in the poorest quintile living in urban areas were 4 indicate that private schools of the lowest quality were not interested in participating in the program. Second, a simple comparison of the scores on achievement tests in mathematics and language shows that, on average, students in participating schools perform at least as well as students in public schools in both tests. Hence, for about two-thirds of the per-pupil cost of public school, the government is able to offer many poor students secondary education that is comparable in quality to public schools. The sections below elaborate on these findings. Before turning to a discussion of Colombia's program and our evaluation, however, we first review the relevant policy literature. II. VOUCHERS AS A POLICY TOOL Pros and Cons Broadly speaking, all public subsidies to private schools are vouchers. However, the term has been most commonly associated with the use of coupons which partially or fully compensate parents for the cost of private school tuition. A distinguishing feature of vouchers is that instead of determining which private schools to subsidize and focusing on the "supply side", the government transfers this power to parents and students by allowing them to choose among schools. This type of "demand-side" financing has been touted as a means by which market forces can be used to compel public and private schools to provide better quality education efficiently. Nonetheless, most systems of public education allow parents only a limited choice of where their children may attend school. In the absence of private schools, local public schools have monopoly power and lose no resources if they provide a poor education. Voucher proponents claim that by allowing parents to choose where their children can attend school, vouchers will force schools to compete with one another for students. Schools that provide poor educational services will be forced to improve or face the threat of closure. enrolled in school compared to 75 percent of the richest quintile (World Bank 1994). 5 The view that vouchers will improve school efficiency and quality is not universally held, however. Perhaps the strongest objection is that schools will use selective admissions policies to avoid higher cost students: those with mental or physical disabilities, the less able, and the disruptive. The high cost students will be relegated to public schools that will have fewer resources to deal with such students. In addition, to the extent that poorer children will have greater difficulty meeting entrance requirements at selective schools, the true choices available to poor parents may not, in fact, expand. It is argued that vouchers may result in more, not less, segregation of students, and will simply be a means of shifting resources from public schools educating the relatively poor to private schools educating the relatively wealthy. Finally, critics also contend that an over-reliance on vouchers would lead to a loss of civic values and weaken a country's common social fabric by allowing students to select schools with a specialized curriculum, specific cultural orientation or schools that serve a particular constituency. However, voucher programs need not define the way public subsidies are distributed. They do not have to be universal like the Chilean program which any student can use in a public or private school. Rather, they can be targeted toward the poor, as is the case in the Colombian program and in the Milwaukee, Wisconsin system, or toward particular demographic groups, as is the case in the Bangladesh secondary scholarship program for girls. When used properly, targeting helps avoid the concerns that vouchers constitute a net subsidy of the wealthy by the poor, or a way of robbing the public education system. In countries where the demand for schooling may exceed the public sector's capacity to build and manage schools, vouchers represent a relatively inexpensive and rapid way to expand school places. This eases the pressure on public school resources, possibly improving the quality of learning as a consequence. In addition, private schools have been found to provide better learning than public schools (Jimenez, Lockheed and associates 1996; James, King and Suryadi 1996), so this shift of enrollment could lead to an overall improvement in student achievement. 6 By design, Colombia's voucher program is targeted toward poor students who have already been attending public school, thus countering any claim that it is a net subsidy of the wealthy by the poor. It was established to relieve the overcrowding in public schools, but not to substitute for them. Its principal aim is to increase net enrollment without sacrificing quality. The program takes advantage of the fact. that private schools can relatively inexpensively and rapidly expand opportunities for children who would otherwise be unable to enroll in any secondary school given the space constraints in public schools. There are potential benefits too for students enrolled in public schools since the policy is that public schools will retain their resources even if they lose students. Finally, by encouraging more private provision without directly subsidizing particular schools, the voucher program can be a vehicle for demanding better performance from private schools. Evidence from Other Countries How successful have voucher systems in other countries been in achieving their objectives of increasing enrollment or targeting specific groups? The empirical literature to draw upon is not extensive, and there is especially little for developing countries or for the use of targeted, demand-side subsidies in the education sector. Chile is one of the few developing countries to have used an educational voucher program. Although the final word on the success of this program is yet to come, the results from existing studies are instructive. The program introduced in 1980 in pre-school, primary and secondary schools provided a per-student subsidy equivalent to the average per-student expenditures of the Ministry of Education. The program allows students to enroll in the participating private school or public school of their choice. To participate, schools have to comply with norms governing the curriculum and infrastructure as established by the Ministry of Education; they also have to offer their educational services free of charge to their students (Espinola 1995). As part of the reform, administration of the government-owned public schools was transferred directly to the municipalities, administration of the vocational secondary 7 schools was transferred to private non-profit organizations often associated with a specific industry, and private groups and individuals were encouraged to set up their own schools. Both privately and municipally-administered voucher schools are paid directly by the central government on the basis of their total enrollment, whereas the vocational schools run by non- profit organizations are paid lump-sum amounts (Schiefelbein 1991). A few evaluations of Chile's program have been conducted. The impact on overall enrollment rates is not clear, but private school enrollment appears to have risen sharply as a 2 result of the program. With respect to achievement, studies of math and language test scores have revealed that participating private schools consistently outperformed public schools at both the primary and secondary level but never reached the achievement levels of private non- participating private schools (Rodriguez-Grossi 1985 as quoted by Castafieda 1992; Aedo and Larraiaga, as quoted by Espinola 1995). Other assessments have brought out less laudatory results of the program. The principal objection to the program is a charge that the program can be blamed for the widening of the gap in achievement among schools (Schiefelbein 1991; Prawda 1993). One of the principal differences between the programs in Chile and Colombia is that Colombia's program targets poor students. Two cases from very different locations -- Bangladesh and Milwaukee, Wisconsin-- more closely resemble Colombia's program in this respect. In Bangladesh, the Female Secondary Education Scholarship Project was introduced as a pilot in 1982 and by 1988 had reached over 20,000 girls. This program is targeted to very low-income households and allows the girls to attend secondary school by reducing tuition by half in the first three grades of secondary school. In project areas, female secondary school enrollment has gone from 27.3 percent before the program was initiated to 43.5 percent by 1987 (Bellew and King 1993). 2 During the period 1980-86, enrollment in private schools participating in the program more than doubled, and by 1986, over 30 percent of the students in preschool, secondary and adult education were enrolled as voucher students (Castafieda 1992). This rise in private school 8 In 1990 the city of Milwaukee, Wisconsin, USA introduced a publicly-funded voucher scheme that allows low-income students to attend a non-sectarian, private school of their choice. Despite its small size (832 students enrolled in eleven schools in 1994), the program has received a great deal of attention and has been evaluated. The value of the voucher is set at the annual per-student subsidy given by the state to public schools ($2,987 in 1993-1994) which is received by private participating schools in lieu of tuition and fees from the students (Witte 1996). The first evaluation of the program found that it appears to be well targeted -- the average income of the beneficiary households is half the average family income in the city's public schools -- but the variance in test scores between public and private participating schools offers no clear evidence that choice schools produce better or worse students than public schools (West 1996; Witte 1996). Another evaluation obtained more positive results. Using a natural experiment generated by the voucher schools' use of a lottery to select qualified applicants, it found that "students enrolled in the choice schools for three or more years substantially outperformed, on average, a comparable group of students attending Milwaukee public schools" and that "attendance at a choice school enhances academic performance, as measured by standardized test scores." (Greene, Peterson and Du 1996). III. FEATURES OF COLOMBIA'S VOUCHER PROGRAM Colombia's voucher program was launched as part of a national transformation that began in the 1980s and was consolidated in 1991, shifting power from the central government to local institutions and expanding the role of the private sector in the provision of public services3. This wave of decentralization encompassed the provision of social services as well. enrollment is directly related to the program since enrollment in private non-program schools fell marginally during this period. 3 During the course of the 1980's, a series of decentralization initiatives were introduced that began to reverse the well-established legacy of central control over politics, finance and the provision of public services. These initiatives included introducing the popular election of mayors and governors instead of continuing with the tradition of having appointments for local 9 Municipalities were given direct responsibility for the construction and maintenance of schools, and a greater say in the provision of basic education. The voucher program was introduced as part of the 1992 Decentralization Bill which transferred decision-making authority and financial resources directly to departments and municipalities and provided for greater autonomy for schools. The bill and the voucher program therein support the government's goals of expanding secondary enrollment to full coverage by the end of the century, and of seeking more opportunities for private sector participation. The Central-Local Government Partnership The voucher program is a partnership between the central government and local governments. Municipalities may or may not participate in the program. If they do, they must co-finance the cost of the vouchers. The central government pays 80 percent of the cost of the voucher program and participating municipalities pay the remaining 20 percent. Municipalities must also determine the number of vouchers they are willing to fund up to the maximum set by the central or departmental government.4 Promotion, information collection offices made by the central government (Montenegro 1995) as well as a bold move by Congress forcing the central government to share revenues from the newly-created valued-added tax with local jurisdictions (Hommes 1995). 4 The total number of vouchers to finance annually is decided upon by the central government. It does so by reviewing first the number of renewed vouchers to be funded in order to ensure that existing voucher recipients are covered in the budget, along with the administrative fees needed for running the program. A system for distributing new vouchers across departments and municipalities is described in the legal resolution establishing the voucher program. This directive instructs the central government to consider present levels of secondary school enrollment among lower-income households as well the unmet demand for secondary education. To assess the unmet demand for secondary education, a formula is suggested that uses department or municipal level data to construct a "demand for secondary education gap" based on the number of primary school graduates minus the number of available secondary school spaces in public schools. The new vouchers can then be distributed according to the size of the gap, taking into account the available information on enrollment rates among low-income students. 10 and dissemination, and monitoring aspects of the program are also shared by central and local authorities. Fragmentary evidence suggests that entry into the program has been less structured than the legal directive would suggest. Official correspondence between the Ministry and the departmental governments indicate that there were a significant level of consultations within departments as to which municipalities would participate in the program. In certain cases departmental governments have played a direct hand in the participation by assisting their municipalities with co-payments fees or setting up alternative voucher programs.5 All in all, it would seem that the process by which municipalities participate in the program has been a mixture of persuasion, consultation and selection. In Section IV, we "reconstruct" the selection process by identifying the municipal characteristics that made program participation more or less likely. Despite the multiple paths to participation, we find systematic patterns. Box 1. Qualifications for Student Participation in the Voucher Program * Vouchers are issued to entering 6th grade students whose reside in low-income neighborhoods and have previously attended public school. * Vouchers can be renewed upon the completion of each academic year until the student graduates from secondary school. Students who fail a grade drop out of the program automatically. * When the demand for vouchers exceeds their supply, vouchers are allocated through a lottery. * Starting in 1996, participation has been restricted to private, not-for-profit schools. Departments are sub-national political entities roughly equivalent to states in the United States of America. In the municipalities of Cesar, Casanare, Meta, Norte de Santander and Guajira, the departmental government is assisting the municipality with co-financing the voucher program. 11 Qualifying for the Voucher Program: Schools and Students To participate in the voucher program, a private school must be located in a participating municipality and must be a licensed school offering a traditional academic or vocational curriculum. All schools must provide information on their matriculation and tuition fees at the time of entry. To regulate the quality of participating schools, new schools that are interested in the program are required to present a three-year curriculum plan. As of 1996, participation has been restricted only to non-profit schools. To qualify for a voucher, students must be entering the sixth grade, must be under 16 years old, must reside in a poor 6 neighborhood, must have gained admission to a secondary school participating in the program, and must have attended a public primary school. The final requirement was introduced in order to avoid subsidizing students who would have attended a private secondary school even without the voucher program. During the voucher application process, supporting evidence is required about the student's socioeconomic status. This verification usually comes in the form of a utility bill which contains the status classification of the student's neighborhood. The application form also requires the signature of the rector of the admitting school. Program participation is renewable for the duration of a students' secondary school education provided that the student performs satisfactorily in the previous grade. Vouchers are not transferable among schools or students. Each year, the demand for the vouchers has exceeded supply in almost all participating municipalities. Although systematic data are not available, conversations with administrators in the local ICETEX offices suggest that anywhere from 20 percent (in Atlantico) to 90 percent (in Antioquia) of the qualifying applicants receive the subsidies. In the majority of cases of 6 The targeting criteria used to ensure that vouchers are issued to low income students is based on a neighborhood stratification scheme that ranks neighborhoods on a scale of 1-6 from poorest to richest. Only students residing in a stratum 1 or stratum 2 neighborhood are eligible to participate in the program. 12 over-subscription, a lottery has been used to select the beneficiaries, although in one case the local ICETEX office made an effort to select the most academically qualified students.7 Program Administration, Cost and Finance The voucher program is managed by ICETEX, the Colombian Institute for Education Credit and Training Abroad, a public sector institution whose main responsibility is administering study abroad programs. Under a special contract with the Ministry of Education, ICETEX works with participating municipalities in determining the number of vouchers to be funded, certifying that schools meet requirements for participation, and monitoring the implementation of the program. ICETEX's central office in Bogotd is responsible for setting program norms and ensuring compliance. Its responsibilities include training local offices in the use of the software developed by the central office that allows for the random selection of students in the case of over-subscription for the program; setting the maximum value of the voucher each year; monitoring the inscription of schools and students in the program; and maintaining national records on program costs and coverage. The 14 ICETEX regional offices throughout the country are responsible for promoting the program locally; recruiting schools; selecting students; and conducting on-site monitoring in the schools at least three times a year. ICETEX was not prepared for the immense burden of monitoring the program, as is evidenced by the central office's complete lack of municipal-level program data in 1992 and 1993, and by the wide variation in administrative capacity in the regional offices. In these regional offices, monitoring the terms of the program is often the responsibility of part-time workers facing arduous travel demands in order to reach and monitor the participating schools. Financing for the voucher program is centrally administered by the Colombian Social Investment Fund (SIF) which is responsible for channeling the central government's share of 7 The use of this lottery provides some valuable avenues for assessing program impact, given the random selection of students for participation in the program. These avenues are presently being explored for use in a student-level evaluation. 13 payments. A participating school receives voucher funds through a bank account set up for each of them in the Central Mortgage Bank (Banco Central Hipotecario), Colombia's national mortgage bank which maintains branches throughout the country. The account is credited three times a year after the school presents its voucher students' signatures. Reports of payment delays are common. ICETEX estimates that it takes close to two months for a school to receive payments for its voucher students once the school has submitted its roster of enrolled students. As of March 1996, for example, there were still outstanding 1995 payments that had not been made to the schools. Much of the delays have to do with the complex procurement channels that must be navigated three times a year in order for schools to finally receive payments. The payments process illustrated in Figure 1 begins at the Ministry of Finance which authorizes the Department of Treasury to release the central government funds to the Social Investment Fund which in turn transfers them to the municipal control board, since under the new legislation, municipalities must administer many aspects of their social sector budgets. Then, once the municipality has contributed its share of the cost, the municipal fiduciary agent transfers the appropriate amount into each school's bank account. Whether the payment delays can be attributed to the fact that the Social Investment Fund has been involved only since its creation in 1994 and needed to learn the financial administration ropes, or whether there are more systemic problems obstructing the payments process needs to be examined more closely. The voucher is designed to cover an annual matriculation fee and ten monthly tuition payments. The value of the voucher depends on the current tuition and matriculation fees in each school, with a maximum set on this value each year. The maximum value of the voucher has grown from about 104,000 colombianos in 1992 (about US$128 using 1992 nominal exchange rates) to 190,000 colombianos in 1996 (about $177 using June 1996 nominal exchange rates). This ceiling is adjusted annually according to the estimated national inflation rate. This inflation rate is also used to automatically adjust the value of all vouchers, irrespective of the actual changes in the tuition fees of individual schools. 14 The mean value of a voucher grew from 94,603 colombianos in 1994 to an estimated 194,510 colombianos in 1996. In 1994, a year for which actual (not estimated) payments are available, the mean value of vouchers by department, ranged from 59,679 colombianos in Choco to 119,122 colombianos in Quindio. Voucher payments to the schools are made three times a year. The first two payments and a portion of the third use the central government's contributions, with the municipal government's share paying the rest of the third payment. When voucher students drop out in the middle of the academic year, an adjustment is made in the remaining annual payments to the school. These adjustments are made with reference to a signed list of voucher students which must be presented by the school director in order for the school to receive its payments The Social Investment Fund manages the transfer of funds from both the central government and the municipalities directly to each school's bank account. Figure 1. The Voucher Payments Process Ministry of Finance Treasury Social Investment Fund Municipal Control Board Municipality's Fiduciary Agent School's Central Mortgage Bank Account 15 What does it cost to administer the program? ICETEX charges a commission fee of 3.5 percent of the central government's share of the cost of the program. The Social Investment Fund charges less than 0.2 percent of the total cost of the program and, for each of the three transactions carried out annually with a participating school, the Central Mortgage Bank charges 1,000 colombianos. Table 1. Cost of the Voucher Program, 1994-1996 (in colombianos) 1994 1995 1996 Central Government 6,872,678,671 11,077,328,074 12,605,664,000 Municipalities 1,718,169,668 2,769,332,018 3,151,416,000 Commissions 324,297,206 443,093,123 504,226,560 Total Cost 8,915,145,545 14,289,753,215 16,261,306,560 Cost per Beneficiary (A)) 98,175 161,222 200,735 Cost per student in a Public School 182,815 203,309 261,397 (B) Relative Cost of Voucher Program 1.86 1.26 1.30 (B/A) Notes: Actualfigures are used for 1994 central government payments and commissions; estimated figures are used for 1994 municipal payments and all payments in 1995 and 1996. Estimates of the share of education expenditures devoted to secondary education are based on an average from 1980 to 1990 as reported by UNESCO since the figures reported by MEN do not include a breakdown between spending at the primary level and the secondary level. Sources: ICETEX Central Office for costs and number of voucher students; DNP and MEN for public education expenditures; UNESCO Yearbook 1995 for share of education expenditures in secondary education. Even when the administrative costs of the program are added to the value of the vouchers themselves, the total program cost per beneficiary appears to be substantially lower than the cost of providing public schooling. As Table 1 shows, in 1995 the average cost per secondary student in public schools was 203,308 colombianos (about US$232 using June 1995 nominal exchange rates), about 25 percent more than the 161,222 colombianos (about US$184 using June 1995 nominal exchange rates) average cost of supporting a voucher student. This 16 calculation is based on a very conservative estimate of the per-student cost of public education, so as not to tilt the analysis in favor of the cost-effectiveness of the voucher program. Program size and coverage The program was established in late 1991 and began in earnest in 1992. By 1995, 216 out of the 1,067 municipalities in the country had decided to join the program and close to 90,000 students were using a voucher to attend approximately 1,800 participating private schools. As of 1995, voucher students accounted for just under 1 percent of students in all secondary schools and 2.6 percent in private secondary schools (National Planning Department 1996). The growth in the size and coverage of the program is detailed in Tables 2-4 which list the number of participating municipalities, schools and students, by department, from 1992 to 1996. These participation data which have not been previously compiled at all these levels have been collected from three sources: the ICETEX central office, ICETEX regional offices and the Ministry of Education. Following suggestions from the Ministry of Education and ICETEX, we have used ICETEX regional office data when possible, although these have been generally unavailable prior to 1994. Despite the care that we took in compiling these data, small discrepancies among the sources remain. Nonetheless, we are reasonably confident that the conclusions in this report are valid, because they depend less on exact numbers than on the robustness of the patterns.8 The majority of the participating schools are in large urban areas where private schools are generally concentrated. The number of participating schools in these cities is often twice or three times the number of public secondary schools. Colombia's ten largest municipalities account for 55 percent of vouchers students and 62 percent of voucher schools, as is illustrated in Figure 2. In each of these cities, over 40 percent of the private schools have elected to participate in the program, with the participation rate reaching as high as 91 percent in Barranquilla. 17 Table 2. Number ofParticipating Municipalities by Department, 1992-1996 Department 1992 1993 1994 1995 1996 Antioquia 15 19 25 23 20 Arauca 0 0 1 1 2 Atlantico 1 10 11 11 11 Bogota 1 1 1 1 1 Bolivar 1 6 7 7 7 Boyaca 0 5 15 13 13 Caldas 1 2 3 3 3 Caqueta 0 1 1 1 1 Casanare 0 2 3 3 2 Cauca'A' 0 3 3 3 3 Cauca'B' 9 9 11 13 13 Cesar 0 9 11 10 9 Choco 0 1 2 2 2 Cordova 0 8 10 10 10 Cundinamarca 0 14 18 20 20 Guajira 0 2 3 4 4 Huila 1 11 15 15 15 Magdalena 1 6 7 7 7 Meta 5 8 8 8 8 Norte de Santander 6 6 6 6 6 Nariio 13 14 11 11 11 Putumayo 1 1 1 2 2 Quindio 1 1 2 1 1 Risaralda 1 3 4 4 4 Santander 2 4 4 4 4 Sucre 3 .7 7 7 7 Tolima 0 5 6 7 7 Valle 16 15 19 19 18 COLOMBIA 78 173 215 216 211 Source: ICETEX Central Office Statistics for data from 1992-1993; ICETEX Regional Office Statistics for data from 1994-1996 Tables 2-3 point to a slight overall decline in municipal and school participation from 1995 to 1996. In some departments, this decline is more pronounced. In certain departments, such as Antioquia and Nariflo, may at least partially be explained by the introduction of locally- funded and administered educational subsidy compete with the national voucher program. In 8 For additional information on the sources and reliablity of the data used in this report, please refer to King, Orazem, and Wohlgemuth (1997). 18 general, schools and municipalities may also be reacting to the decline in the availability of new vouchers (Figure 3). Since the municipal government decides on the distribution of the vouchers among its schools and students, it is not surprising that the number of voucher schools and students has also declined during the same period. Table 3. Number ofParticipating Schools by Department, 1992-1996 9 Department 1992 1993 1994 1995 1996 Antioquia 126 131 127 116 113 Arauca 0 0 1 4 4 Atlantico 206 244 245 224 224 Bogota 577 417 450 450 400 Bolivar 69 80 92 85 86 Boyaca 0 17 29 29 29 Caldas 19 19 21 21 21 Caqueta 0 3 4 4 4 Casanare 0 3 4 4 3 Cauca'A' 0 4 5 5 5 Cauca 'B' 26 33 35 42 42 Cesar 0 33 39 45 44 Choco 0 12 14 7 7 Cordova 0 26 30 30 30 Cundinamarca 0 70 63 65 65 Guajira 0 7 13 16 16 Huila 15 29 35 35 35 Magdalena 38 46 60 60 60 Meta 23 36 36 37 37 Norte de Santander 58 57 63 70 70 Narifio 49 49 38 48 48 Putumayo 1 1 1 2 2 Quindio 0 4 6 5 5 Risaralda 14 19 18 17 20 Santander 60 66 66 56 57 Sucre 31 37 41 41 41 Tolima 0 30 34 36 36 Valle 217 215 216 241 240 COLOMBIA 1,529 1,688 1,786 1,795 1,744 Source: ICETEX Central Office Statistics for data from 1992-1993; ICETEX Regional Office Statistics for data from 1994-1996 9 A participating school is defined as a school with voucher students. If a school decides to no longer participate in the voucher program, ICETEX helps relocate the voucher students to another participating private school in the area. 19 Table 4. Number of Voucher Students by Department, 1992-1996 Department 1992 1993 1994 1995 1996 Antioquia 10,019 11,865 10,490 8,728 8,134 Arauca 0 0 83 182 169 Atlantico 5,462 6,444 7,766 7,237 6,614 Bogoti 9,486 6,456 10,735 11,657 11,053 Bolivar 3,952 5,882 6,624 6,609 5,983 Boyaca 0 289 794 882 841 Caldas 617 562 577 603 630 Caqueta 0 484 884 962 933 Casanare 0 53 119 157 99 Cauca'A' 0 188 245 265 230 Cauca 'B' 1,760 2,690 3,872 3,458 3,163 Cesar 0 762 1,660 2,037 1,596 Choco 0 208 274 286 241 Cordova 0 899 2,277 3,095 2,635 Cundinamarca 0 0 2,034 2,338 2,072 Guajira 0 215 400 557 490 Huila 1,363 2,503 2,692 2,436 2,380 Magdalena 1,186. 1,441 2,690 2,973 3,239 Meta 1,000 1,589 2,403 2,462 2,145 Norte de Santander 2,860 4,096 5,584 5,583 5,687 Narifio 1,840 2,900 3,640 3,360 2,564 Putumayo 80 110 157 124 95 Quindio 0 273 465 538 485 Risaralda 1,516 2,353 3,653 3,276 2,589 Santander 2,511 3,389 4,789 4,464 4,715 Sucre 849 1,744 2,832 2,686 929 Tolima 0 679 1,443 1,651 1,548 Valle 5,072 9,075 11,627 10,028 9,750 COLOMBIA 49,573 - 67,149 90,809 88,634 81,009 Source: ICETEX Central Office Statistics for data from 1992-1993; ICETEX Regional Office Statistics for data from 1994-1996 Table 4 reveals that the number of students grew yearly by an average of 35 percent from 1992 to 1994, but fell by an average of 5.5 percent per year from 1994 to 1996. Two factors explain the decline of students in the voucher program. The first, and key factor, is the drastic reduction in the number of new vouchers (see Figure 3). A second factor that accounts for the tapering off of the curve for the number of renewed vouchers in Figure 3 is the rate at which students drop out of the program. This in turn reflects two forces at work. One is the prevailing non-promotion rate (a combination of dropout and grade repetition rates) from grade 20 6 to 10, which is 28 percent on average from 1992 to 1996.10 Additionally, the large increase in tuition costs in 1996 as a result of the change in the pay schedule of teachers, coupled with the loss of political support for the voucher program, has resulted in a less than adequate rise in the maximum allowable value of the voucher. The voucher, in other words, has lost some of its purchasing power, and some students have had to supplement the voucher with out-of-pocket expenditures. Since some students are enrolled in private schools only because of the voucher program, the poorest of them have had to drop out of school. In 1993 and 1994, the government issued just under 30,000 new vouchers each year, but in 1995 this level fell to under 18,000, and in 1996 not even 5,000 vouchers were issued. This drop in the number of new vouchers is not a reflection of a large decrease in the demand for secondary education or in the public support of it. A quick inquiry carried out by the Ministry of Education during a World Bank mission in December 1996 demonstrated that in the 30 percent of the participating municipalities that expressed interest and ability to continue their participation in the program in 1997, requests for vouchers amounted to 16,000. The sharp reduction in the number of new vouchers is really due to the decision by the Ministry of Education to limit the number of new vouchers to those funded by the World Bank loan project which provides 5,500 vouchers annually to selected municipalities." 10 This has been computed using actual and projected data from the Ministry of Education Statistics Division. We calculated the mean non-promotion rate for the cohort of students who entered grade six in 1992. The sum of the dropout and grade repetition rates are the relevant numbers because a student who repeats a grade is automatically out of the voucher program. " When the program was designed, the Ministry of Education planned to issue a total of 464,500 new and renewed vouchers between 1991 and 1997, of which 55,000 would be financed through a World Bank loan beginning in 1994 at a rate of 5,500 per year. It was anticipated that in 1997 the program would have a total of 153,500 students enrolled, assuming a program dropout rate of 5 percent. The assumed dropout rate from the program is clearly over-optimistic, given the prevailaing non-promotion rates in secondary education. 21 Figure 2. Number ofSecondary Schools in the Ten Largest Municipalities, By Participation 1,400 1,200 - O Public 1,000 n 8 Private non-voucher 800 0 Private voucher 600 400 200 0 0~ aCa S ource ofpopulation data for municipal rankings: 1985 census; Source of data on the total number ofpublic and private schools: 1993 DANE School Census; source ofdata on voucher participation: ICETEX regional offices, 1994 data. 22 Figure 3. Number ofAnnual New and Renewed Voucher Recipients 80,000 60,000 40,000 New Vouchers - - - Renewed Vouchers 20,000 - / 1992 1993 1994 1995 1996 Source: Ministry of Education Report "Evoluci6n del Programa de Becas PACES 1992-1996", November 1996 IV. DETERMINANTS OF MUNICIPAL AND SCHOOL PARTICIPATION Data Sources This section discusses the factors that have determined the participation of municipalities and schools. The data on schools, municipalities and the voucher program were compiled from various sources besides the administrative data on the voucher program from ICETEX12 * household data from the National Socioeconomic Survey (Encuesta de Caracterizaci6n Socioecon6mica de la Poblaci6n Colombiana or CASEN) was aggregated at the level of the municipality; * municipal finances and poverty estimates came from a municipal database prepared by the Department of National Planning (DNP); 12 A thorough review of the data sources and limitations is presented in King, Orazem, Wohlgemuth (1997). 23 * administrative information on public and private schools, such as enrollment and number of teachers, from the Ministry of Education (MEN); and * school characteristics and achievement test data from the Sistema Nacional de Evaluaci6n de la Calidad de la Educaci6n (SABER), and national high school leaving exam test scores from the Colombian National Testing Service. The statistical results discussed in this section are derived from King, Orazem and Wohlgemuth (1997). In their analysis, program data from ICETEX on the participating municipalities are linked to the municipal finance data and the 1992/3 CASEN household survey data aggregated at the level of the municipality.13 For the analysis at the school level, the ICETEX program data were linked to the SABER survey and test data. The SABER sample of 440 private and public schools was drawn randomly only from a subset of secondary schools -- those offering a complete,secondary cycle, that is, up to grade eleven.14 Data are available on the student's socioeconomic background and achievement test scores in two subject areas, language and mathematics. Additional surveys were administered to a random sample of mathematics and language teachers and school directors, in order to gather information on the characteristics of schools, and the education and experience of school directors and teachers. The linkage among the databases is summarized in Table 5 below. 13 The CASEN sample was chosen randomly, ensuring that our total sample of municipalities is unbiased. It covered 26,680 households in 149 sampling units (which were generally defined as municipalities). 14 Within each school, up to 35 students per grade were randomly chosen, for a total of about 12,000 students in each grade. 24 Table 5. Intersection of the databases used for the analysis Data sources and samples CASEN and DNP data SABER and MEN data (sample = 893 municipalities) (sample = 440 schools) ICETEX municipalities (217 municipalities in 1995) 208 municipalities 111 municipalities ICETEX schools (1,795 schools in 1995) Not applicable 90 schools Municipal Participation Given Colombia's national decentralization program, it is hardly surprising that a voucher program that gives participating municipalities additional central funds equal to 80 percent of the value of a voucher would be attractive. Many local leaders were obviously interested in expanding secondary education for only a fraction of its cost to their budgets, and did join the program. Why then did 80 percent of the municipalities not join the program? Alberto Calderon Z. (1996) enumerates a few pragmatic reasons why some municipalities have shied away from the program: the credibility of the public sector was so low that private schools were not willing to render their tuition revenues dependent on the government bureaucracy; the private sector feared the possible politicization of the program; and the teachers' union (FECODE) has been opposed to any form of privatization of education. The last factor swayed the design of the program; the union's interests were addressed by limiting the use of the voucher such that public schools would not lose any public resources. These are the political factors. Socioeconomic and demographic considerations must have also played a role. These factors are the focus of the analytical results presented here. The municipality's decision to participate would probably depend on the perceived demand for secondary education; the ability of its public schools to meet that need; the ability of its private schools to absorb additional students; and the municipal government's ability to incur the (direct and administrative) expenses of the program. The potential need for private 25 provision at the secondary level could be measured by the ratio of the number of teachers in public secondary schools to the total number of primary students. The larger this ratio, the greater the existing public capacity to absorb the future secondary school population and the smaller the need for the program to induce additional supply. However, the smaller the ratio, the larger the excess demand for secondary school places. This excess demand could, of course, be met either by building more public schools or by using any excess supply in private schools. It is not difficult to imagine that these options are not mutually exclusive and also that a municipal government may opt to build more public capacity only if the excess were large and persistent enough as to make optimal use of the additional schools. This way of thinking about the decision-making by municipal governments would suggest a non-linear relationship between excess demand and participation in a program that uses private schools. The private school capacity to absorb vouchers can be measured as the ratio of the number of private school teachers to the total number of primary students. Familiarity with private schooling would be another factor to consider. Lastly, although household demand for the voucher may exist, the municipality may lack the fiscal resources to act on this demand. The level of per-capita tax revenues in the municipality measures this ability to pay for the program. It must be noted, however, that it also measures the potential fiscal gain to a participating municipality. Municipalities with lower tax revenues stand to gain more by leveraging their scarce local funds with central resources. The net result of these two opposing effects is likely to vary across municipalities. Table 6 presents the means and standard deviations of the variables measuring these factors, by participating and non-participating municipalities. Twenty-two percent of the municipalities in the sample were participating in the program in 1994. A few characteristics would appear to distinguish participants from non-participants. Participating municipalities had primary enrollments that were at least ten times larger than, and secondary enrollments that were more than ten times that, in non-participating municipalities. They had a much 26 higher average pupil-teacher ratio in public secondary schools, thus, denoting space constraints. At the same time, they had an overwhelmingly greater private school capacity -- six times as many private secondary teachers per primary student. Together, these statistics suggest that participating municipalities had both the need for space and the capacity to expand private delivery of secondary school services. Moreover, with regards the ability to co-finance the program, per-capita tax payments in participating municipalities were double those in non-participating municipalities. Table 6. Means and Standard Deviations of Variables Used in the Municipality Participation Models Variable Full sample Non- Participants Participants Participate 0.22 -- -- (.41) Ratio of secondary private teachers to 0.0052 0.0025 0.015 number of primary students in 000s (.012) (.0097) (0.015) Proportion of primary students who 0.052 0.028 0.14 attend private schools (.10) (.07) (.13) Tax return per capita (COOO,000) 2.77 2.32 4.26 (4.77) (3.97) (6.57) Proportion rural of primary schools 0.30 0.31 0.29 (0.34) (0.35) (0.31) Excess demand for school spacesa 12.16 11.27 15.31 (24.84) (23.60) (28.65) Student-teacher ratio in private 15.2 13.1 16.4 secondary schools Student-teacher ratio in public 18.5 17.5 21.9 secondary schools Number of primary students (000) 3,537 1,131 12,107 Number of secondary students (000) 3,022 792 10,963 Sample size 917 716 201 Source: King, Orazem and Wohlgemuth (1997) Notes: LExcess demand = (Primary students-secondary students)/Number of public secondary teachers 27 Whether the final decision to join the program belonged to the municipal government or to the departmental government, there appears to have been a systematic pattern in which municipalities participate in the program (Table 7). Three factors significantly explain municipal participation -- excess demand, private capacity, and familiarity with private provision; poverty rates or literacy levels do not seem to have played a paramount role: (a) The probability of municipal participation rises proportionally with the municipality's excess potential demand for secondary education, as measured by the difference between the gap between the numbers of primary and secondary students divided by the number of public school teachers. Moreover, this effect is highly non-linear. The probability of participation first rises up to the point when excess demand reaches 16 students per public secondary teacher, and then falls at a decreasing rate thereafter. Since the average student-teacher ratio in public secondary schools is 18, the participation probability drops as the excess demand reaches this point at which adding more public classrooms rather than paying for vouchers becomes a reasonable option for local government. Table 7. Determinants of municipal participation Dependent variables Probit Z statistic Elasticity coefficient Excess demand for secondary education (S)` 2.620 3.700 *** 0.351 SU2 -2.802 -2.586 *** s3 0.802 2.079 ** Ratio of secondary private teachers to number 0.356 8.923 *** 0.255 of primary students in 000s Proportion of primary students who attend 4.649 8.584 *** 0.332 private schools Tax return per capita (COOO,000) 0.0573 0.536 0.023 Proportion rural of primary schools -0.211 -1.053 -0.088 Constant -1.50 -15.981 *** Sample size 917 Pseudo R-square 0.27 Log Likelihood -352.022 Notes: -Excess demand = (Primary students-secondary students)/Number of public secondary teachers Source: King, Orazem and Wohlgemuth (1997) 28 (b) Existing private school capacity within the municipality significantly increases municipal participation in the program both for reasons of familiarity with private provision and of capacity to implement the program. The proportion of primary school enrollment in private schools raises the probability of participation with an elasticity of 0.33; that is, a ten- percent rise in private schooling at the primary level will increase participation by 3.3 percent at the mean values of these rates. Likewise, the ratio of number of teachers in private secondary schools to the number of primary students raises program participation with an elasticity of 0.26. School Participation We now turn to the factors behind a school's decision to participate in the voucher program. This analysis is based on data from the 1992-93 SABER survey of schools. Note that the results do not include information on schools established after 1992-93, nor do they pertain to the impact of admitting voucher students once the school adopted the program. That impact analysis will have to wait for new data on schools and students. Table 8 compares the means of participating and non-participating schools in order to examine how participating private schools differed ex ante from non-program private schools and public secondary schools. One would observe the following: Public secondary schools had much larger enrollments than private secondary schools; they had an average of about 900 students per school as compared with 550 students in private schools. Among private schools, the enrollment in participating schools was larger than in the non-participating schools, even prior to joining the program. The average enrollment in program schools was 518 students as compared with 489 students in non-program schools. Participating private schools tended to have a less academic, but more vocational or technical, focus in their course offerings than non-participating private schools. The average monthly tuition fee in participating schools was significantly smaller -- 40 percent less -- than that in non-participating private schools. This was because the non-participating group included schools which were charging very much higher fees. While the median fees in both groups were 29 about equal, the fee for the schools at the 90th percentile of the non-participating group was 73 percent higher than that in the schools at the 90th percentile of the participating group. Table 8. Means and Standard Deviation of Variables Used in School Participation Functions Variable Full Sample Participating Non-Participating Voucher Schools Private Schools Participate 0.48 1.0 0.0 (0.50) -- -- Student enrollment 501.99 517.87 488.56 (319.68) (358.44) (283.10) Monthly fee per pupil 10509.46 7777.09 13161.51 (9048.30) (4643.07) (11140.29) Non-profit school 0.52 0.51 0.53 (0.50) (0.50) (0.50) Academic education 0.84 0.75 0.92 (0.37) (0.44) (0.27) Average assets of pupils' 10.40 9.88 10.90 families (1.57) (1.34) (1.61) School owns building 0.78 0.79 0.77 (0.42) (0.26) (0.43) Sample 148 71 77 Data source: 1992/3 SABER Source: King, Orazem and Wohlgemuth (1997) Table 9 presents the determinants of school participation in the program. They are: (a) By law, program participation by a school depends on whether the municipality in which it is located is itself participating. A question worth asking then is if the demand-driven nature of program participation at the municipal level limited some schools from participating. The possibility of bias emerging from this limitation was addressed by incorporating into the model a selection variable -- the inverse Mills ratio derived from the municipal participation function. The statistically insignificant coefficient suggests that the two-tier program participation did not unduly restrict schools from entering the program. Had the program been open to all schools in all municipalities, the schools that would have participated are not significantly different than those that did participate in the program. This finding is not 30 surprising given that the central government and many departmental governments worked together to bring into the program those municipalities that they deemed likely to benefit from and be able to implement the voucher program. (b) The multivariate results affirm many of the patterns observed in Table 8. Non- profit schools were, at the mean, 11 percent more likely to join the program than for-profit schools, since the objectives of these schools are probably better served by admitting more poor students. Academic schools were 29 percent less likely to participate than non-academic schools (that is, technical or vocational, or art schools) at the mean. And as intended by the designers of the program, schools which serve students with poorer socioeconomic background indeed were indeed more likely to be in the program. (c) The amount of the monthly fee that a private school charges its students reflects various things about the school. It reflects, first of all, the cost of running the school, and thus, to the extent that cost reflects quality, the fee is also a proxy measure for school quality." How fees would affect participation depends on the value of the voucher.'6 If the value limit of the voucher is much lower than the fee, then the school may not find program participation to be feasible. If the value of the voucher greatly exceeds the fee, the school would find participation a profitable endeavor but participation would probably depend on the school's ability to attract voucher students. If the school's quality is perceived to be low, then it may not attract students who could use the voucher in a better school. There is a significantly nonlinear effect of fees on school participation, with participation rising as fees become larger and approach the limit value of the voucher, and then 15 Indeed, tuition fee levels are positively and signicantly correlated with average student test scores in the schools. The correlation coefficients between the proportion of Grade 9 students who scored at the highest level in the SABER tests and test scores are 0.55 for mathematics and 0.52 for language, both of which are significant at the five percent confidence level. 16 In the absence of scholarships, the fee amount will also have the effect of rationing its places only to those who can afford to pay it, and is thus some other measure of the socioeconomic background of its students. But the latter may not be a dominant effect because other socioeconomic variables have been included in the analysis. 31 decreasing thereafter (Table 9). It is not difficult to see why the highest-priced schools for which fees exceed the value of the voucher did not participate. They attract students by offering better quality education and/or a more exclusive peer group for its students, and probably neither want to attract poor students nor need to do so. The lowest-priced schools also did not join the program. Being the most affordable to the poor without the voucher, they may not have anticipated an increase in enrollment through the program and thus did not have much to gain from participation. Moreover, voucher students, if given the choice, probably preferred to enroll in the best private school that the voucher could buy. Table 9. Determinants of School Participation Dependent variables Probit Z statistic Elasticity coefficient Total enrollment (000) 0.492 1.207 0.122 Monthly fee per pupil (COO) 0.296 2.813 *** 0.215 Monthly fee squared (COOOOOO) -0.0139 -3.145 Whether non-profit school 0.429 1.636 * 0.108 Whether offers academic stream 0.648 -1.887 ** -0.292 Average assets of pupils' families -0.260 -2.524 *** -1.333 School owns building 0.173 0.605 0.064 Municipal participation (Inverse-Mills ratio) 0.259 0.766 -0.0007 Sample size 148 Pseudo R-square 0.22 Log-likelihood -79.718 Data source: 1992/3 SABER Notes: * significance at ] percent level; **= sign ficance at 5 percent level; *= significance at 10 percent level. Source: King, Orazem and Wohlgemuth (1997) 32 V. QUALITY IN VOUCHER AND NON-VOUCHER SCHOOLS Finally, we turn now to the question, Do voucher schools differ in quality from non- program private schools or public schools? Due to lack of post-participation data, this question is answered only with respect to the private schools that existed at the initial stage of the voucher program, and not with respect to the impact of the program on quality. One fairly common way of measuring the quality of education offered is by comparing the levels of inputs in the school.'7 By quality, however, we ultimately mean levels of student cognitive achievement. Below, we compare the performance of students in voucher schools to students in other schools on mathematics and Spanish criterion-referenced achievement tests. These tests were administered by the Colombian National Testing Service to students in grades seven and nine. The comparison between students in voucher schools and those in public schools is important because, if private schools generally offer higher quality of education than do public schools, then poor children who have been awarded the vouchers are clear beneficiaries in terms of the quality of their schooling. The test results presented in Table 10 show that there are no statistically significant differences between the average mastery levels of students in participating private schools and students in public schools in the fields of language and mathematics. Thus, concerns about the low-quality education of voucher schools are not borne out by the evidence from schools which existed at the start of the voucher program. However, the table shows also that students in both public and participating private schools perform significantly worse than students in non- participating private schools. 17 One measure of quality improvement is an increase in the level of educational inputs, including an increase in teacher-pupil ratio, improvement in teacher attendance, and change in inputs which are more difficult to measure such as method of instruction. Another way is to monitor improvements in output measures such as improved daily attendance of students and higher achievement scores. 33 Table 10. Average Math and Language Mastery Levels, by Type of School'8 Private Private Non- Public .Participating Participating Mastery Level 3 Math Grade 7 0.652 0.719 0.646 Language Grade 7 0.487 0.610 0.467 Math Grade 9 0.714 0.764 0.716 Language Grade 9 0.749 0.780 0.716 Mastery Level 4 Math Grade 7 0.262 0.298 * 0.258 Language Grade 7 0.232 0.317 ** 0.217 Math Grade 9 0.345 0.389 0.332 Language Grade 9 0.243 0.305 *0.215 * = Difference in means (with respect to participating schools) significant at the 5 percent level ** = difference (with respect to participating schools) significant at the 1 percent level Source: SABER data from the Colombian National Testing Service, 1992-93 Going beyond average mastery levels, Figure 4 shows the relative distributions of the test results for voucher and non-voucher schools. The test results for the 50th percentile (median), the 10th percentile (lowest decile) and the 90th percentile (highest decile) in the voucher schools are marked also in the distributions for public and non-participating private schools. These comparisons show that students in the lowest decile in voucher schools, representing the poorest the poorest performers, are no worse (and may be better) than the students in the lowest decile in public and in non-participating private schools, and that their best students are the equal (or even better than) of the best students in public schools. Specifically, the results are: * Except for the language test results for grade seven, the median results for the voucher schools are nearly identical to those for the public schools, thus, corroborating the evidence 18 Table 10 does not present mean scores. It shows the proportion of students in each type of school who have mastered levels 3 and 4 of the criterion-referenced achievement tests. Criterion-reference tests measure the competency levels reached by a student in specific areas of the curriculum. Levels 3 and 4 correspond to the two highest levels of difficulty in the math and language curriculum for grades seven and nine. Level 3 is defined as the test items which should be have been mastered by at least one-half of the students, and level 4 as the test components which should have been mastered by at least one-quarter of students. 34 from the averages. The median results for the non-participating private schools are clearly higher than for all other schools, especially with respect to the grade seven results. For the grade seven tests, the median results for voucher and public schools are below those of the 30th percentile in the non-participating private schools. For the grade nine tests, the advantage of the non-participating private schools is drastically dampened. * Comparisons of students in the lowest decile in participating and non-participating schools allow us to focus on the students who need the most attention.19 The results indicate that, with respect to the grade nine tests, the students in the lowest decile in voucher schools compare well with those in public and other private schools in the language test, and appear to do no worse than the students in the lowest decile students in other private schools in the math test. With respect to grade seven math, the students in the lowest decile in voucher schools compare evenly with those in other private schools in math and quite favorably with the those in public schools. * Turning now to the top 10-percent of students (i.e., students in the 90th percentile or highest decile) in each group of schools, the results show that voucher schools compare well with respect to public schools in the language tests, and slightly worse in the math tests.20 In other words, the best students in the voucher schools are quite similar to the best students in the public schools. The 10-percent best students in the non-participating 1 Note, however, that these comparisons are made regarding mastery at level 3. We can identify even worse students by using mastery at level 2 to define our deciles. Using this definition, our results indicate that the 10-percent worse students in the voucher schools perform quite comparably with the 10-percent worse students in other schools, public or private, in the grade nine exams. They also do better than students in public schools in both grade seven exams, but not as well as the 10-percent worse students in non-participating private schools. 20 If mastery-at level 4 were used to define the cut-off for the 10-percent best students, then voucher schools rate considerably better than public schools. 35 Figure 4. Mastery Levels in Math and Language Tests by Grade and Type of School Math Grade 7 Language Grade 7 .8 .8 .6- .6 50 26 52 .444 .4 .4 .2 .2 - OO PriPart PriNonpa Public PriPart PriNonpa Public Math Grade 9 Language Grade 9 .8 .8 - 50 41 52 50 1 41 6 .71195 .75 s .6 .6 .4 .4 0 0 .2 0 0 .2 0 PriPart PriNonpa Public PriPart PriNonpa Public Note: The numbers in the Y axis represent proportion of students who performed at level 3 or better. 36 private schools, however, are clearly better students than similarly ranked students in the voucher and public schools. We take a step aside here in order to compare the level of a few school inputs in voucher schools with that in other schools. These inputs are among those generally thought to influence the degree of learning that takes place in the school (Lockheed and Verspoor 1993). The pupil-teacher ratio, which is a popular indicator a school's quality, is the same in the participating schools as in public schools, but is significantly lower in non-participating schools than in all other schools. This is essentially consistent with our conclusions from the test results. The average class size is slightly larger in participating schools, but this difference is not statistically significant. With respect to some school infrastructure, private schools, especially non-participating schools, are better-off than public schools. Table 11. School Inputs as Indicators of Quality Private Private Public Participating Non-Participating Pupil-teacher Ratio 22.0 17.9 ** 21.8 Infrastructure % with auditorium 37 46 30 % with library 94 93 92 % with computer room 39 69 ** 33 Average number of students per classroom 43.2 40.2 ** 41.4 classroom * = Difference between means (relative to participating schools) significant at the 5 percent level. Source: SABER 1992/3 There are other questions about the relative quality of voucher schools that we have not been able to address due to lack of available data. One has to do with changes in school quality after program participation. A force that may lead to improvement is competition for voucher students. On the other hand, voucher schools may not have been prepared for the resulting increase in their enrollments, and thus may have been unable to maintain their quality. Moreover, several schools have opened in response to the program. The quality of these new 37 schools would need to be examined. There are incentives for setting up a new school and setting the fees covered by the voucher at levels that allow for profits to be generated. Since few quality controls and no performance standards have been used to limit the participation of private schools, there may well be more substandard schools in the program than we have been able to discover from the data available to us. This issue deserves empirical examination based on the last five years of program experience. VI. CONCLUSIONS AND FURTHER QUESTIONS Colombia's voucher program shows that the government has been able to effectively mobilize private schools to order to alleviate the space constraints that hinder poor primary school graduates from enrolling in secondary school. At the end of five years, the program has provided subsidies to more than 100,000 students from poor neighborhoods around the country. The per-beneficiary cost of the program was about 77 percent of the unit cost of public secondary education, using very conservative estimates of the cost of public secondary education. There are no currently available data on the burden that individual families have assumed in order for students to participate in the voucher program or on whether the targeting has been carried out successfully. Although an early evaluation found that, at least in Bogota, the program was reaching its intended beneficiaries (Morales-Cobo 1993), this finding would need to be re-examined given the program's expansion. The program has successfully attracted the participation of municipalities that stood most to gain from the program and those that it set out to reach. But there was clearly a decline in participation after the third year of the program. This decline can be attributed to the difficulties in the administration of the program, especially with respect to the delays in the disbursement of funds and the burden of program monitoring. Moreover, the central government has sent mixed signals regarding its own commitment to the program as illustrated by the steady drop since 1994 in the total number of new vouchers it was willing to fund. In 38 fact, at the time of this writing, there are plans to terminate the program in lieu of another system of subsidy. Many questions remain unanswered about the impact of the program. First, has the voucher program truly raised net enrollment rates in secondary education? The answer to this question awaits the availability of enrollment data from a household survey conducted after the program was implemented; such data can then be compared with the enrollment data taken at the beginning of the program. Second, how great has been the response of the private sector in terms of creating new school places? There is no estimate of this supply response. Our study examined only the response of already existing private schools. Third, has the quality of education been affected? The evidence in this report shows that existing private schools that opted to enroll voucher students were of comparable quality to public schools. Whether this is the case with new schools that may have been formed as a response to the program, and whether the existing private schools that joined the program have been able to maintain their quality, need to be addressed. In addition, if the program succeeded in relieving overcrowded public schools and in improving learning in those schools, then the program will have also increased the quality of education for non-voucher recipients enrolled in public schools. These questions could be explored through another SABER survey, particularly if a subset of the schools in the 1992-1993 application are revisited in order to gain a longitudinal perspective. It is harder politically to revive an existing program that has lost support not only from its beneficiaries but also from its administrator than to establish a new one, and Colombia's targeted voucher program is likely to be abandoned after the last of its student beneficiaries has completed secondary education. Whatever the new system of subsidy is going to be, the current program has provided some valuable lessons that should be considered as the next steps are planned. 39 REFERENCES Calderon Z., Alberto. "Voucher Program for Secondary Schools: The Colombian Experience." Processed. 1996. Castafieda, Tarsicio. 1992. Combating Poverty Innovative Social Reforms in Chile During the 1980s. International Center for Economic Growth, San Franciso, CA. Espinola, Viola. 1995. "Seminario Regional sobre Descentralizaci6n Educativa: Organizaci6n y Manejo de las Escuelas a Nivel Local: El Caso de Chile" presented at IDB, World Bank, Dominican Republic's Ministry of Education conference in Santo Domingo, March 1995. Greene, Jay P., Paul E. Peterson, Jiangtao Du with Leesa Boeger and Curtis Frazier. 1996. "The Effectiveness of School Choice in Milwaukee: A Secondary Analysis of Data from the Program's Evaluation," Program in Education Policy and Governance, Occasional Paper 96-3, August 1996, Harvard University. Hommes, Rudolf. 1995. "Conflicts and Dilemmas of Decentralization" presented at the Annual World Bank Conference on Development Economics, 1995. James, Estelle, Elizabeth M. King, and Ace Suryadi. 1996. "Finance, Management, and Costs of Public and Private Schools in Indonesia," Economics ofEducation Review 15(4): 387-398.. Jimenez, Emmanuel and Marlaine Lockheed. 1994. "Private and Public Secondary Schools in Developing Countries: What Are the Differences and Why Do They Persist? " HRO Working Paper #43, Human Resources Development and Operations Policy, the World Bank. King, Elizabeth M. and Rosemary Bellew. 1993. "Educating Women: Lessons from Experience." In King and Hill (eds.) Women's Education in Developing Countries: Barriers, Benefits, and Policy. Baltimore, MD: Johns Hopkins University Press. King, Elizabeth, Peter Orazem, and Darin Wohlgemuth. 1997. "Central Mandates and Local Incentives: Municipal and School Participation in Colombia's Voucher Program." The World Bank, processed Ministry of Education, Government of Colombia. 1996. "Evoluci6n del Programa de Becas PACES 1092-1996" Ministry of Education, Santaf6 de Bogot, processed. 40 Montenegro, Armando. 1995. "An Incomplete Educational Reform: The Case of Colombia," Working Paper HCOWP60, Human Capital Development and Operations Policy, the World Bank. Morales Cobo, Patricia.1993. "Demand Subsidies: A Case Study of the PACES Voucher Program" University of the Andes, Economics Department, processed. National Planning Department, Government of Colombia. 1996. Colombia. Indicadores de Coyuntura Econ6mica: Evoluci6n de la Economia Colombiana en 1996. Vol. 1, #3. Unidad de Anilisis Macroecon6mico, Santaf6 de Bogota. World Bank. 1994. Poverty in Colombia. World Bank Publication Prawda, Juan. 1993. "Education Decentralization in Latin America: Lessons Learned" International Journal ofEducational Development, Vol. 13, No. 3, 1993 Schiefelbein, Ernesto. 1991. "Restructuring Education through Economic Competition: The Case of Chile" Journal ofEducation Administration, Volume 29, No. 4, 1993 West, Edwin. 1996. "Education Vouchers in Practice and Principle: A World Survey". Human Capital Development Working Paper #64, the World Bank. Witte, John F. 1996. "Who Benefits from the Milwaukee Choice Program?" in Who Chooses? Who Looses? Culture, Institutions and the Unequal Effects of School Choice, edited by Bruce Fuller and Richard Elmore with Gary Orfield. New York: Teachers' College Press. 41