贫困人口从工作福利制中获得的收益：对阿根廷Trabajar计划的评估

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ t o s -4 l/4 2' POLICY RESEARCH WORKING PAPER 2149 Income Gains to the Poor A workfare program was introduced in response to from Workfare high unemployment in Argentina An ex-post Estimates for Argentinas Xevaluation using matching methods indicates that the Trabajar Program program generated sizable net income gains to generally Jyotsna jalan poor participants Alartin Ravallon The World Bank Development Research Group Poverty and Human Resources July 1999 0  POLICY RESEARCH WORKING PAPER 2149 Summary findings Jalan and Ravallion use propensity-score matching percent of them are in the poorest quintile - reflecting methods to estimate the net income gains to families of the self-targeting feature of the program design. workers participating in an Argentinian workfare Average gains for men and women are similar, but program. The methods they propose are feasible for gains are higher for younger workers. evaluating safety net interventions in settings in which Women's greater participation would not enhance many other methods are not feasible. The average gain is average income gains, and the distribution of gains about half the gross wage. would worsen. Even allowing for forgone income, the distribution of Greater participation by the young would raise average gains is decidedly pro-poor. More than half the gains but would also worsen the distribution. beneficiaries are in the poorest decile nationally and 80 This paper - a product of Poverty and Human Resources, Development Research Group - is part of a larger effort in the group to improve methods for evaluating the poverty impact of Bank-supported programs. Copies of the paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433. Please contact Patricia Sader, telephone 202-473- 3902, fax 202-522-1153, Internet address psader@worldbank.org. Policy Research Working Papers are also posted on the Web at http ://www.worldbank.org/html/dec/Publications/Workpapers/home.html. The authors may be contacted at jjalan@isid.ac.in or mravallion@worldbank.org. July 1999. (32 pages) The Policy Research Working Paper Series disseminates the finiings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Produced by the Policy Research Dissemination Center  Income Gains to the Poor from Workfare: Estimates for Argentina's Trabajar Program Jyotsna Jalan and Martin Ravallion' Indian Statistical Institute and World Bank 1 The work reported in this paper is one element of the ex-post evaluation of the World Bank's Social Protection II Project in Argentina. The support of the Bank's Research Committee (under RPO 681-39) is gratefully acknowledged. The paper draws on data provided by the SIEMPRO unit of the Ministry of Social Development, Government of Argentina. The authors are especially grateful to Joon Hee Bang and Liliana Danilovich of SIEMPRO for their help with the data. The authors' thanks also go to staff of the Trabajar project office in the Ministry of Labor, Government of Argentina who provided the necessary data on their program and gave this evaluation their full support. Petra Todd kindly advised us on matching methods. Useful comments were received from Polly Jones, Dominique van de Walle, and seminar participants at the World Bank, the Indian Statistical Institute, Delhi, and the Institute of Fiscal Studies, London.  1. Introduction Workfare programs require that participants must work to obtain benefits.2 They are often turned to in crises such as due to macroeconomic or agro-climatic shocks, in which a large number of poor able-bodied people have become unemployed. Typically, the main aim of workfare is to raise the current incomes of poor families hurt by the crisis. To assess the impact of such a program, we need to measure the income gain conditional on income in the absence of the program. The income gain is the difference between household income with the program and that without it. The "with" data can be collected without great difficulty. But the "without" data are fundamentally unobserved, since an individual cannot be both a participant and a non-participant of the same program. Common practice has been to estimate the gains by the gross wages paid.3 In other words, the unobserved income without the program is taken to be equal to income with the program, minus wages received. This assumption would be a reasonable one if labor supply to a workfare program came only from the unemployed. But that is difficult to accept. Even if a participating worker was unemployed at the time she joined the program, that does not mean that she would have remained unemployed had the program not existed. Even a worker who has been unemployed for some time will typically face a positive probability of finding extra work during a period of search, including self-employment in an informal sector activity. Joining the program will leave less time for search. There are also ways in which behavioral responses help reduce foregone income. There are likely to be effects on time allocation within the household. For example, 2 On the arguments and evidence on this class of interventions see Ravallion (1991, 1999a), Besley and Coate (1992), Lipton and Ravallion (1995), Mukherjee (1997), and Subbarao (1997). 3 See, for example, the various assessments of the cost-effectiveness of workfare programs reviewed in Subbarao et al., (1997). 2  Datt and Ravallion (1994) find that other family members took up the displaced productive activities when someone joined a workfare program in rural India. Such behavioral responses will reduce foregone income, though we can still expect it to be positive. This paper estimates the income gains from a workfare program and how those gains vary with pre-intervention incomes. We study the Trabajar Program instituted by the Government of Argentina, and supported by a World Bank loan and technical assistance. We use propensity-score matching methods (Rosenbaum and Rubin, 1983, 1985; Heckman et al., 1997, 1998) to draw a comparison group to workfare participants from a larger survey of non- participants. A number of features of this setting lend themselves to matching methods. It was possible to assure that the same questionnaire was administered to both the participants and the non-participants, and that both groups were from the same economic environment. The Trabajar participant could be identified in the larger survey.4 Furthermore, using kernel density estimation techniques, we are able to ensure that participants are matched with the non- participants over a common region of the matching variables. Any remaining bias in the matching estimator can thus be attributed to unobserved characteristics. The design of the program can be expected to entail considerable rationing of participation according to observables; the sample of non-participants is very likely to include people who wanted to participate but were unable to do so due to say non-availability of the program. While our application is well suited to matching methods, bias due to unobservables cannot be ruled out by matching alone, since the method is basecl solely on observables. So we also propose and implement a test for any remaining selectivity bias after matching. 4 The researcher may not be able to identify whether an individual participated in the program or not in the larger population sample. In such cases, one can still go ahead with the matching procedure though this adds a "contamination bias" to the impact estimator. In our application this is not an issue. 3  The following section discusses the evaluation problem and our methods. Section 3 describes the Trabajar program. Our data are described in Section 4. Section 5 presents the results, and offers an economic interpretation. Section 6 concludes. 2. Estimating the Income Gains from Workfare In assessing the gains from a workfare program, the workers' earnings are naturally the main focus, and that will be the case here. However, it should be noted that earnings net of foregone income are only one of the potential benefits. There could also be risk benefits from knowing that the program exists. There may well also be benefits from the outputs, depending on (amongst other things) how well targeted the workfare projects are to poor areas.5 We first outline what we see as the model of self-targeting underlying arguments for workfare, pointing to the key role played by foregone incomes. We then describe the matching method we use to estimate foregone incomes. 2.1 The Problem The following rudimentary model has the essential features necessary to characterize the "self-targeting" argument often made in favor of workfare (Ravallion, 1991). The model assumes that foregone income from accepting a workfare job is F(Y), a smoothly increasing function of pre-intervention income Y(scaled to lie between zero and one). Foregone income increases with pre-intervention income due to differences in education, experience and so on that are naturally correlated with both earnings and family income. The workfare program offers a wage W, with F(O)<W<F(l). Workers only care about the net wage gain (i.e., the work alternatives are judged to be the same in other respects). S This issue is examnined further in Ravallion (1999b), which presents results on poor-area targeting for the sarne program studied here. 4  It is evident that under these assumptions, only those workers with pre-intervention income less than F-1(W) will participate; the program will perfectly screen "poor" (Y<F-'(W)) from "non-poor" (Y>F-'(W)). The schedule of gains is G=W-F(Y) for Y<F-1() and G=O for Y>F-'(W), yielding post-intervention incomes Y+G. In this simple model, underestimating the foregone income will lead the evaluator to overestimate the impact on poverty. To see why, suppose that, in assessing the gains from the program, we use a biased estimate of foregone income, namely O(A$<F(Y) for all Y. Then we will overestimate the gains for all Yup to 5 -'(W). The distribution of incomes under the biased estimate of foregone incomes must first-order dominate the actual distribution. So the error in assessing foregone incomes will overestimate the impact on income poverty.6 This model also suggests that in the extreme-though commonly assumed-case in which the foregone income is zero, a workfare program would make little sense as a means of reaching the poor. There will be no self-targeting mechanism, and the government would have to rely on some form of indicator targeting or means test. So using the program wage to measure the income gain is antithetical to the logic of a workfare program as a means of self-targeting. How can one estimate the foregone income? This is a counterfactual concept in that participants' incomes in the absence of the program cannot be data. There are several methods one might adopt to assess the counter-factual, drawing on the literature on impact evaluation. One can do reflexive comparisons by collecting baseline data on probable (eligible) participants before the program was instituted. These data are then compared with data on the same individuals once they have actually participated in the program. In this case, the counterfactual group is the set of participating individuals themselves, but observed before the program is 6 This holds for a broad-class of poverty measures (Atkinson, 1987). 5  actually implemented. This method can be extended to include observations on non-participants, before and after the intervention, allowing a "double-difference" estimate of the program's impact. Alternatively, potential participants are identified and data are collected from them. However, only a random sub-sample of these individuals is actually allowed to participate in the program. The identified participants who do not actually participate in the program (the "randomized out" group) form the counterfactual in this case. Another possible approach is to use propensity-score matching methods, following Rosenbaum and Rubin (1983, 1985) and Heckman et al. (1997, 1998). Here, the counterfactual group is constructed by matching program participants to non-participants from a larger survey such as the population census or an annual national budget survey. The matches are chosen on the basis of similarities in observed characteristics. Each of these methods has both strengths and weaknesses. For example, reflexive and double-difference comparisons raise concerns about attrition, whereby a non-random subset of the baseline sample drops out for various reasons. Randomization is ideal in theory, since the comparison group has the same expected distribution of characteristics as the treatment group in the absence of the intervention. However, randomization is not often feasible, and there can also be problems of selective non-participation amongst those randomly chosen for the program. Both baseline survey methods and randomization also require that the evaluation is set up prior to the program. This is unlikely to be feasible in crisis situations. A government concerned about the social impact of a macroeconomic or agro-climatic crisis is not likely to agree to wait for the evaluation to be put in place. Matching methods can avoid these problems, though they create their own. An advantage is that, since most countries now have a nationally representative socio-economic survey instrument, the marginal cost of using matching methods only includes the survey of program 6  participants. The same survey instrument can then be taken to a sample of participants after the program has started, possibly with an extra module to cover specific questions related to the program. Matching estimates will be reliable if: (i) participants and controls have the same distribution of unobserved characteristics; failure of this condition to hold is often referred to as the "selection" problem in econometrics; (ii) they have the same distribution of observed characteristics; (iii) the same questionnaire is administered to both groups; and (iv) participants and controls are from the same economic environment. In the absence of these features, the difference between the mean eamings of the participants in a social program and the matched non-participants will be a biased estimate of the mean impact of the program. 2.2 Method of Estimating the Gains from Workfare Suppose we have data on N participants in a workfare program, and another random sample of size rN (r>l) from the population. The second set of data might be the national population census or an annual national household budget survey that has information relevant in the participation decisions of the individuals. Using the two sets of data, we try to match the N program participants with a comparison group of non-participants from the population. The two surveys must include infonnation that helps predict participation in the program. LetXbe the vector of such variables. Ideally, one would match a participant with a non- participant using the entire dimension of X, i.e., a match is only declared if there are two individuals, one in each of the two samples, for whom the value of Xis identical. This is impractical, however, because the dimensicin of X could be very high. Rosenbaum and Rubin (1983) show that matching can be performed conditioning on P(A) alone rather than onX, where P(X) = Prob(D=I1 X) is the probability of participating conditional onX, the "propensity score" of X. If outcomes without the intervention are independent of participation given Xthen they are 7  also independent of participation given P(X). This is a powerful result, since it reduces a potentially high-dimensional matching problem to a single dimensional problem. The propensity score is calculated for each observation in the participant and the comparison-group samples using standard logit models.7 Choice-based sampling methods suggested by Manski and Lerman (1978) can be used to weight the observations given that there is over-sampling of participants. In our case however, we do not know the sampling weights to do the choice-based sample re-weighting. But we can still carry out the matching using the odds ratio pi = P/( 1 -Pi) where Pi is the estimated probability of participation for individual i. Using the propensity score, one constructs matched-pairs on the basis of how close the scores are across the two samples. The nearest neighbor to the i'th participant is defined as the non-participant that minimizes [p(Xj)-p(X) )]2 over allj in the set of non-participants, where p(Xk) is the predicted odds ratio for observation k. In their comparisons of non-experimental methods of evaluating a training program with a benchmark experimental design, Heckman et. al (1997, 1998) find that failure to compare participants and controls at common values of matching variables is the single most important source of bias - considerably more important than the classic econometric problem of selection bias due to differences in unobervables. To ensure that we are matching only over common values of the propensity scores, we estimated the density of the scores for the non-participants at 100 points over the range of scores. We use a biweight kernel density estimator and the optimal bandwidth value suggested by Silverman (1986). Once we estimate the density for the non- ' One could use semi- and non-parametric methods to estimate the propensity scores though Todd (1995) argues that such methods do not make any difference to the impact estimator. Thus for computational simplicity, we use standard parametric likelihood methods to compute the estimated propensity scores. 8  participants, we exclude those non-participEnts for whom the estimated density is equal to zero. We also exclude 2% of the sample from the top and bottom of the non-participant distribution. The mean impact estimator of the program is given by: P NP II Yj E W9jYV() /P (1) j=p i=l where Yjl is the post-intervention household income of participantj, Yyo is the household income of the ith non-participant matched to thejth participant, P is the total number of participants, NP the total number of non-participants and the Wy's are the weights applied in calculating the average income of the matched non-participants. There are several different types of parametric and non-parametric weights that one can use. In this paper we use three different weights and thereby report three different matching estimators. Our first matching estimator is the "nearest neighbor" estimator where we find the closest non-participant match for each participant and the impact estimator is a simple mean over the income difference between the participant and its matched non-participant.8'9 Our second estimator takes the average income of the closest five matched non-participants and compares this to the participant's income. We also report a kernel- weighted estimator where the weight are given by: P Wij = Kuj 1 K,0 (2) j=l where K= K[(P(X,) - .P(Xj )) / a,0 ] (3) E K[(P(Xi ) -P(Xj ))/aNo ] j=1 s The closest match is chosen by the distance metric discussed above. Also we allow for replacement of the non-participants, so a non-participant could be the closest match for more than one participant. 9  and where aNo is the bandwidth parameter, K(.) is the kernel as a function of the difference in the propensity scores of the participants and the non-participants. In our analysis, we have used Silverman's (1986) optimal bandwidth parameter and a biweight kernel function. (The results were very similar using either a rectangular or parzen kernel function.) Lastly, in each of these cases, the associated standard errors of the mean impact estimator are also calculated. We calculated both the parametric and bootstrapped standard errors for the impact estimators. The two were virtually identical. We report the parametric standard errors in the paper. (The bootstrapped standard errors are available from the authors on request.) 2.3 Testingfor Bias due to Unobservables The matching estimate described above will give a biased estimate of the income gains from workfare if there are unobserved variables that jointly influence incomes and workfare participation, conditional on the observed variables in the data used for matching. A natural test for such a bias is look for a partial correlation between incomes and the residuals from the participation model (used to construct the propensity scores) controlling for actual participation. We call this the test for selection bias in the matching estimator. It is a straightforward application of the standard Sargan-Wu-Hausman test. There will, of course, also be heterogeneity in other characteristics relevant to incomes. By performing the test for selection bias on a sample combining the participants and their matched non-participants we will have already eliminated some of this heterogeneity. One can also explicitly introduce a vector of control variables (Z) to give a test equation for income Y of the form: Y1 = a +