Sustainability monitoring : the VIP way - a ground-level exercise

23740 1999 World Bank Sus-inability Monitoring: Water and Sanitation The VIP Way Program - South Asia A Ground-level Exercise (In collaboration with the World Bank) "With schemes commissioned in 30 batch-I villages in the I Karnataka (India) Rural Water Supply and Environmental Sanitation Project (Cr 2483-IN), A _ -' the November 1998 Bank supervision mission made a \. 4 'i serious attempt at assessing _ : results on the ground in terms of benefits to the users, village-level A capacity building and prospects of long-term sustainability. In the process, the task team successfully initiated the process of mainstreaming the Village Immersion Program (VIP) in Bank operations in India. This field note describes the - methodology, process and findings of the joint government- Bank exercise in measuring sustainability of rural water and sanitation investments." Proeect Components The lProblem v Drinking water supply and groundwater recharge For most projects, the end of * Environmental sanitation disbursements is just the beginning of (droinage and latrines) scheme operations. How do we, * Developing replicoble INDIA however, assess the prospects of long- implementation model supportive INDI ,> term sustainability of rural water of a demand-responsive approoch supply and sanitation service delivery? * Hygiene and sanitation How do we know that the physical awareness programs - and social infrastructure created in rural communities will continue to function Project Size effectively and eflficiently over the entire design Number of villages: 1,200 period? Such questions usually remain Bank assistance (IDA): US $92 million unanswered since no well-tried and tested tool Period: 1993-1999 exists for measuring sustainability of rural water and sanitation services.  Breaking New Ground Objectives In Karnataka The main objectives of the joint government-Bank exercise were: In an attempt to address these * to develop and field test a methodology for measuring difficult questions, the Bank task team, sustainability at the village level; with full participation of the project . to obtain a better appreciation of the ground realities through a staff, consultants, NGOs and village 24-hour long Village Immersion Program; * to bring together different project stakeholders in a grassroots- communities, undertook the level team building exercise; and sustainability measuring exercise in 15 * to identify key issues and problems affecting sustainability. project villages in nine districts. The visiting teams stayed 24-hour cycles in Conceptual Framework the villages and assessed system performance in partnership with the The exercise was based on the broad conceptual framework user communities. The event was provided by the PLA (participatory learning assessment) guide highly successful and a useful learning developed by the UNDP-World Bank Water and Sanitation Program. The framework was modified to suit project-specific needs. It was also decided to focus on measurement of sustainability indicators at The user communities strongly the community level, leaving out factors external to the communities supported the exercise and keenly (such as implementing agency effectiveness and policy environment). participated in an inward-looking self The four key parameters (see box below) were further divided into 71 discrete indicators for recording in the field and subsequent analysis. Each indicator was assigned a score on a scale of 0 to 2 (in some building, the output is a tested cases, the scale range is wider). sustainability monitoring tool for use The 71 indicators capture the key aspects of sustainability as by the project staff for schemes follows: Infrastructure functioning and use: Quantity, quality, that are progressively being put reliability, equitable access and user perceptions into operation. Institutional: Effectiveness of village-level institutions in service delivery and management Financial: Adequacy of cost recovery for scheme operations and sustenance Social: Participation in benefits and management by women and socially disadvantaged groups Conceptual Framework Matrix EFFECTIVE DEMAND PARTICIPATION PARTICIPATION FUNCTIONING & RESPONSIVENESS IN SERVICE IN PLANNING, USE OF SERVICES OF SERVICE MANAGEMENT (O&M) ORGANIZATION AND CONSTRUCTION INDICATORS INDICATORS INDICATORS INDICATORS * Functioning System * Is demand of users being Economic participation Participation in initiating (25%) met (30%) (25%) project (5%) * Effective Use (25%) * Value aotached to service Management Extent of informed choice * Effective Management by users (40%) participation (25%) (20%) (25%) Perception of cost Participation in benefits Contribution in * Effective Financing effectiveness (30%) (25%) construction (20%) (25%) Perception of ownership Community organization (15%) for managing services, Perception of formation, composition, responsibility (10%) roles, legal status, bank account status (15%) * Accountability of implementers to community (20%) * Whose capacity built for what, when, why and how (20%) Source- PLA Methndalrnnv G,iirdr. IJNDP-World Bank Wnt,zr and Sanitation Proorom  The Process: The village visit M ethodology generated great enthusiasm amongst the village community as well as the team members. Unused to the idea of project Pro-field: A total of 15 villages were visited; an average of three staff staying for more than a few hours, the villages by each of the five teams set up for the exercise. Every team communities were touched at this display stayed a full 24-hour cycle in each village and consisted of project of sincerity in attempting to understand staff, partner consultants and NGOs. One Bank mission member them better, and delighted to host the participated in each team and mainly acted as a facilitator. A one- teams for a day and a night. In the day preparatory workshop was held for the teams to brainstorm process, a great deal of information and about the concept and the methodology as well as to thoroughly knowledge was shored by both sides. In understand (and modify wherever necessary) the questionnaires most meetings, there was good participation from women and girls. Debates went on late into the evenings, and the degree of excitement and V_ community involvement was so high that -71 -, the event assumed a carnival-like i atmosphere. Many villages also arranged late night Pi' cultural programs and, in one case, a ,T. volleyball match also took place between I visitors and the community. Needless to say, the community won! Such informal i i - events helped in b-eaking the ice and improving communication between .outsiders" and the "insiders". Apart from this, the village immersion and the night Wrap-up meeting with VWSC stay also served as a good bonding exercise amongst the team members specially designed for the field visit. A set of nine questionnaires themselves, most of whom did not know were used for this purpose (see box). A variety of survey instruments each other well prior to this VIP (field observations, general meetings, focus group discussions, household surveys and public tapstand monitoring) were used for the exercise. The data from the filled in questionnaires was to be Karnataka Rural Water analyzed and converted into points, on the basis of which an overall Supply and Sanitation Project sustainability index would be derived for each village. Sustainability Monitoring Field: In the field, the teams first had a general meeting with the Questionnaire Village Water and Sanitation Committees (VWSCs), where they finalized the 24-hour schedule of activities and formed task teams, Questlemmuire Titl- each with community representatives. The different task teams Questionnaire 1 Village Socio-economic obtained information simultaneously from their respective target Profile audiences such as women's groups and households. The technical Questionnaire 2 Technical: Water Supply task team systematically visited and studied each of the water and (Asset Condition and Profile) sanitation system installations (tubewells, pumphouses, overhead Questionnaire 3 Technical: Sanitation tanks, water supply pipes and taps, drains and household latrines). (Droinage, Soak-Pits The event in which there was maximum community participation, And Dust-Bins) however, was the tapstand monitoring exercise. The exercise, which Questionnaire 4 Technical: Sanitation was preceded by a training session, involved a team of about 20 (Household Latrines) Questionnaire 5 Financial (Costs, Tariff, persons, including community members, each setting out at the same Billing and Collections) time during water supply hours to monitor quantity, quality, reliability Questionnaire 6 Institutional (VWSC and equitable access, by observation as well as by talking to the Composition, Functions, villagers collecting water from the tapstands. The environmental Effectiveness) conditions around the topstands, the number of men, women and Questionnaire 7 Household Questionnaire c f (Fads and Perception of children fetching water and the number of pots of water collected Demand Met) were also studied. After all the sub-teams had completed their tasks Questionnaire 8 Social (Participation by and surveys, a final 'wrap-up' meeting was held with the VWSCs. Women and Poor) Here, each of the 71 parameters were awarded a score (on the Questionnaire 9 Tapstand Monitoring predetermined scale for each) by a voice vote from all those present, both outsiders and VWSC members.  Analysis Issues Affecting Sustainability This intensive village-based sustainability monitoring exercise Post-field: On return from the field, led to identification, by both the communities as well as by all the data colleded through the field outsiders, of a range of issues which could adversely affect the questionnaires and scores of the 71 sustainability of water and sanitation services. The issues indicators were converted into a common to most villages were: sustainability index for each village. The Teshnical: Unstable electrical supply affecting pump analysis revealed that nine out of the 15 operations and reliability of water service villages visited fell into the 'likely to be Institutional: VWSCs yet to take full charge of scheme sustainable' category (score of above management and the need to clarify legal provisions 0.65), five into the "uncertain" category empowering VWSCs vis-a-vis gram panchayats (between 0.50 and 0.64) and one in the Finandal: Full cost of O&M not yet being recovered "unlikely" category (below 0.50). from users Soial: Women and socially disadvantaged groups yet to be fully empowered Uncertain 33x 1 \ Looking Forward Institutionalizing 'wrap-up' with VWSC: The business-like 60% final wrap-up meeting with the VWSC, stipulating actions to be taken and by whom, increased their sense of responsibility and determination to overcome the shortcomings discovered during the exercise. The task team felt that this should become a standard / feature of supervision by the project staff. Repeat exercise: The sustainability monitoring exercise needs to be repeated in the same village to confirm that the issues identified in the earlier exercise have been satisfactorily resolved. Scaring system: The range of values assigned to the 71 primary indicators and relative weightage given while aggregating scores for each of the four main parameters need to be revisited and, if necessary, modified to better correlate them to ground realities. Name Overal Assessment ft Measuring water quantity: Quantifying the water received at the user end was found the most difficult to estimate. The Sridharagadda 0.7 L methodology used (number of pots, size of storage vessels and Holalu 0.67 L tanks) for estimating the amount of water received by private Kinnala 0.66 L B. Hatna 0.67 L connections and public tapstands was not very accurate and DMG Halli 0.65 L needs to be improved. Udigala 0.65 L Mainstreaming sustainability monitoring: Inspired by this Gabbur 0.65 L pilot venture, the Karnataka project staff have now started Belekera 0.71 L independently undertaking the exercise as a part of regular Markcundo 0.65 L Sovena Holli 0.64 UNC implementation supervision. The additional spin-off is that this Shantigrama 0.62 UNC exercise is also being undertaken in the other Bank-assisted project Garl Gunji 0.55 UNC in India (Uttar Pradesh Rural Water Supply and Sanitation Project). Bailur 0.62 UNC Bank operations and VIP. The exercise demonstrated the Kambalu 0.62 UNC benefits of using VIP as means of achieving greater development impact. It could also be a useful tool in seeking participation of the L: Likely UNC: Uncertain UNL: Unlikely communities in designing projects which more accurately reflect their needs and aspirations. For further information, please contact O.V Ahyankar, World Bank, 70, Lodi Estate, New Delhi 110 003 (gabhyankar@aworldbank.org) or Parameswaran lyer, Water bnd Sanitation Program - South Asia, 55, Lodi Estate, New Delhi 110 003 (piyer@?worldbank.org).