Validation of spot-testing kits to determine iodine content in salt.

Validation of spot-testing kits to determine iodine content in salt Chandrakant S. Pandav,1, 2 Narendra K. Arora,1, 2 Anand Krishnan,1 Rajan Sankar,1 Smita Pandav,1 & Madhu G. Karmarkar1 Iodine deficiency disorders are a major public health problem, and salt iodization is the most widely practised intervention for their elimination. For the intervention to be successful and sustainable, it is vital to monitor the iodine content of salt regularly. Iodometric titration, the traditional method for measuring iodine content, has problems related to accessibility and cost. The newer spot-testing kits are inexpensive, require minimal training, and provide immediate results. Using data from surveys to assess the availability of iodized salt in two states in India, Madhya Pradesh and the National Capital Territory of Delhi, we tested the suitability of such a kit in field situations. Salt samples from Delhi were collected from 30 schools, chosen using the Expanded Programme on Immunization (EPI) cluster sampling technique. A single observer made the measurement for iodine content using the kit. Salt samples from Madhya Pradesh were from 30 rural and 30 urban clusters, identified by using census data and the EPI cluster sampling technique. In each cluster, salt samples were collected from 10 randomly selected households and all retailers. The 15 investigators performing the survey estimated the iodine content of salt samples in the field using the kit. All the samples were brought to the central laboratory in Delhi, where iodine content was estimated using iodometric titration as a reference method. The agreement between the kit and titration values decreased as the number of observers increased. Although sensitivity was not much affected by the increase in the number of observers (93.3% for a single observer and 93.9% for multiple observers), specificity decreased sharply (90.4% for a single observer and 40.4% for multiple observers). Due to the low specificity and resulting high numbers of false-positives for the kit when used by multiple observers (‘‘real-life situations’’), kits were likely to consistently overestimate the availability of iodized salt. This overestimation could result in complacency. Therefore, we conclude that until a valid alternative is available, the titration method should be used for monitoring the iodine content of salt at all levels, from producer to consumer, to ensure effectiveness of the programme. Keywords: sodium chloride, analysis; iodine content; colorimetry, methods; evaluation studies; India. Voir page 979 le re´sume´ en franc¸ais. En la pa´gina 979 figura un resumen en espan˜ol. Introduction Universal salt iodization (USI) is the most widely practised intervention in eliminating iodine deficiency disorders (IDDs) (1). Salt iodine testing is an important ‘‘process’’ indicator formonitoring progress towards USI. Under the National Iodine Deficiency Disorders Control Programme in India, iodization of salt is the recommended strategy, with the level of iodization fixed at a minimum of 15 parts per million (ppm) at the consumer level and 30 ppm at the production level. Most Indian states have introduced mandatory salt iodization through legislation (2). The salt department and the state governments are responsible for monitoring the salt iodine content at both the production and consumption levels (2). Iodometric titration, the traditional method for determining iodine content, is an accurate method (3). But it is time-consuming, and requires capital infrastructure and trained personnel. In India, such laboratories are currently located either at the district or state headquarters. Hence, the length of time between collecting the sample and the availability of results is considerable. Iodine spot-testing kits do not require any infrastructure, are inexpensive, and most impor- tantly, provide immediate results suitable for rapid feedback. The kits give either a qualitative or a semiquantitative estimation of the iodine content. ‘‘Qualitative’’ means that salt samples are classified as (adequately) iodized or uniodized. The test gives no indication of the actual level of iodine in the salt. Some kits give a semiquantitative estimate of iodine content of the salt. For example, one kit categorizes salt samples as having an iodine content of 0, 7, 15, or 30 ppm. In addition to monitoring, iodine spot- testing kits are powerful tools for advocating good health and educating consumers about health. Programme managers in several countries have been quick to adopt the use of kits, and occasionally use 1 International Council for Control of Iodine Deficiency Disorders (ICCIDD) Study Group. 2 Clinical Epidemiology Unit (CEU), All India Institute of Medical Sciences, New Delhi – 110 029, India. Correspondence should be addressed to Dr Pandav, Additional Professor, Centre for Community Medicine, All India Institute of Medical Sciences, New Delhi – 110 029, India. Ref. No. 99-0013 975Bulletin of the World Health Organization, 2000, 78 (8) # World Health Organization 2000 them as the sole tool for monitoring iodine content (4). Previous studies of the validation of iodine spot-test kits for the analysis of salt have been largely laboratory-based or used a single observer (5–7). However, there is a need to test the kits in a field situation where multiple observers are involved in their use. In this study we address the validation and agreement of the iodine spot-testing kit used in actual field conditions in two Indian states, as compared with the iodometric titration method. Methods The data for the present study were collected during two USI surveys carried out by our team in the state of Madhya Pradesh (central India) in 1995 and the National Capital Territory of Delhi (northern India) in 1996. The survey inMadhya Pradeshwas a part of the independent survey evaluation of USI carried out in November 1995 (8). The EPI 30 cluster sampling method based on probability proportional to size of cluster was followed (1). The state was divided into rural and urban strata. In each stratum, 300 house- holds were selected, i.e. 10 in each cluster. The study team comprised 15 senior physicians from different national institutions and medical colleges in India. In each cluster, salt samples were collected from selected households and all the retail shops in that village or urban cluster. The samples were tested in the field by physicians using the salt-testing kit. The survey in the National Capital Territory of Delhi was a school-based survey carried out to estimate the use of iodized salt at the household level (9). Thirty schools from South Delhi were selected using the EPI 30 cluster sampling methodology. After briefing the school principals, teachers, and students about IDD and the importance of consum- ing adequately iodized salt at home, the children from grades 6 to 8 were requested to bring salt samples from their homes on a prearranged day.On that day, a public health nurse from the project visited the school and tested the salt samples using the iodine spot-testing kit. The results were communicated to the students and teachers, and the opportunity was also used to reinforce messages about IDD. In both surveys, the investigators were given training for half a day in using the spot-testing kit and collecting salt samples for transportation to Delhi. About 20 g of salt samples were collected from each selected household in airtight plastic envelopes and brought to the Iodine Monitoring Laboratory of the International Council for Control of Iodine Deficiency Disorders (ICCIDD) located at the Centre for Community Medicine, All India Institute of Medical Sciences, NewDelhi, for iodometric titration. Laboratory procedures The test kit, produced by MBI chemicals, Chennai (India), is a starch-based test. According to the manufacturer’s instructions, the test can be used semiquantitatively to measure iodine in salt at 0, 7, 15, and>30 ppm, depending on the intensity of the colour obtained. If the test showed ‘‘no iodine’’ on the first testing, the testsolutionwasaddedagainafteracidifying the salt sample. This was required to neutralize the presenceofalkali in thesalt. If the test showedno iodine when tested for a second time, thiswas taken as the true test result. The kits were used at least eighteen months before the expiry date marked on the pack. Iodometric titration was the reference method used to estimate the iodine content of salt. This method uses the thiosulfate–starch reaction as an external indicator (10). Salt samples were analysed for their iodine content by carrying out single titrations using 10 g of salt. All the samples were tested within 1 month of collection. Analysis was carried out under the supervision of a senior biochemist (M.G.K.). For internal quality control, one iodized salt sample was tested 20 times for standardization. For every 75 test samples, 1 standard sample was retested. The value for the standard was within the acceptable limits. After the two surveys had been completed, as part of our routine procedure to assess agreement, two laboratory workers were asked to test 100 con- secutive salt samples sent to the laboratory using spot-testing kits identical to those used in the survey. The two workers chosen had been working for about 2 years in our laboratory. Data analysis The data were processed and analysed using Epi Info 6 (11).Tests forproportionand95%confidence intervals were estimated as appropriate. To validate the spot- testing kit, sensitivity and specificity were calculated. The agreement was assessed using the k statistic. The main purpose of the study was to assess the performance of the kit in monitoring the iodine content of salt in situations of multiple observers. The goal of monitoring USI is to estimate the availability of ‘‘adequately’’ iodized salt (515 ppm of iodine). The results were presented accordingly. In addition, results were also divided into two groups and analysed according to the presence (>0 ppm) or absence (0 ppm) of iodine in the salt. In order to compare the performance of the kit with iodometric titration, distribution curves of the iodine content of the salt sample as estimated by titration were plotted separately for categories labelled as 0, 7, 15, and 30 ppm by the kit. Results The results of the study are presented separately for single and multiple observers. Single-observer data from Delhi A total of 1258 salt samples were collected from schools in Delhi and tested both by titration and by Research 976 Bulletin of the World Health Organization, 2000, 78 (8) kit. Bothmethods showed a similar proportion of salt samples with iodine 515 ppm (63.6% by kit and 64.4% by titration) (Table 1). However, most of the samples found by kit to have 0 ppm did have some iodine in them. The sensitivity (ability to correctly identify salts with adequate iodine content) of the kit was 93.3% (95% confidence interval (CI) = 91.3– 94.9) and specificity (ability to correctly identify salts with inadequate iodine content) was 90.4% (95% CI = 87.2–92.9). The positive predictive value was 88.2% (95% CI = 84.8–90.9) while the negative predictive value was 94.6% (95% CI = 92.8–96.0) (Table 2). Multiple-observer data from Madhya Pradesh A total of 682 salt samples was collected from retail shops and households in Madhya Pradesh. The kit overestimated iodine content as compared with titration owing to a high number of false-positives. The proportion of samples classified as having an iodine level >15 ppm was 83.4% using the kit as compared with 69.5% using the titration method (Table 3). As observed in Delhi, most of the samples that tested as 0 ppm using the kit contained some iodine. With multiple observers, the spot-testing kit reporting adequate and inadequate iodine had high sensitivity (93.9%) (95% CI = 91.2–95.8) but poor specificity (40.4%) (95%CI= 33.7–47.4) in detecting adequately iodized salt. The positive predictive value was 78.2% (95% CI = 74.5–81.5%) while the negative predictive value was 74.3% (95% CI = 65.1–81.9) (Table 2). Salt samples were also divided into two groups: iodine present or iodine absent. The results were similar to those described above with specificity and agreement decreasing for multiple observers as compared to a single observer. However, there were very few samples in the not iodized category. We plotted single-observer data by iodine content as assessed by the kit against titration results (Fig. 1). A large overlap was observed for the actual iodine content of the salt sample for the values of 0, 7, and 15 ppm as assessed by the kit. The salt samples assessed at 30 ppm by the kit showed a clear demarcation from the rest. Thus, even with a single observer, the kit failed to discriminate between iodine contents of 0, 7, and 15 ppm. In our laboratory, interobserver variation between two people with 2 years’ experience using the kit was found to be acceptable (k = 0.64). However, the agreement for samples with 7 or 15 ppm was low (k = 0.34), which confirmed the overlap mentioned above. Discussion This study analyses the performance of spot-testing kits in evaluating the status of salt iodization in two states of India using single and multiple observers. There are two limitations of the study which must be kept inmind. First, the data used for analysis were not originally collected for this comparison. This limita- tion does however eliminate any bias due to the Table 1. Comparison of spot-testing kits versus the iodometric titration method for the determination of iodine content in salt: single-observer dataa Spot-testing Iodine concentration Total kit (ppm by iodometric titration of iodine) (ppm) 0 0.1–14.9 515 0 6 287 27 320 (25.4) 7 0 111 27 138 (11.0) 15 and 30 0 43 757 800 (63.6) Total 6 (0.5)b 441 (35.1) 811 (64.4) 1258 a Data from Delhi. b Values in parentheses are percentages. Table 2. Validation of the iodine spot-testing kit as a qualitative method Interpretation of test Indicator (%) Sensitivitya Specificitya PPVb NPVc Iodine present (> 0 ppm) versus iodine absent (0 ppm) Multiple-observer datad 93 (90.0–95.0) 14.3 (0.3–57.8) 99.0 (98.0–99.6) 2.0 (0.1–12.0) Single-observer datae 74.9 (72.4–77.3) 100.0 (54.0–100.0) 100 (99.5–100.0) 2.0 (0.8–4.2) Iodine adequate (5 15 ppm) versus iodine inadequate (< 15 ppm) Multiple-observer datad 93.9 (91.2–95.8) 40.4 (33.7–47.4) 78.2 (74.5–81.5) 74.3 (65.1–81.9) Single-observer datae 93.3 (91.3–94.9) 90.4 (87.2–92.9) 88.2 (84.8–90.9) 94.6 (92.8–96.0) a Values in parentheses are 95% confidence intervals. b PPV = positive predictive value. c NPV = negative predictive value. d Data from Madhya Pradesh. e Data from Delhi. Validation of iodine spot-testing kits for salt analysis 977Bulletin of the World Health Organization, 2000, 78 (8) investigators having prior knowledge of the hypoth- esis. Second, very few samples in the study area contained no iodine. This is to be expected where salt iodization has already been introduced. The study showed that the use of multiple observers resulted in a sharp decline in agreement between the titration method and the kit when iodine content was used either as a qualitative (adequate/ inadequate) or as a semiquantitative tool (four levels: 0, 7, 15, and 30 ppm). The sensitivity of the kit in differentiating adequate and inadequate iodine con- tent of salt was similar for single and multiple observers. However, the number of false-positives was much higher with multiple observers. Multiple observers simulate actual field conditions. Some of the difficulties that arise in using the kit are due to the fact that the kit manufacturers do not precisely make clear what a positive test result of, for example, 15 ppmmeans. Greater than 15 ppm, approximately 15 ppm, 11–22.5 ppm or 7–30 ppm all satisfy the criterion for a positive result. This also applies to the spot-test kits for other concentrations (0, 7, 30 ppm). The variability in the results for the same kit was also shown in a multicentric study conducted in India (7). Although the kit was used by a single observer at each centre, the results among the eight centres were quite variable. The sensitivity varied between 81% and 95.5%, but specificity varied from 50.4% to 100%. In monitoring salt iodization, the objective is to ensure that the community consumes adequately iodized salt. False-positives would lead to over- estimates of iodine content. This could result in complacency on the part of the programme managers. Incorrectly classifying a sample as ‘‘inadequate’’ when it actually has ‘‘adequate’’ iodine (false negative) would result in extra efforts and waste scarce resources. As evident from the current study, the spot-testing kit appeared to have similar sensitivity with both single and multiple observers. Hence, underestimating programme performance is less likely. Because the number of false-positives are indirectly proportional to prevalence, the error diminishes as the availability of adequately iodized salt improves. This phenomenon would result in underestimates of the progressmade to make iodized salt available. In Madhya Pradesh, before the current survey was carried out, programme managers claimed that the availability of adequately iodized salt was 84.21% according to their internal monitoring system. This figure was based on 80 000 salt samples in 42 districts collected and tested by over 4000 health workers. The availability of adequately iodized salt to the community was overestimated by >20%. Because of the high number of false-positives, programme performance was overrated and resulted in compla- cency among programme managers and field staff. Moreover, if inadequately validated tests are used for monitoring a public health programme, the commu- nity as well as the programme managers may lose faith in the effectiveness of intervention. Such an outcome could lead to decreases in the efforts to control IDD and the consequent persistence of IDDs in the community. We have presented results of the performance of one particular kit that is used extensively in India. However, many other kits are available and are being used worldwide (12). Also, the recommended levels of iodine in salt vary from country to country. The performance of any particular kit at relevant levels of iodization needs to be assessed for each country separately. Our study emphasizes the need for a full evaluation of the spot-testing kit using multiple observers before introducing it to monitor pro- grammes at the national level. Iodine spot-testing kits for salt analysis are very useful components of public health programmes. Thus, efforts should bemade to improve the kits that are currently available. Until a valid alternative is available, the titration method should continue to be used for monitoring the iodine content of salt collected from production sites, distribution points (retail shopkeepers), and households. n Table 3. Comparison of spot-testing kits versus the iodometric titration method for the determination of iodine content in salt: multiple-observer dataa Spot-testing Iodine concentration Total kit (ppm by iodometric titration of iodine) (ppm) 0 0.1–14.9 5 15 0 1 38 9 48 (7.1) 7 2 43 20 65 (9.5) 15 and 30 4 120 445 569 (83.4) Total 7 (1.0)b 201 (29.5) 474 (69.5) 682 a Data from Madhya Pradesh. b Values in parentheses are percentages. Research 978 Bulletin of the World Health Organization, 2000, 78 (8) Re´sume´ Validation des ne´cessaires permettant de tester ponctuellement la teneur en iode du sel : expe´riences indiennes Les troubles dus a` une carence en iode constituent un proble`me important de sante´ publique et l’iodation du sel est l’intervention la plus largement employe´e pour les e´liminer. Toutefois, il est indispensable de surveiller re´gulie`rement la teneur en iode du sel pour que cette intervention soit efficace et durable. Le dosage iodome´- trique, me´thode traditionnelle de mesure de la teneur en iode, pre´sente des proble`mes du fait qu’il est peu accessible et couˆteux. Les nouveaux ne´cessaires d’e´preuves utilisables pour des tests ponctuels sont peu couˆteux, exigent une formation minimale et donnent imme´diatement les re´sultats. A l’aide des donne´es des enqueˆtes visant a` e´valuer la disponibilite´ en sel iode´ dans deux Etats indiens, le Madhya Pradesh et le Territoire de Delhi, nous avons teste´ l’applicabilite´ d’un tel ne´cessaire sur le terrain. Des e´chantillons de sel ont e´te´ recueillis dans 30 e´coles de Delhi, choisies en appliquant la technique du sondage en grappes du Programme e´largi de Vaccination. Un seul observateur s’est charge´ de mesurer la teneur en iode au moyen du ne´cessaire. Les e´chantillons de sel recueillis dans le Madhya Pradesh l’ont e´te´ dans 30 grappes rurales et 30 grappes urbaines identifie´es par la meˆme technique, en se servant des donne´es du recensement. Dans chaque grappe, les e´chantillons de sel ont e´te´ recueillis dans 10 foyers choisis au hasard et chez tous les de´taillants. Les 15 enqueˆteurs ayant proce´de´ a` cette enqueˆte ont fait une estimation de la teneur en iode des e´chantillons de sel sur le terrain au moyen du ne´cessaire d’e´preuve. Tous les e´chantillons ont e´te´ ramene´s au laboratoire central de Delhi, ou` la teneur en iode a e´te´ estime´e au moyen du dosage iodome´trique, qui a servi de re´fe´rence. La concordance entre les valeurs obtenues avec le ne´cessaire d’e´preuve et le dosage iodome´trique dimi- nuait au fur et a` mesure que le nombre d’observateurs augmentait. Si la sensibilite´ n’a pas e´te´ tre`s touche´e par l’augmentation du nombre d’observateurs (93,3 % pour un seul observateur et 93,9 % pour de nombreux observateurs), la spe´cificite´ a chute´ brutalement (90,4 % pour un seul observateur et 40,4 % pour de nombreux observateurs). Etant donne´ la faible spe´cificite´ et le grand nombre de faux positifs qui en est re´sulte´ lorsque le ne´cessaire d’e´preuve a e´te´ employe´ par de nombreux observateurs (« situation re´elle »), il est probable que ces derniers surestiment re´gulie`rement la disponibilite´ en sel iode´. Cette surestimation pourrait entraıˆner un relaˆ- chement des efforts. Par conse´quent, jusqu’a` ce qu’on dispose d’une autre me´thode fiable, il faut utiliser le dosage iodome´trique pour surveiller la teneur en iode du sel a` tous les e´chelons, depuis le producteur jusqu’au consommateur, si l’on veut s’assurer de l’efficacite´ du programme. Resumen Validacio´n de kits de determinacio´n in situ del contenido de yodo de la sal: experiencia en la India Los trastornos por carencia de yodo son un grave problema de salud pu´blica, y la yodacio´n de la sal es la intervencio´n ma´s empleada para eliminarlos. Sin embargo, es fundamental vigilar regularmente el contenido de yodo de la sal para asegurar el e´xito y la sostenibilidad de esa intervencio´n. El me´todo empleado tradicionalmente para medir el contenido de yodo, la valoracio´n yodome´trica, plantea problemas relacionados con su accesibilidad y costo. Los kits ma´s recientes para ana´lisis in situ son baratos, apenas exigen adiestra- miento, y permiten obtener resultados inmediatos. Partiendo de los datos aportados por estudios de evaluacio´n de la disponibilidad de sal yodada en dos Estados de la India, Madhya Pradesh y el Territorio de Delhi, procedimos a analizar la utilidad del kit sobre el terreno. En Delhi se obtuvieron muestras de sal de 30 escuelas, empleando para ello la te´cnica de muestreo por conglomerados del Programa Ampliado de Inmuni- zacio´n (EPI). Las mediciones del contenido de yodo mediante el kit fueron efectuadas por un solo observador. Las muestras de sal de Madhya Pradesh procedı´an de 30 conglomerados rurales y 30 urbanos, identificados mediante la te´cnica de muestreo por conglomerados del EPI a partir de datos censales. En cada grupo, se obtuvieron muestras de sal a partir de 10 hogares seleccionados al azar y de todos los minoristas. Los 15 investigadores participantes en el estudio calcularon el contenido de yodo de las muestras de sal empleando el kit sobre el terreno. Todas las muestras se llevaron al laboratorio central de Delhi, donde el contenido de yodo se determino´ mediante la valoracio´n yodome´trica como me´todo de referencia. El grado de coincidencia entre los niveles obtenidos con el kit y los obtenidos mediante la valoracio´n disminuyo´ paralelamente al aumento del nu´mero de observadores. Aunque la sensibilidad no se vio demasiado afectada por el aumento del nu´mero de observadores (93,3% para un solo observador, y 93,9% para varios observadores), la especificidad disminuyo´ pronunciadamente (90,4% para un solo observador, y 40,4% para varios observadores). Considerando la baja especificidad y el nu´mero consiguientemente elevado de falsos positivos obtenidos cuando lo usaron varios observadores («condiciones reales»), cabe concluir que el kit tendı´a a sobreestimar sistema´ticamente la disponibilidad de sal yodada. Puesto que esa sobreestimacio´n podrı´a conducir a la compla- cencia, mientras no se disponga de una alternativa va´lida deberı´a seguirse utilizando el me´todo de valoracio´n para controlar el contenido de yodo de la sal en todos los niveles, desde el productor hasta el consumidor, a fin de garantizar la eficacia del programa. Validation of iodine spot-testing kits for salt analysis 979Bulletin of the World Health Organization, 2000, 78 (8) References 1. WHO/UNICEF/ICCIDD. Indicators for assessing iodine deficiency disorders and their control through salt iodization. Geneva, World Health Organization, 1994: 29–31 (unpublished document WHO/NUT/94.6; available upon request from Nutrition for Health and Development, World Health Organization, 1211 Geneva 27, Switzerland). 2. Salt Department, Ministry of Industry. Universal Salt Iodisation (USI) – India: progress and current status. New Delhi, 1996: 8. 3. Sullivan KM et al. Monitoring universal salt iodisation programmes. UNICEF/PAMM/MI/ICCIDD/WHO. Atlanta, GA, Program Against Micronutrient Malnutrition/Micronutrient Initiative/International Council for Control of Iodine Deficiency, 1995: 11. 4. Salt Department, Ministry of Industry/UNICEF. Universalization of access to iodised salt: a mid-decade goal. New Delhi, 1994: 1–17. 5. Kapil U, Nayar G, Goindi G. Utility of spot testing kit in the quantitative estimation of iodine content in salt. Indian Pediatrics, 1994, 31: 1433–1435. 6. Bhasin SK, Dubey KK. Validity of spot testing kit for estimation of iodine content of salt under field conditions in National Capital Territory of Delhi. Indian Journal of Maternal and Child Health, 1997, 8(2): 42–43. 7. Department of Human Nutrition. Utility of spot testing kit in the assessment of iodine content of salt: a multicentric study. New Delhi, All India Institute of Medical Sciences, 1999. 8. International Council for Control of Iodine Deficiency Disorders (ICCIDD). Independent survey evaluation of Universal Salt Iodisation (USI) in Madhya Pradesh. New Delhi, 1996. 9. Pandav CS et al. Using government schools to monitor iodine content of salt at household level in Delhi. Indian Journal of Pediatrics, 1999, 66: 179–183. 10. Karmarkar MG, Pandav CS, Krishnamachari KAVR. Principle and procedure for iodine estimation: a laboratory manual. New Delhi, Indian Council of Medical Research, 1986. 11. Dean AG et al. Epi Info, Version 6: a word processing, database, and statistics program for epidemiology on microcomputers. Atlanta, GA, Centers for Disease Control and Prevention, 1994. 12. Monitoring universal salt iodisation programmes. UNICEF/PAMM/ MI/ICCIDD/WHO. Atlanta, GA, Program Against Micronutrient Malnutrition/Micronutrient Initiative/International Council for Control of Iodine Deficiency, 1995: 81. Research 980 Bulletin of the World Health Organization, 2000, 78 (8)