Implementing predictive models for domestic decision-making against dengue haemorrhagic fever epidemics Halmar Halide# Physics Department, FMIPA, Hasanuddin University, Makassar 90245, Indonesia
Abstract The efficacy of two simple models for predicting dengue haemorrhagic fever (DHF) epidemics are evaluated. One model uses persistence while the other uses past dengue cases and climate factors to make predictions. It is shown that the efficacy of the models is not significantly different. The value of the prediction is also investigated when it is used to decide whether it can protect a household from epidemics. When the model predicts that a DHF epidemic is forthcoming, a highly effective but lowcost DEET product is applied by the whole family as protection against mosquito bites. It is found that the cost of implementing such a model for prediction is much cheaper than other options such as: (i) using protection without any forecast; and (ii) neglecting any protection. It is also found that the value of a forecast depends on the forecast skill and the cost-to-loss ratio. Keywords: DHF epidemics; predictive model; forecast value; decision-making; DEET.
Introduction Dengue haemorrhagic fever (DHF) causes a substantial burden to a family in terms of loss of life and economic impact.[1,2,3,4,5] The number of people suffering from the illness is also predicted to increase in the years ahead due to global warming.[6,7,8] Therefore, an early warning system (EWS), even with a one-month lead prediction for an upcoming dengue haemorrhagic fever (DHF) epidemic,[9] is urgently needed.[10,11,12] Such a system can be used to make an informed decision to
prevent the occurrence of an epidemic at a family scale. There are a few models that could serve as an EWS. The models range in complexities and use biotic and abiotic factors to make dengue predictions. More recently, a simple statistical model, HR2008, has been able to give a useful epidemic prediction up to six months in advance.[9] In this study, the HR2008 model and a persistence model are implemented in a
#
E-mail: halmar@unhas.ac.id 1
Dengue Bulletin – Volume 33, 2009
Implementing predictive models for domestic decision-making against DHF epidemics
decision-making problem as an attempt to prevent an epidemic in the city of Makassar, Indonesia (5.1°S, 119.6°E). The decision of whether or not a family applies a protective measure is made based on the model’s prediction. The value of a forecast is assessed through expenses resulting from several decision options.
Methods Data The monthly number of confirmed DHF cases was recorded by the Public Health Division at the city of Makassar, Indonesia. Predictive models were developed using these cases. Length of stay (LoS) and cost to patients were obtained at a regional hospital, RS Wahidin Sudirohusodo, at Makassar during DHF epidemics, i.e. the months of January and April. The focus was on patients who occupied rooms with the least expensive rates. Other demographic data such as household size was obtained from the Makassar Bureau of Statistics.
(2) a DHF predictive model HR2008 developed earlier. [9] This model uses both past DHF cases and local meteorological variables such as relative humidity h and average temperature Tave to predict cases in the following month. The model was run on DHF data from the period January 1999 to December 2005 and gives the following closed-form formula for predicting the number of cases a month in advance: N(t+1)=0.73N(t)–3.44h(t-4)16.43Tave(t-5) +732.45 [2]
Note that the HR2008 model is capable of producing a useful prediction skill up to six months in advance against a no-skill random forecast.[9]
Prediction skill assessment In order to assess the prediction skill of these two models, we use predictions covering the period from February 1999 to December 2005, i.e. 83 months. The skill of each model is determined by its Peirce score using a contingency table as in Table 1. In this table a, b, c and d refer respectively to the number of times the epidemic is forecast and also observed, the epidemic is forecast but did not occur, the epidemic is not forecast but did occur, and the epidemic is neither forecast nor observed. The score and its error estimate are calculated using data from Table 1 and the following formulas below.[14] Peirce skill score PSS = (ad-bc)/(a+c)(b+d) [3]
Model and predictions The two models used to give a one-month lead prediction of DHF epidemics are briefly described. An epidemic is defined when the number of DHF cases exceeds the 75th percentiles.[13] The models are: (1) a persistence model which states that the number of DHF cases in the following month is the same as that of the present month, i.e. N(t+1)=N(t) [1]
Standard error ePSS = [(n2-4(a+c)(b+d)PSS2)/4n(a+c)(b+d)]1/2 [4] where the total number of predictions and observations n = a+b+c+d.
where N(t) is the number of cases at time t (measured in months).
2
Dengue Bulletin – Volume 33, 2009
Implementing predictive models for domestic decision-making against DHF epidemics
Table 1: Contingency table for the Yes/No of DHF epidemic forecast[9] DHF epidemic predicted Yes No DHF epidemic observed Yes a (hit) c (miss) No b (false alarm) d (correct rejection)
Forecast value evaluation The value of a decision is examined in terms of cost C and loss L. The former occurs when the family uses a daily protection method and the loss is incurred when the unprotected family suffers from an epidemic. Note that one could also perform a cost-benefit analysis, i.e. a benefit is the savings resulting from taking a protection. Beside a forecast-led decision, there are also other options to consider. They are: the family applies a daily protection regardless of any forecast and the family does not use any protection at all. The expense E for each decision is calculated using Thorne and Stephenson (2002) formulation.[20] E1 (for not using any protection) = (a+c) × L [9]
The prediction skill of a model is usually compared against a random no-skill forecast by first transforming the above a, b, c, d values as: ar=(a+c)(a+b)/n br=(b+d)(a+b)/n cr=(a+c)(c+d)/n dr=(b+d)(c+d)/n [5] [6] [7] [8],
E2 (for a daily protection regardless of any forecast) = (a+b+c+d) × C [10] E3 (for using a predictive model) = ((a+b) × C) + (c × L) E4 (for using perfect forecast) = (a+c) × C [11] [12]
and then the transformed values (5–8) are substituted into (3) and (4) to obtain score and error for the random forecast.
Decision-making problem A household based its decision whether or not to take any protective measures depending on a model forecast. The family will only take protective measures against an epidemic when a model predicts that the event is forthcoming. In this case, the family member applies a highly effective but low-cost DEET product daily for personal protection.[15] Note that this mode of protection is selected from among other forms of domestic interventions[16,17,18,19] because it directly protects a person both in and outside the house from mosquito bites. The economic value of using such a model forecast for taking a decision is examined below.
The value of a forecast is presented as a value index and calculated using the above expenses as: VI = (E2-E3)/(E2-E4) [13]
Results Models skill Observed DHF cases (circled) and out-ofsample predictions (lined) of cases for both predictive “HR2008” and “persistence” models are presented in Figure 1. We also
Dengue Bulletin – Volume 33, 2009
3
Implementing predictive models for domestic decision-making against DHF epidemics
plot a horizontal dotted-line at dengue cases equalling to 134 at 75% percentiles to assign epidemic events. Figure 1 shows that the HR2008 model correctly predicts the moderately severe epidemic peaks from 2001 to 2005. These epidemics, however, are predicted to occur one month later by the persistence model as expected. It was also found that the HR2008 wrongly predicted higher cases in 1999 and 2000 and a few negative cases in 1999. None of the latter problems are found in the persistence model. The contingency parameters and forecast skills for both models are presented in Table 2 and Figure 2. The one-month delay in predicting these epidemics seems to lower the number of hit rates a, and the correct rejections d obtained by the persistence model compared with that of the HR2008
model. The Peirce skill score, however, is not significantly different from each model. Both models have a much higher skill than that of the random forecast. Table 2: Prediction skill of the HR2008 and persistence models and their corresponding no-skill forecasts (in parenthesis) Parameters a b c d Peirce skill score Model HR2008 18 (7) 5 (6) 7 (8) 53 (42) 0.63±0.10 (0.0±0.12) Persistence 16 (6) 7 (17) 6 (16) 54 (44) 0.61±0.10 (-0.01±0.12)
Figure 1: Data (observed DHF cases) and the out-of-sample predictions of DHF cases at one month in advance for the HR2008 and persistence models (The horizontal dotted line represents the 75% percentiles of DHF cases)
4
Dengue Bulletin – Volume 33, 2009
Implementing predictive models for domestic decision-making against DHF epidemics
Figure 2: Peirce skill scores including the error estimates (error bar) for both predictive models HR2008 (circle) and persistence model (upper triangle) and their associated no-skill random models in crosses (×), respectively
Models’ forecast value Cost of protection The household size in Makassar ranges from 3.16 to 5.26 persons, with an average of 4.26 in a total population of about 1 223 540.[21] The minimum monthly regional wage in 2006 was US$55.64[21] (US$ 1=11 000 Indonesian Rupiahs). Let us suppose a family of four is to be protected against an epidemic. The mode of protection uses an insect-repellent called AUTAN. This product comes in a lotion which contains 12.5% DEET. It is packed in a sachet weighing 10 g. Each person applies the product twice a day, i.e. two sachets, for 12-hour protection during daytime according to an efficacy test.[22] One sachet of AUTAN costs 4.5 cents. The total cost of protecting a family of four for 30 days, therefore, equals US$ 10.9.
Loss due to DHF epidemics If a member of the family is not protected against dengue-carrying mosquito bites, he/ she has the risk of getting hospitalized due to DHF. The length of stay (LoS) (in terms of nights) of a DHF patient during the 2008 epidemic in Wahidin Sudirohusodo Hospital ranges from one to eight days, with an average of 4.8 days. The economic loss for each night spent in the least expensive room is presented in Table 3. The cost includes: blood examination, treatment, meals, visits by physicians and nurses, and room rent. The cost-to-loss ratio (C/L), expenses and the value index of the two predictive models are also presented in Table 3 and Figure 3. In Table 3, the expense resulting from implementing a forecast E3 is cheaper than that of the no-protection E1 and protection
Dengue Bulletin – Volume 33, 2009
5
Implementing predictive models for domestic decision-making against DHF epidemics
Table 3: Forecast value of the HR2008 and persistence models (expenses and value index for their corresponding no-skill forecasts in brackets. The cost C for protecting a family of four people is US$ 10.9. E2 and E4 are the same for all nights. Note that the figures in squared-brackets are the number of patients with corresponding LoS) Length of stay in hospital LoS (nights) Parameters Model 1 [3] 15.0 0.73 375.0 (375.0) 2 [13] 23.2 0.47 579.5 (579.5) 3 [8] 31.4 0.35 784.1 (784.1) 4 [9] 39.5 0.28 988.6 (988.6) 5 [3] 47.7 0.23 1193.2 (1193.2) 6 [0] 55.9 0.20 1397.7 (1397.7) 7 [2] 64.1 0.17 1602.3 (1602.3) 8 [1] 72.3 0.15 1806.8 (1806.8)
Loss (L) (US$) C/L
HR2008
E1 (US$) E2 (US$) E3 (US$) E4 (US$) VI
905.5 (905.5) 355.9 (520.9) 413.2 (668.2) 470.4 (815.4) 527.7 (962.7) 585.0 (1110.0) 642.3 (1257.3) 699.5 (1404.5) 756.8 (1551.8)
272.7 (272.7) 0.87 (0.61) 330.0 (330.0) 0.78 (0.38) 510.0 (510.0) 0.69 (0.14) 690.0 (690.0) 0.60 (-0.09) 870.0 (870.0) 0.51 (-0.32) 1050.0 (1050.0) 0.42 (-0.56) 1230.0 (1230.0) 0.32 (-0.79) 1410.0 (1410.0) 0.23 (-1.02) 1590.0 (1590.0)
Persistence
E1 (US$) E2 (US$) E3 (US$) E4 (US$) VI
905.5 (905.5) 340.9 (490.9) 390.0 (621.8) 439.1 (752.7) 488.2 (883.6) 537.3 (1014.5) 586.4 (1145.5) 635.5 (1276.4) 684.5 (1407.3)
240.0 (240.0) 0.85 (0.62) 0.77 (0.43) 0.70 (0.23) 0.63 (0.03) 0.55 (-0.16) 0.48 (-0.36) 0.41 (-0.56) 0.33 (-0.75)
without any forecast E2 option. Table 3 also shows that both models give similar forecast values. Their corresponding no-skill random forecasts have lower forecast values due to their low skill (Table 2). It is also found that as the C/L ratio gets smaller, the forecast value decreases (Figure 3). Note that the value index (VI) of the no-skill forecast contains some non-positive value. In such a case, the forecast has no value, i.e. it is better just to use a daily protection regardless of any prediction. 6
Discussion This study is the first to implement and determine the value of a prediction by using a single mode of protection against DHF epidemics with an insect repellent. It is shown that the forecast implementation has an economic value. The value depends on factors such as forecast skills and the cost-toloss ratio. Simple protection using a DEETbased repellent is rarely used as a means for community protection against epidemics. Dengue Bulletin – Volume 33, 2009
Implementing predictive models for domestic decision-making against DHF epidemics
Figure 3: Calculated forecast values of predictive models including the no-skill random forecasts for a DHF patient at the hospital
The DEET-based product is highly effective and offers a broad-spectrum protection against mosquitoes, ticks, flies and insect bite.[23,24] Depending on application dosage and DEET concentrations, the product is able to give protection up to seven hours.[23,25] This product is also safe for adults and children provided the dose is correctly applied.[26] It is not surprising that DEET has been considered the singlemost effective personal protection for many years.[27] However, this mode of protection has not been widely used in a population against DHF epidemics. There are at least two reasons why the population at large is still reluctant to use a DEET product against epidemics. First, it might affect the human skin since the product is known to be corrosive to fabrics, plastic and vinyl.[28] Secondly, skin irritation, poisoning and toxicity occurrence have been reported in cases of excessive dosage. [29,30] Therefore, it is important to
ensure that the product is used properly. The recommendations to be followed are: there should be a six-hour interval between DEET applications, and the repellent should not be orally ingested.[31] In addition, for infants aged above two months, the product is limited to one application per day and the maximum DEET concentration should be 30%.[31]
Conclusion The skill of two simple models for predicting DHF epidemics is assessed using a Peirce score. The skill of HR2008 model is not significantly different than that of a persistence model. Both models have a much higher skill than that of their corresponding no-skill random forecast. Both model predictions are also applied to determine whether or not a family should take protective measures against mosquito bites.
Dengue Bulletin – Volume 33, 2009
7
Implementing predictive models for domestic decision-making against DHF epidemics
In order to avoid mosquito bites, use of a DEET-based repellent is proposed and simulated. It is found that the cost of implementing DEET application based on model predictions is lower than that of other options such as: never using any protection and never using any forecast when applying a protection. It is also shown that both models have a similar forecast value and they have a much higher economic value than that of noskill forecast. The forecast value gets smaller as the C/L ratio decreases.
Acknowledgements We thank Mr Suherman, a medical staff at Wahidin Sudirohusodo Public Hospital in Makassar, who provided us with the expenses and LoS data of DHF patients in the hospital. We also thank an anonymous reviewer for the constructive comments, and Dr Peter Ridd of James Cook University and Dr David McKinnon of the Australian Institute of Marine Science for proof-reading the manuscript.
References [1] Gubler DJ. Epidemic dengue/dengue hemorrhagic fever as a public health, social and economic problem in the 21st century. Trends in Microbiology. 2002;10(2):100-103. [2] van Damme W, van Leemput L, Por Ir, Hardeman W, Meessen B. Out-of-pocket health expenditure and debt in poor households: evidence from Cambodia. Tropical Medicine and International Health. 2004;9(2):273-280. [3] Shepard DS, Suaya JA, Halstead SB, Nathan MB, Gubler DJ, Mahoney RT, et al. Costeffectiveness of a pediatric dengue vaccine. Vaccine. 2004; 42, 1275-1280. [4] Anderson KB, Chunsuttiwat S, Nisalak A, Mammen MP, Libraty DH, Rothman AL, et al. Burden of symptomatic dengue infection in children at primary school in Thailand: a prospective study. Lancet. 2007;369:1452-1459. [5] Mimura N, Nurse L, McLean RF, Agard J, Briguglio L, Lefale P, et al. Small islands. In: Parry ML, Canziani OF, Palutikof JP , van der Linden PJ, Hanson CE. Eds. Climate change 2007: impacts, adaptation and vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press, 2007. p. 687-716. [6] Hales S, de Wet N, Maindonald J, Woodward A. Potential effect of population and climate changes on global distribution of dengue fever: an empirical model. Lancet. 2002;360:830-834. [7] Confalonieri U, Menne B, Akhtar R, Ebi KL, Hauengue M, Kovats RS, et al. Human health. In: Parry ML, Canziani OF, Palutikof JP , van der Linden PJ, Hanson CE. Eds. Climate change 2007: impacts, adaptation and vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge: Cambridge University Press, 2007. p. 391-431.
8
Dengue Bulletin – Volume 33, 2009
Implementing predictive models for domestic decision-making against DHF epidemics
[8] Barclay E. Is climate change affecting dengue in the Americas? Lancet. 2008;371:973-4. [9] Halide H, Ridd P. A predictive model for dengue haemorrhagic fever (DHF) epidemics. Int J Environ Health Res. 2008;18(4):253-65. [10] Drake JM. Fundamental limits to the precision of early warning systems for epidemics of infectious diseases. PLoS Medicine. 2005;2(5):e144. [11] Farrar J, Focks D, Gubler D, Barrera R, Guzman MG, Simmons C, et al. Towards a global dengue research agenda. Trop Med Int Health. 2007;12:695-9. [12] Runge-Ranzinger S, Horstick O, Marx M, Kroeger A. What does dengue disease surveillance contribute to predicting and detecting outbreaks and describing trends? Trop Med Int Health. 2008;13(8):1022-1041. [13] Nisalak A, Endy TP, Nimmannitya S, Kalayanarooj S, Thisayakorn U, Scott RM, et al. Serotype-specific dengue virus circulation and dengue in Bangkok, Thailand from 1973-1999. Am J Trop Med Hyg. 2003; 68: 191-202. [14] Stephenson DB. Use of the “odds ratio” for diagnosing forecast skill. Weather and Forecasting. 2000;15:221-232. [15] Klun JA, Strickman D, Rowton E, Williams J, Kramer M, Roberts D, et al. Comparative resistance of Anopheles albimanus and Aedes aegypti to N,N-Diethyl-3-methylbenzamide (Deet) and 2-Methylpiperidinyl-3-cyclohexen1-carboxamide (AI3-37220) in laboratory human-volunteer repellent assays. J Med Entomol. 2004;41(3):418-22. [16] Kay BH, Nam VS, Tien TV, Yen NT, Phong TV, Diep VTB, et al. Control of Aedes vectors of dengue in three provinces of Vietnam by use of Mesocyclops (Copepoda) and communitybased methods validated by entomologic, clinical, and serological surveillance. Am J Trop Med Hyg. 2002;66(1):40-48.
[17] McConnell KJ, Gubler DJ. Guidelines on the cost-effectiveness of larval control programs to reduce dengue transmission in Puerto Rico. Rev Panam Salud Publica. 2003;14(1):9-16. [18] Paeporn P, Komalamisra N, Deesin V, Rongsriyam Y, Eshita Y, Thongrungkiat S. Temephos resistance in two forms of Aedes aegypti and its significance for the resistance mechanism. Southeast Asian J Trop Med Public Health. 2003;34(4):786-792. [19] Suaya JA, Shepard DS, Chang M-S, Caram M, Hoyer S, Socheat D, Chantha N, Nathan MB. Cost-effectiveness of annual targeted larviciding campaigns in Cambodia against the dengue vector Aedes aegypti. Trop Med Int Health. 2007;12(9):1026-1036. [20] Thornes JE, Stephenson DB. How to judge the quality and value of weather forecast products. Meteorological Application. 2001;8:307-314. [21] Central Board of Statistics (BPS). Makassar in Figures, 2006 and South Sulawesi in Figures, 2006. [22] Costantini C, Badolo A, Ilboudo-Sanogo E. Field evaluation and the efficacy and persistence of insect repellents DEET, IR3535, and KBR3023 against Anopheles gambiae complex and other Afrotropical vector mosquitoes. Trans R Soc Trop Med Hyg. 2004;98(11):644-52. [23] Fradin MS, Day JF. Comparative efficacy of insect repellents against mosquito bites. N Engl J Med. 2002 July; 347 (1): 13-18. [24] Klun JA, Khrimian A, Debboun M. Repellent and deterrent effects of SS220, Picaridin and DEET suppress human blood feeding by Aedes aegypti, Anopheles stephensi, and Phlebotomus papatasi. J Med Entomol. 2006;43(1):34-39. [25] Kalyanasundaram M, Mathew N. N,N-diethyl phenylacetamide (DEPA): a safe and effective repellent for personal protection against hematophagous arthropods. J Med Entomol. 2006;43(3):518-525.
Dengue Bulletin – Volume 33, 2009
9
Implementing predictive models for domestic decision-making against DHF epidemics
[26] Katz TM, Miller JH, Hebert AA. Insect repellents: Historical perspectives and new developments. J Am Acad Dermatol. 2008;58(5):865-871. [27] W-Smith A, Schwartz E. Dengue in travellers. New England Journal of Medicine 2005 Sept; 353 (9): 924-932. Brown M, Hebert AA. Insect repellents: An overview. J Am Acad Dermatol. 1997;36:243-249. [28] A-Donia M, Dechkovskaia AM, Goldstein LB, A-Rahman A, Bullman SL, Khan WA. Coexposure to pyridostigmine bromide, DEET, and/or permethrin causes sensorimotor deficit and alterations in brain acetylcholinesterase activity. Pharmacology, Biochemistry and Behavior. 2004;77:253-262.
[29] Vijayaraghavan R, Rao SS, Suryanarayana MVS, Swamy RV. Acute and subacute inhalation toxicity studies of a new broad spectrum insect repellent, N, N-diethylphenylacetamide. Toxicology. 1991;67:85-96. [30] Schaefer C, Peters PWJ. Intrauterine Diethyltoluamide exposure and fetal outcome. Reprod Toxicol. 1992;6:175-176. [31] Hexsel CL, Bangert SD, Hebert AA, Lim HW. Current sunscreen issues: 2007 Food and Drug Administration sunscreen labelling recommendations and combination sunscreen/ insect repellent products. J Am Acad Dermatol. 2008;59(2):316-323.
10
Dengue Bulletin – Volume 33, 2009