Running an international survey in a small country: challenges and opportunities

292 Volume 2 | Issue 3 | sePTemBeR 2016 | 249-400PuBlIc healTh PanoRama background: national and international authorities recognize that health surveys are major sources of information on health conditions. Smaller states may prefer using health surveys to registries because they are cheaper to maintain. nevertheless, smaller states carry out far fewer national health surveys than larger states. one reason could be that the value of surveys depends on the number of people interviewed rather than the proportion of the population. Therefore, survey costs per capita are substantially higher in smaller states. methods: malta is a small state with a population of under half a million. It forms part of the european union, which has provided financial assistance and external expertise in performing international health surveys. We present the european health Interview Survey in malta as a case study to review the challenges for small states and the typical adaptations necessary for implementing national health surveys and meeting international health data obligations. results: We identified the lack of health survey infrastructure, difficulties in recruiting the large samples recommended by international organizations, survey fatigue, and a lack of resources for marketing, incentivization, analysis and dissemination. Low-cost solutions have been devised to address some issues, such as marketing and incentives, which exploit specific characteristics of small states. conclusion: In the absence of administrative data or epidemiological registers, surveys are important tools for evidence-based policy-making in small states. The experience of malta could help other small states to minimize the resources required to run national health surveys. Case study runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS Neville Calleja1, Paul H Garthwaite2 1 Directorate for Health Information and Research, Ministry for Health, Malta 2 Department of Mathematics and Statistics, Open University, United Kingdom Corresponding author: Neville Calleja (email: neville.calleja@um.edu.mt) abSTraCT keywords: InTernaTIonaL heaLTh SurveyS, SmaLL CounTrIeS, SmaLL STaTeS, ChaLLenGeS, oPPorTunITIeS, beST PraCTICe InTroDuCTIon Health surveys have tangible benefits: they identify needs for targeted actions and enable resources to be efficiently utilized where they will do most good. Many countries prefer to perform health surveys instead of active disease or lifestyle surveillance because the latter is costlier and may be too expensive to implement. In Europe, most national health surveys tend to be based on health examinations or health interviews because the response rate is highest with face-to-face contact than with other modalities (1,2). Those based on health examinations are costlier to run because they require trained interviewers and specialist clinical investigations. For these reasons, health interview surveys are more widespread (3). However, the accuracy of survey results depends on the actual number of responders rather than the proportion of the population. Hence, in smaller countries a greater proportion of the population needs to be included for a survey to be useful. The per capita cost of a useful survey is therefore greater in smaller than in larger countries. Limited resources in smaller countries are more likely to be invested in the health service infrastructure than health surveys. For these reasons, health surveys are less common 293 Том 2 | выпуск 3 | сЕНТЯБРЬ 2016 г. | 249-400паНоРама оБщЕсТвЕННого здРавоохРаНЕНиЯ in smaller countries. For example, Fig. 1 shows the proportion of countries that have implemented the World Health Organization STEPwise approach to surveillance (4). This survey was implemented in all World Health Organization regions, except for the World Health Organization European Region. Of the eligible countries, just under half of the 110 countries with a population above 1 million implemented the survey. In contrast, only 28% of the 68 countries with a population under 1 million people have implemented it. However, given the benefits that accrue from knowledge of a population’s health profile, it seems likely that smaller nations will need to make greater use of health surveys. Malta is the smallest Member State of the European Union, with a population of around 400 000. It has developed an effective survey system over a number of health surveys. Indeed, the European Health Interview Survey (EHIS) is the only health survey to be carried out more than once in Malta. All health surveys in Malta have been conducted by the Directorate for Health Information and Research (DHIR), under the portfolio of the Ministry for Health. DHIR is registered with Eurostat as a national statistical institute, thereby extending the remit of the Malta Statistics Authority Act to cover the DHIR as a separate body with a complementary function to the National Statistics Office (NSO) (5). Thus, the DHIR is bound to enact the provisions of European Union (EU) Regulations 1338/2008 and 141/2013, which specify the requirement for Member States to conduct a harmonized EHIS (6,7). EHIS datasets are anonymous (as explained to respondents). Potential ethical issues are resolved by the DHIR liaising with the Health Ethics Committee – a semi-independent body within the Ministry for Health that advises on data privacy and ethical principles, especially for research and clinical trial purposes (8). We review the challenges of health surveys that have been conducted in Malta, together with the workarounds, solutions and opportunities that Malta has identified to deal with these challenges. These issues are likely to be relevant to other smaller states. meThoDS The EHIS was not the first epidemiological study carried out in Malta. Three previous surveys were conducted in the 1980s: a diabetes prevalence survey (1981), the Monitoring Trends and Determinants in Cardiovascular Disease (MONICA) survey (1984) and the International Study of Sodium, Potassium, and Blood Pressure (INTERSALT) study (1986) (9–11). The EHIS has been held thrice in Malta: in 2002, 2008 and 2015 (9–11). One of the authors (NC) has been involved in the DHIR survey unit since 2002. We focus on the three editions of the EHIS as these are the more recent surveys (with similarities among all three editions), to identify the main challenges to their implementation and how these were overcome in the Maltese context. reSuLTS reSourcing The major challenges for health surveys in Malta are funding and human resources or expertise. In the first two editions of EHIS, these were provided by World Health Organization (in 2002) and Eurostat (in 2008). Without this assistance, Malta would have found it difficult to implement the survey. Current legislation now obliges all Member States to carry out this survey using national funding as, according to European funding policy, funding can only be made available for the first edition of an EU-level survey – which presents a proportionately bigger challenge to Malta than to larger countries in view of the higher costs per capita of a national survey, as described earlier. runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS fig. 1. proportion of countrieS implementing the WorlD health Survey 2002, by population Size 60 50 40 30 20 10 0 Source: World health organization (4). p ro po rt io n of c ou nt ri es (% ) population size (in millions) <1 1-10 10-100 >100 294 Volume 2 | Issue 3 | sePTemBeR 2016 | 249-400PuBlIc healTh PanoRama runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS While European legislation was originally thought to facilitate the funding of such initiatives by national public financial authorities, dedicated funding for the 2015 edition was not forthcoming and resources had to be identified internally and with great difficulty. As with several other western European countries, health surveys in Malta are the responsibility of the Ministry for Health rather than the NSO (12). The costs of running a free health care system are substantial, particularly for a small island nation. Logistical difficulties and the lack of economies of scale limit the purchasing power of the Maltese health care system and drive up per capita overhead costs. Understandably, delivering health care services takes priority over research into gathering health information. Follow-up has been a challenge for most health surveys conducted in Malta. While a number of other European states have managed to establish a sustained health survey infrastructure to repeat a survey with a set frequency for establishing trends, the high per capita costs have been prohibitive in Malta. In contrast, the NSO, established in 1948, has regular data collection in place for other non-health sectors. For example, the Household Budgetary Survey has been held every five years in Malta since 1969 (13). Currently, the Directorate is seeking external funding to build a digital survey platform. Along with recruiting a dedicated information technology business support officer, this would enable computer- assisted personal interviewing (CAPI) to replace paper-and-pencil interviewing (PAPI), the technique currently employed. This is certainly overdue, but the capital cost involved has been prohibitive for DHIR. Nevertheless, the recurrent costs, in terms of funding and human resources, of finalizing a dataset from a CAPI-based survey (if separate funding were obtained for the fixed cost of hardware and for developing the survey tool) are expected to be substantially lower than the current PAPI-based model. The remaining material costs are those of the interviewers’ time and training, but the costs associated with printing questionnaires and validating and processing PAPI data are virtually eliminated (14). Survey organization The EHIS is run by the DHIR, an entity with 25 staff members which is tasked with compiling all strands of national health statistics for the NSO. Another challenge, also related to cost, is its lack of expertise and trained human resources. In the limited Maltese labour market, training for health care service delivery takes priority over training for research into gathering health information. Full-time employment opportunities in such highly specialized areas are few; when available, maintaining competence is difficult due to limited demand. For this reason, most health care professionals tend to remain generalists to preserve mobility across the health care labour market, and those who do specialize are more likely to do so in clinical areas. DHIR has a dedicated survey unit that includes a full-time statistician and a part-time public health specialist with statistical expertise. Providentially, DHIR is a key placement in the local public health medicines specialist training programme. Therefore, while the EHIS were being conducted, a number of public health trainees were attached to the DHIR to help with the logistics of the survey, together with most of the permanent staff. While this is clearly convenient for the operations required within the DHIR, it also provides the trainees with surveying skills. Sampling As the number of respondents is what really matters in surveys, a higher proportion of citizens in smaller states need to be sampled, and this drives up the per capita cost. For the EHIS in 2014, EU Task Force III made recommendations on the number of completed questionnaires that each Member State should obtain to obtain coefficients of variation of about 5%. The initial recommendation was that Malta should obtain at least 5635 completed questionnaires, although this was later revised to 3975. The target level of accuracy would be achieved in more focused strata in Germany, the largest member of the EU, as the total sample size recommended for Germany was 15 260 (15). Therefore, the proportion of the population to be sampled is far smaller in Germany than in Malta. Nevertheless, a small state such as Malta has one advantage for sampling. In larger countries, two-stage sampling is commonly employed for logistical reasons: stage one samples localities and stage two samples individuals within chosen localities. In contrast, its smaller size means that single-stage sampling is viable in Malta. Interviewers can cover the entirety of the Maltese territory without major logistical difficulties. An advantage of single-stage sampling over two-stage sampling is that the sample size can be smaller; two- 295 Том 2 | выпуск 3 | сЕНТЯБРЬ 2016 г. | 249-400паНоРама оБщЕсТвЕННого здРавоохРаНЕНиЯ stage sampling is subject to a design effect of 1.5, which implies that the final sample size should be inflated by an additional 50% to compensate for the sampling technique (15). In Malta, the person identification number system allocates a unique number to each person at birth that remains valid for their entire life and beyond. These identification numbers are stored in an identity card number register that is used as the sampling frame for health surveys. In the case of administrative barriers preventing access to the identity card register, an alternative would be to use Malta’s electoral register. reSponSe rate EHIS response rates in Malta have been dropping: they were 78% in 2002, 72% in 2008 and 60% in 2015 (16,17). The flowchart in Fig. 2 shows how the 2008 survey the original sample size of 5500 was reduced to a final dataset based on 3680 completed questionnaires. The DHIR, which carries out the EHIS, is home to the National Death Register, so the selected sample population is linked to the death records, permitting a better estimation of the response rate in addition to avoiding unnecessary distress for relatives of any recently deceased persons in the sample. Contact with study subjects is first made by mail via an official letter. It is followed up with a phone call (when a phone number is available) to check that the addressee received the letter and confirm her/his willingness to participate. The Maltese community is fairly tight knit and people tend to be concerned about confidentiality, so both the initial letter and follow-up phone call emphasize that all responses are anonymous and that the identity of participants will be known to only the interviewer. They also stress the importance of the data for health service planning by the government and the fact that replacement of nonresponders is not permitted (as per Eurostat guidelines) (15). Respondents are also given the number of a mobile phone (manned by DHIR staff) that they can call throughout the survey period. If the respondent agrees to participate, he/she is informed that an interviewer will be in contact to arrange an appointment and that his/her details will be passed to the subcontractor performing the fieldwork. A problem in organizing surveys in Malta, not faced in some other countries, is that telephone contact numbers for the sampling dataset are not usually readily available. In some countries, linkage to telephone company registers is also possible, but this linkage service has been discontinued in Malta as the service was running at a loss because of the limited market. Hence, for health surveys in Malta a substantial number of person-hours used to be spent searching online through various directories for the telephone contact details of respondents. In 2011, the national population register maintained by NSO was updated to include telephone numbers. While these may not always be up to date, the burden of actively looking for individual numbers in telephone company websites is now reduced to include only those who cannot be contacted on the number provided. marketing Marketing of the survey usually involves a multipronged approach. Cost limitations mean that advertising is typically restricted to billboards along runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS fig. 2. floWchart Depicting attrition of the original Sample Size in malta in ehiS 2008 original Sample Size 5500 Deceased – 44 emigrated – 82 untraceable – 290 non-respondents 1444 eligible Sample Size 5124 respondents 3680 296 Volume 2 | Issue 3 | sePTemBeR 2016 | 249-400PuBlIc healTh PanoRama the two main traffic routes that traverse the island of Malta and adverts in the three main Sunday newspapers. Otherwise, the survey team engages with the media using free-of-charge methods. To date, most ministers of health have been willing to launch a survey themselves, achieving substantial media coverage for the survey. The Ministry’s communications office also provides the survey team with various talk-show slots on television and radio, which further promote the survey and effectively increase the response rate. In some talk-show slots, participants who had been contacted by letter but not by phone were encouraged to telephone the survey team. This was reinforced by a second mailshot making the same request, with the survey contact number placed prominently on the letter. This approach prompted a quarter of respondents whose telephone details had not been found to contact the survey team, and questionnaires were duly administered. incentiveS In a bid to promote participation, small incentives are given to respondents who complete the survey. Neither national nor European funds are available for such incentives, so their provision depends on the private sector. Sponsors provide discount vouchers for goods or services or small items (such as dried fruit portions or toiletries) in sufficient numbers to provide each respondent with a pack upon completing the survey. Larger items, such as gym equipment or flights, are used as prizes in a lottery of all successful respondents. Malta has had a positive experience with these incentives. While the pack is clearly too small to represent payment for participation, the token is appreciated by most respondents and helps health surveys maintain a positive profile in the Maltese population. Providing such incentives may require clearance from other ministerial bodies, such as fiscal authorities or, as in Malta’s case, with the Lotteries and Gaming Authority. fielDWork Fieldwork requires a network of about 40 interviewers assigned to specific localities covering the whole country. Four attempts by phone and two attempts at the doorstep are required before a selected individual is reported to the department as a nonrespondent, for follow-up by departmental staff. Validation (back- checking) is carried out on 15% of the study sample to verify that the survey questionnaire was properly administered and to assess respondent satisfaction with the process. European guidelines recommend spreading the fieldwork for a health survey across a full 12-month period to minimize seasonal biases. However, this option tends to be more expensive in terms of human resources compared with a shorter, more intense survey. When a 12-month survey is impractical, the guidelines recommend conducting the survey in autumn. This was the practice in Malta until 2015, when the survey was actually staggered over a 12 month period for operational reasons (15). Fieldwork is always suspended from mid-December to mid-January because the Christmas period would introduce bias and probably reduce the response rate. Summer also tends to decrease the availability of respondents. the queStionnaire The questionnaire used in Malta is based on the one adopted by Eurostat (18), supplemented by extra questions defined through consultation with policy- makers within the Ministry. Conceptual translation of the tool into Maltese was carried out following European guidelines (19). The option of using a translation made by another EU state is not available, as Maltese is only spoken in Malta. Given a choice between a questionnaire in English or Maltese, 80– 85% of the population opted for the Maltese version (Neville Calleja, Directorate for Health Information & Research, Malta, unpublished observations, 2002, 2008). input anD analySiS The questionnaires are returned to DHIR by the interviewers throughout the period of fieldwork. After matching the two parts, responses are coded and data scanned into a dedicated computer with a built-in backup system that is run at the end of every working day. Data input takes place concurrently with the fieldwork, thus shortening the timescale of the project. As data input is potentially time-consuming, an optical mark reader is used to enhance efficiency, requiring specific formatting of the questionnaire. Simple validation procedures are implemented in the software, such as verifying that the pages being scanned have consecutive page numbers and prompting operator intervention in cases of multiple responses. This system enables one person to input up to 100 questionnaires of 60 pages daily. runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS 297 Том 2 | выпуск 3 | сЕНТЯБРЬ 2016 г. | 249-400паНоРама оБщЕсТвЕННого здРавоохРаНЕНиЯ runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS Data are validated according to the validation routines specified in the Communication and Information Resource Centre for Administrations, Businesses and Citizens (CIRCABC) online portal before transmitting the survey microdata to Eurostat. Once data input is complete, data are exported into SPSS format (IBM SPSS Statistics software, Chicago, IL, USA) for further analysis. For the 2008 survey, in-house testing established that the respondent dataset did not vary significantly from the original survey sample in terms of demographic variables, so weighting was not necessary. Missing data were not imputed, so the body mass index was not estimated for the 20% of participants who had missing values for height. This resulted in a disclaimer by Eurostat that Maltese data on obesity was unreliable (20). This was unfortunate because obesity is a problem in Malta. Fig. 3 shows the prevalence of obesity in the 17 countries that participated in the 2008 EHIS: of these, Malta had the highest prevalence. For most other diseases, the prevalence estimates in Malta were reasonably similar to those of the other countries that participated in the EHIS, as for hypertension (shown in Fig. 4). Diabetes was another exception (see Fig. 5), partly due to the link between diabetes and obesity. DiSSemination When only one report presenting summary statistics was published following the 2002 EHIS, much criticism was levelled at the Ministry for making too little use of an expensive resource. Consequently, five reports were published over 24 months after the 2008 EHIS. The first contained summary statistics and the other four were separate thematic reports focusing on the use of health care services, the elderly, mental health and lifestyle. Each was launched by the responsible minister or parliamentary secretary, leading to good media coverage. Improved dissemination of the 2008 survey results led to a larger demand for customized data queries from this dataset, suggesting that the Maltese scientific community had become more aware of the resource. fig. 3. prevalence of Self-reporteD obeSity in malta anD other eu member StateS, ehiS 2008 Source: eurostat (http://ec.europa.eu/eurostat). prevalence of obesity (%) 0 5 10 15 20 25 romania bulgaria france austria belgium Slovakia cyprus Spain germany poland Slovenia latvia greece czech republic estonia hungary malta 298 Volume 2 | Issue 3 | sePTemBeR 2016 | 249-400PuBlIc healTh PanoRama runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS fig. 4. prevalence of Self-reporteD hypertenSion in malta anD other eu member StateS, ehiS 2008 fig. 5. prevalence of Self-reporteD DiabeteS mellituS in malta anD other eu member StateS, ehiS 2008 0 5 10 15 20 25 30 35 40 0 1 2 3 4 5 6 7 8 9 Source: eurostat (http://ec.europa.eu/eurostat). Source: eurostat (http://ec.europa.eu/eurostat). france romania belgium Spain malta estonia cyprus greece austria poland Slovenia germany latvia czech republic bulgaria Slovakia hungary romania estonia latvia bulgaria belgium poland france austria cyprus Spain Slovakia czech republic Slovenia malta germany greece hungary prevalence of hypertension (%) prevalence of diabetes mellitus (%) 299 Том 2 | выпуск 3 | сЕНТЯБРЬ 2016 г. | 249-400паНоРама оБщЕсТвЕННого здРавоохРаНЕНиЯ Following the recent validation of the 2015 dataset, data analysis is due to commence. Although better public awareness was achieved following the 2008 dissemination, generating the thematic reports was a laborious task (particularly the textual part, as opposed to the contingency tables). For this reason, although more thematic outputs are planned for the third edition, it is being proposed that these could take any form from simple factsheets up to full reports, depending on data availability and the target audience. Stakeholders are being invited to propose themes and, if successful, will be invited to contribute to the content and coauthor the respective report. Metadata will be kept concise and reports will concentrate more on contingency tables which, according to feedback, seem to be the most sought after parts of the EHIS 2008 reports. DISCuSSIon The challenges we have described are considered those most likely to affect small states (or small regions within larger countries) attempting to implement similar national scale surveys. As documented for other sectors, the Maltese experience is an example of vulnerability leading to resilience through generating innovative solutions (21). As expected, funding and resourcing were the limitations behind most challenges. However, better infrastructure now appears to be in place for other surveys within the NSO. It is possible that surveys are made a higher priority, and the need for and benefits of an economic or social survey are better understood, when sponsored by a central statistical office rather than when funded from a health budget. There is limited discussion of this in the literature; however, in most European countries health surveys tend to be within the portfolio of the ministries of health rather than with the national statistical authorities. In the face of such scarce resources, maximizing the response rate becomes paramount. The observed drop in response rates may be due to survey fatigue as a consequence of the increasing annual number of surveys carried out in Malta by the NSO, market research companies, European bodies and other entities. Survey fatigue has also been reported in other limited populations, the classic case being the Aboriginal population of Western Australia. This population is heavily surveyed and most studies report a low response rate (22). Survey fatigue is also evident in larger populations such as the United Kingdom (23) where, for example, lifestyle surveys have regularly reported survey fatigue since the 1990s (24,25). However, the decline in survey response rates is an international phenomenon. The use of incentives in Malta appears to have been more successful than in other countries, where cash incentives as large as €50 have not substantially improved response rates (26,27). In terms of selective nonresponses, the problem of low awareness of body height in self-reported data is not limited to Malta. In 2010, Marston et al. reported a high degree of missing height data in primary care databases that tend to rely on self-reported data (28). ConCLuSIon The Maltese EHIS dataset is intensively used for policy setting and service planning, generating many background papers for the government. The co-occurrence of a wide range of sociodemographic, lifestyle and morbidity variables in this resource helps to overcome the lack of specialized epidemiological registries, which, as a small country, we are limited in the number we can afford. The current pressure to expedite validation of the latest 2014/2015 dataset testifies to its wide-ranging use from a resource for answering parliamentary questions to preparing hospital design briefs to collaborative exercises for planning health programmes. We hope that the ever- increasing demand for evidence-based policy-making will make resourcing of health surveys easier for small states. acknowledgements: We acknowledge the contribution of the whole team at DhIr throughout the three editions of the survey, in particular, amanda Cassar, Dorothy Gauci, audrey Galea, kathleen england and renzo Pace asciak, who were the major contributors over the years. Source of funding: none declared. conflicts of interest: none declared. Disclaimer: The views expressed in this publication are those of the authors alone, and do not necessarily represent the decisions or policies of the World health organization. runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS 300 Volume 2 | Issue 3 | sePTemBeR 2016 | 249-400PuBlIc healTh PanoRama referenCeS 1. Picavet hSJ. national health surveys by mail or home interview: effects on response. J epidemiol Community health. 2001;55:408–13. doi: 10.1136/jech.55.6.408 2. Sinclair m, o’Toole J, malawaraarachchi m, Leder k. Comparison of response rates and cost–effectiveness for a community-based survey: postal, internet and telephone modes with generic or personalised recruitment approaches. bmC med res methodol. 2012;12:132. doi: 10.1186/1471-2288-12-132. 3. aromaa a, koponen P, Tafforeau J, vermeire C, hIS/ heS Group. evaluation of health interview surveys and health examination surveys in the european union. eur J Public health. 2003;13(3 Suppl):67–72. doi: 10.1093/ eurpub/13.suppl_1.67. 4. STePwise approach to surveillance (STePS). Geneva: World health organization; 2016. (http://www.who.int/ chp/steps/en/, accessed 10 July 2016). 5. Government of malta. malta Statistics authority act. 2001. valletta, malta: Government of malta (http:// www.justiceservices.gov.mt/DownloadDocument. aspx?app=lom&itemid=8888&l=1, accessed 10 July 16). 6. european Parliament and Council. regulation (eC) no 1338/2008 of the european Parliament and of the Council of 16 December 2008 on Community statistics on public health and health and safety at work. official Journal of the european union. 2008;L354/70–81. 7. european Parliament and Council. Commission regulation (eu) no 141/2013 of 19 february 2013 implementing regulation (eC) no 1338/2008 of the european Parliament and of the Council on Community statistics on public health and health and safety at work, as regards statistics based on the european health Interview Survey (ehIS). official Journal of the european union. 2013;L47/20–48. 8. health ethics Committee. In: ministry for health [website]. valletta, malta: Government of malta; 2016. (https://health.gov.mt/en/appbodies/hec/Pages/hec. aspx, accessed 10 July 2016). 9. Cuschieri S, mamo J. malta: mediterranean Diabetes hub – a journey through the years. malta medical Journal. 2014;26(3):27–31. 10. malta InTerSaLT blood Pressure Data 1986. In: Global health Data exchange [website]. Seattle: Institute for health metrics and evaluation; 2016 (http://ghdx. healthdata.org/record/malta-intersalt-blood-pressure- data-1986, accessed 10 July 2016). 11. Surveys. In: ministry for health [website]. valletta, malta: Government of malta; 2016. (https://health.gov. mt/en/dhir/Pages/surveys.aspx, accessed 10 July 2016). 12. unit f-5: education, health and social protection statistics. modernisation of social statistics (Doc 2012- Ph-02). Luxembourg: eurostat; 2012. (https://circabc. europa.eu/d/a/workspace/SpacesStore/cefc4af1- 81a8-4d85-8023-2a9fae5f7baa/Item%203.4%20-%20 modernization%20°f%20social%20statistics.pdf, accessed 10 July 2016). 13. vella C. malta’s Statistical Institute: a concise history 1947–2011. valletta, malta: national Statistics office; 2011. 14. Caeyers b, Chalmers n, De Weerdt J. a comparison of CaPI and PaPI through a randomized field experiment. uSa: Social Science research network. 2010 (http:// ssrn.com/abstract=1756224) doi:10.2139/ssrn.1756224. 15. axelson m, bihler W, Djerf k, Lehtonen r, molenberghs G, Scavalli e et al. european health Interview Survey: Task force III report on sampling issues. Luxembourg: eurostat; 2009. 16. Department of health Information. The first national health Interview Survey: summary statistics. valletta, malta: ministry of health, the elderly and Community Care; 2003 (https://health.gov.mt/en/dhir/Documents/ national_his_a__summ_stats.pdf, accessed 10 July 2016). 17. Directorate of health Information and research. european health Interview Survey 2008: summary statistics. valletta, malta: Directorate of health Information and research; 2009 (https://health.gov. mt/en/dhir/Documents/ehis_eight__summ_stats.pdf, accessed 10 July 2016). 18. final european health Interview Survey questionnaire. Luxembourg: eurostat; 2006. 19. Tafforeau J, López Cobo m, Tolonen h, Scheidt- nave C, Tinto a. Guidelines for the development and criteria for the adoption of health Survey instruments. Luxembourg: Partnership health/eurostat; 2003. 20. european health Interview Survey: between 8% and 25% of adults are obese across member States [press release, 24 november 2011]. Luxembourg: eurostat; 2011. 21. briguglio L. a vulnerability and resilience framework for small states. valletta, malta: university of malta; 2014 (https://www.um.edu.mt/__data/assets/ pdf_file/0007/215692/briguglio_The_vulnerability_ resilience_framework,_23_mar_2014.pdf, accessed 10 July 2016). 22. ryan m, mcewin m. Improving the quality of data on early childhood education – phase two (IQDeCe2). Canberra, australia: Department of education, employment and Workplace relations; 2009. 23. koponen P, aromaa a. Survey design and methodology in national health Interview and health examination surveys. european health Survey Information Database; 2004. (http://ec.europa.eu/health/ph_projects/2000/ monitoring/fp_monitoring_2000_annexe14_04_en.pdf., accessed 10 July 2016). runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS 301 Том 2 | выпуск 3 | сЕНТЯБРЬ 2016 г. | 249-400паНоРама оБщЕсТвЕННого здРавоохРаНЕНиЯ 24. hill a, roberts J, ewings P, Gunnell D. non-response bias in a lifestyle survey. J Public health med. 1997;19(2):203. doi: 10.1093/oxfordjournals.pubmed. a024610. 25. mcCluskey S, Topping ae. Increasing response rates to lifestyle surveys: a pragmatic evidence review. Perspectives in Public health. 2011;131(2):89. doi:10.1177/1757913910389423. 26. brick Jm, Douglas W. explaining rising nonresponse rates in cross-sectional surveys. The annals of the american academy of Political and Social Science. 2013;645:36 doi: 10.1177/0002716212456834 27. vissink P, venmans L, holleman P, blokstra a, verschuuren m. Sampling, recruitment and response rate in a Dutch health examination Survey 2009-2010. eur J Pub health (2014) 24(S2):3428. 28. marston L, Carpenter Jr, Walters kr, morris rW, nazareth I, Petersen I. Issues in multiple imputation of missing data for large general practice clinical databases. Pharmacoepidemiol Drug Saf. 2010;19: 618–62 doi: 10.1002/pds.1934. runnInG an InTernaTIonaL Survey In a SmaLL CounTry: ChaLLenGeS anD oPPorTunITIeS