Report on the third workshop on patient safety in the Eastern Mediterranean Region, Luxor, Egypt, 12-15 February 2007

Report on the

Third workshop on patient safety in the Eastern Mediterranean Region

Luxor, Egypt 12-1 5 February, 2007

World Health Organization Reglonal Offlce for the Eastern Med~terranean

O World Health Organization 2008 All rights reserved. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers' products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. The World Health Organization does not warrant that the information contained in this publication is complete and correct and shall not be liable for any damages incurred as a result of its use. Publications of the World Health Organization can be obtained from Distribution and Sales, World Health Organization, Regional Office for the Eastern Mediterranean, PO Box 7608, Nasr City, Cairo 11371, Egypt (tel: +202 2670 2535, fax: +202 2670 2492; email: DSA@emro.who.int). Requests for permission to reproduce WHO EMRO publications, in part or in whole, or to translate them - whether for sale or for noncommercial distribution - should be addressed to the Regional Adviser, Health and Biomedical Information, at the above address (fax: +202 2276 5400; email HBIOemro.who.int).

Document WHO-EM/HCD1084IWO1.081164

CONTENTS 1.

INTRODUCTION .............................................................................................................

8

2. 3.

BACKGROUND ............................................................................................................... 2 MEDICAL RECORD REVIEW PROJECT: LOOKING FORWARD BY LOOKING BACK ................................................................................................................................ 3 ................... ............ 3 3.1 RF1 screenin . . . . . 3.2 RF2 review1 3.3 General patient informatio 3.4 Review time .......................................................................................................... 3.5 Adequacy of medical records ......................... 5 5 3.6 Sampling and hospital selection ............................................................................... COUNTRY PRESENTATIONS 4.1 E ~ Y P 4.2 Jordan ..... ............................................................... 4.3 Morocco 4.4 South Africa 4.5 Sudan ............... . . ...... . . ........................................................................................ . . 4.6 Tunlsla ........................................ 4.7 Yemen ....................................................................................................................... 6 6 6 6 7 7 8 8

4.

5.

CONCLUSIONS ............................................................................................................... 8 5.1 Improving medical records 5.2 Use of the medical record 5.3 Ensuring a strong project structure 9 5.4 Use of the data at country level ................................................................................ 5.5 Low RF1+ rates ................................... 10 NEAR FUTURE PLANS OF ACTION .......................................................................... 10 6.1 Egypt .................................................................................................................... 11 6.3 6.4 Morocco South Africa

6.

................................................................ 11

7.

.......................12 REPORTING .... 7.1 National (central) leve 12 7.2 Facility (peripheral) leve 13 7.3 Regional and international level: Publishing data .................................................. 14 7.4 Lessons learnt, challenges and need .14

8.

RECOMMENDATIONS ................................................................................................

14

Annexes 1 . AGENDA ............................................................ 16 ............................. 17 2. PROGRAMME..... ......................... 19 3 . LIST OF PARTICIPANTS 4 . DATA ENTRY ................................................................................................................... 35 5. REQUESTED FORMATS FOR DATA PRESENTATION ................................................. 45 6 . ALTERNATIVE RESEARCH METHODS ........................................................................ 47 7 . REFERENCES .................................................................................................................... 63

1.

INTRODUCTION

The third regional workshop on patient safety research was organized by the World Health Organization (WHO) Regional Office for the Eastern Mediterranean (EMRO) from 12-15 February 2007, in Luxor, Egypt. This meeting represented a follow-up mechanism on plans of action previously agreed upon in the first and second workshops held at the Regional Office for the Eastern Mediterranean in Cairo, in December 2005 and May 2006, respectively. The first two meetings resulted in the establishment of research on the assessment of the nature and magnitude of adverse events in the health care system in six counties of the Region (Egypt, Jordan, Morocco, Sudan, Tunisia and Yemen), as well as two countries in the African Region (Kenya and South Africa). The objectives of the workshop were to: follow up on previously agreed upon plans of action; share the results of the research studies performed in each of the six participating countries of the Region; discuss a follow-up mechanism for the development of a national strategy for implementation of research studies; discuss methods of research application towards national capacity-building. Dr Ahmed Abdellatif, Coordinator, Health Systems Development, Regional Adviser, Health Care Delivery (WHOIEMRO) delivered the opening address of Dr Hussein A. Gezairy, WHO Regional Director for the Eastern Mediterranean. Dr Gezairy commended the national teams and highlighted the tremendous impact that the on-going research studies would have on the research agenda that the World Alliance for Patient Safety was building and on prioritizing the areas in the health care system that required attention. Indeed, research was crucial for advancing the progress of health care in countries and it was through combined and relentless efforts that a critical mass would be created to put institutions in a learning mode and instigate change. The findings of the research studies, with their regional nature and with confidentiality secured, would likely serve the purpose of raising issues for discussion, sharing lessons, establishing quality through teamwork, root cause analysis and evidence-based methodologies and techniques. Dr Riham El-Asady (Egypt) was elected rapporteur. The agenda, programme and list of participants are included as Annexes 1, 2 and 3, respectively.

WHO-EM/HCD/084/E Page 2 Annex 4 contains a comparison of country approaches. Annexes 5, 6 and 7 contain instructions for data entry, requested formats for data presentation and information on alternative research methods, respectively. Annex 8 contains a list of bibliographical references.

2.

BACKGROUND

The World Alliance for Patient Safety (WAPS) was formed in October 2004, in Washington D.C., in response to a World Health Assembly resolution in 2002 urging WHO and Member States to pay the closest possible attention to patient safety. This worldwide endeavour brought together the heads of agencies, health policy-makers, WHO and patient groups to advance the fundamental goal of "First do no harm" and reduce the potential unwarranted harm inflicted on patients. The fundamental purpose of WAPS is to facilitate the development of patient safety policy and practice through the fulfilment of a number of core functions and other short-term initiatives. The development of baseline studies pertaining to patient safety and adverse event measurement in developing countries is of high priority to the Alliance. The notion that preventable patient harm is completely unacceptable is a fundamental concept. The magnitude of the problem may be underestimated and events that culminate in patient harm remain poorly represented and should be the focus of attention by researchers. While validated tools for measurement are available, putting such tools into practice and enhancement of their efficiency is of utmost importance, particularly where medical resources are inadequate. With the purpose of implementation of patient safety measurement tools and training on the methods of hospital record review using the guidelines set by WHO and WAPS, two workshops were held at WHO Regional Office in Cairo in December 2005 and May 2006. Six countries of the Region participated in the workshop: Egypt, Sudan, Jordan, Tunisia, Morocco and Yemen. In addition, two countries from WHO African Region were involved-South Africa and Kenya. Participants included a diverse array of technical expertise including: physicians, nurses, epidemiologists, statisticians, l T specialists and ministry of health officials. The workshops were conducted by an expert panel from Australia (Northern Centre for Healthcare Improvement (NCHI), the United Kingdom (UK) and France and involved a comprehensive analytical review of the current literature, a description of the magnitude of the problem of patient safety and on-site training on the method of record acquisition and analysis. The World Alliance Research Council was established in Washington in April 2006 to oversee the research agenda through a live process, fed by constant input and analysis, During the first training workshop held in Cairo in December 2005, an action plan was developed in accordance with the international guidelines. The plan was formulated to include key elements required for the management, adoption and implementation of a research project aimed at assessing patient safety. The action plan was tailored to accommodate the needs of individual countries and a template was

WHO-EM/HCD/084/E Page 3 created for country proposals which were presented during the final sessions of the first workshop. In the second training workshop, individual countries gave a progress report indicating actions taken for each of the items included in the action plan and highlighting major difficulties faced during practical application of the plan. The focus of the workshop was to use available measurement tools to assess the size and nature of patient harm in developing countries. The workshop was devoted to training on the use of the research tools (Review Form 1 (RF1) and Review Form 2 (RF2)) and discussing pitfalls associated with their use. Each country team embarked on a project with a tentative plan to review retrospective medical records in a number of hospitals and to report results through a web-based data entry system (NCHI, Australia) which will ultimately provide data cleaning and analysis. An on-going follow-up mechanism was created between the expert team and country teams via visits and teleconferences. The outcomes of the projects to date are discussed in the current meeting.

3.

MEDICAL RECORD REVIEW PROJECT: LOOKING FORWARD BY LOOKING BACK RFl screening

3.1

The number of records entered to date for RF1 review ranged from between 201 and 1565 per country. The number of records that showed positive in the RF1 review ranged from 13 to 149, with the percentage of RFl+ve records as low as 6.5% in one country to as high as 30.5% in another. (The latter figures represent the percentage of RFl+ve records among records screened and entered to date). On average, the percentage of RFl+ve records among total records entered in the database for all participating countries to date is 16.3%. These numbers are relatively low compared to international standards. The figures may be explained by the fact that most countries are in the initial phases of data entry and have thus not entered the majority of reviewed records in the database. In addition, some countries started by entering the records that were RF1-ve for convenience. Finally, one country entered data that was dominated by obstetricslgynecology (ObIGyn) records, known to include many cases of normal delivery. As such, the results may be skewed towards records of (commonly uncomplicated, thus RF1-ve) normal delivery. It was additionally noted that very few records showed more than one positive criterion. This most likely indicates that the RF1 screening process was stopped as soon as one positive criterion was detected.

WHO-EMIHCDl084IE Page 4 The following conclusions can be made at this phase of the study: As few as 12 records were entered in the database for RF2 reviewing in one country, while at the opposite end of the spectrum; another country entered 143 records for RF2 reviewing (to date). All country groups were able to detect (RFl+) criteria in accordance with rules previously taught using medical record reviewing. There is a positive correlation between the RFl+ve rate and adverse event rate suggesting that RFI reviewing is a critical cornerstone of the project. It is better to err towards the side of inclusion, i s . it is better to include +ve criteria if in doubt. This will ensure that adverse events (AE) will not be missed. RF1+ records will go through a more rigorous screening process, RF2 review, to determine if the criteria suggested by W 1 do indeed translate into an adverse event. If records entered are dominated by one clinical specialty, case mix adjustment will need to be made. ICD-10 coding is critical for this process. All positive criteria detected on the record should be documented. The screener should not stop the RF1 review process once one positive criterion is found.

3.2 RF2 reviewing Records that have been RF2-reviewed and entered to date were as few as 12 in one country and as many as 143 in another. The adverse event rate per admission (total number of adverse events as a percentage of the total number of records entered) ranged from between 1.9% and 13.5% (mean 5.9%). This is again lower than published international studies. For example, the Quality in Australian Health Care Study (QAHCS),' reported that 16% of hospital admissions were associated with adverse events. These data yet again emphasize the importance of the RF1 screening phase of the project. If a record misses the opportunity to be screened during RFI review, the chances that an adverse event will be missed increases. 3.3

General patient information

On review of analysed interim findings, it was noted that the mean length of stay of patients was relatively short in some countries. Although this may be partially explained by the skewed inclusion of ObIGyn cases (most of which will likely be uncomplicated nonnal deliveries with a short length of stay) in some countries, it may, however, reflect technical errors during the data entry phase and warrants attention. This is particularly important when the cost of adverse events is to be calculated in terns of length of stay. Similarly, missing data on the date of discharge are often noted. It is worthy of note that the platform of the database stores page-by-page of the data entry system. Thus, if one field (e.g. date of admission) on a page is filled out (by the person entering the data), it is unlikely that any other field on that page (e.g. date of discharge) will have been lost as a result of Internet disconnection, etc. That is, a

WHO-EM/HCD/084/E Page 5 whole page in the data entry form may be lost as a result of loss of Internet connection. Nonetheless, individual fields will not be lost in the same manner. Missing individual fields are more likely to be an omission on the part of the person entering the data. Another common problem was encountered in entering patients' ages. Thus, the number of patients aged below 1 year was surprisingly high in some countries. This may be due to errors in entering the year of birth. Day-only admissions were noted in several records. This may either reflect errors in the entry of the date of discharge or, alternatively, that such records were erroneously included.

3.4

Review time

In general, the time spent in RF1 and RF2 reviewing is less than the average international figures (15 and 21 minutes respectively, Wilson, R. McL, personal communication) in most countries. This may reflect the fact that some RF1 screeners opted to stop the review process as soon as one criterion was detected on RFl (see 3.1 RF1 screening).

3.5

Adequacy of medical records

The majority of medical records sampled for the study appear to have been sufficient for the review process as indicated by the participants during data entry. The presence (or absence) of nursing notes may have been subject to individual variation. For example, the mere presence of a vital signs chart was considered by some country teams as available nursing notes, while others did not consider them as ample nursing notes. For the most part, however, the consensus has been that most medical records were adequate, providing credibility to the use of the medical record review methodology for performance of measurement studies.

3.6 Sampling and hospital selection One important item that arose from the discussions was that the collected data can only be reflective of the performance of the selected hospitals. It cannot be generalized to cover the countries from which the hospitals were selected. As such, information must be provided about the selected hospitals and the way the records were sampled. Thus, country teams may have opted to choose flagship hospitals, which do not necessarily reflect the average standard of care in a country but rather the highest standard of care provided. One can conclude, therefore, that the adverse event rate in the country is probably higher than that recorded in the selected hospitals. In some cases though, the standard of the medical records did not necessarily correlate with the reputation of the hospital. Thus, some large flagship hospitals were found to have poor records.

WHO-EM/HCD/084/E Page 6

4. 4.1

COUNTRY PRESENTATIONS Egypt

Among the first challenges that met the Egyptian team was the involvement of health care officials in the problem of avian influenza (recently discovered in Egypt). As such, less interest was directed towards patient safety. It was thus a difficult task to obtain approval of, and raise interest in, a project on adverse event measurement. A conflict of interest arose when quality control units in participating hospitals felt that the project was an intrusion into their affairs. Such units considered patient safety their concern and in one hospital were not forthcoming during the conducting of the project. Finally, training screeners and reviewers in accordance with the guidelines set by WHO Regional Office training workshops proved to be a challenge. The needs of the Egyptian team include: improvement of data entry; training on data analysis for future work; and the provision of training materials in Arabic if possible.

4.2

Jordan

The challenges faced by the Jordanian team while performing the study included: keeping screeners and reviewers on board all the time; moving medical records safely to the quality directorate for data entry; less than optimum resources (e.g. using the Ministry of Health vehicle for transportation, copying a huge number of forms, using dial-up Internet connection). The team leader described the most outstanding needs of the team as: improving communication with NCHI to support data entry; improving Internet connections for data entry; and disbursement of allocated funds in a timely fashion.

4.3

Morocco

The major challenges that the Moroccan team faced during the project were the difficulty in obtaining the acceptance of many professors of having an external body examine their records as the information sometimes had to be transported outside of the hospital for data entry. Furthermore, anonymity was an essential condition for acceptance of the study by health authorities. Because the data entry process could not be performed during the review phase on site, the patient identification data had to be preserved by the review team, a concern that was not well tolerated by hospital officials. Local capacity-building was raised as an important outstanding issue and a major challenge in Morocco. The major needs of the team in Morocco include: access to a national database; providing participating hospitals with de-identifiable results; support in report generation; and local capacity-building to enable independent reproducibility of the project.

WHO-EM/HCD/084/E Page 7 4.4

South Africa

The process of scrutinizing patient folders was frequently labourious due to the size of the folders. Notes were also frequently incomplete. In hospital number 2, more than half of the notes had been microfilmed without regard for chronology. The problem of incomplete notes is even greater in non-teaching hospitals. Also, admission dates in some hospitals were often recorded the day after admission in patients presenting after 4 p.m. This further complicated the review process. The time recorded for reviewing does not take into consideration the time taken to obtain the folders, sort out contents and other administrative issues. Inadequate documentation leading up to, or resulting in, adverse events makes it very difficult to classify adverse events correctly. It was not possible to obtain additional information from relevant staff member(s) to answer queries posed by the case in question. It is often difficult to obtain duplicate reviews on files because the forms left on the side for double reviewing have to be returned back to the record department before the end of the day. The review tools did not allow for the identification of contributory factors and were limited to identifying the frequency and types of adverse events only. Advising hospitals on methods of improvement will be an additional challenge. The following are the most outstanding needs for the South African team: a less resource-intensive methodology; an adequate budget; the training of medical and nursing staff relevant to the institution to be surveyed (rural versus academic hospitals), although combining the current methodology with alternative tools may compensate for some of the difficulties met. 4.5

Sudan

Among the many lessons learnt, the team in Sudan discovered that patient safety is of interest to many people in the health care system. Resistance to measurement studies (for adverse event detection) is minimal and there is willingness to improve. Another important lesson is that reviewers are the cornerstone in this study. An appropriate choice of reviewers is critical for success. Finally, no incident is a "minor incident". Every adverse event should be considered, counted and investigated. The team in Sudan were faced with several problems and challenges while conducting the on-going study. Most staff worked on the project on a part-time basis. It was extremely difficult to get full-time commitment, particularly from doctors. In addition, the language used to conduct the study (English) was foreign, especially to nurses and thus imposed a barrier. There was some minor deviation from the standard protocol used in the study, e.g. in the number of reviewers recruited. Furthermore, the insufficiency of medical records in some hospitals represented an obstacle in the project. Finally, data entry is a slow process with the poor Internet connection in Sudan. Sustainability of the project and national capacity-building are a major challenge to the team. Indeed, keeping the team together and creating a system to

WHO-EM/HCD/084/E Page 8 oversee adverse event measurement will be critical in the long term. The major needs of the Sudanese team include: guidance on report generation and aid in the controlled release of results to different parties such as policy-makers, hospital directors, health care providers and scientific journals; improvements to the data entry process; and the proposal of a plan for sustainability.

4.6

Tunisia

The Tunisian team faced a challenge in engaging local leadership and sensitizing clinical departments to obtain collaboration on the project. For the most part, there were no central archives and medical records had to be pulled out from each department separately. On doing so, the team met occasional resistance. The Tunisian team described the following items as outstanding needs. There is a need for continuous technical support for successful completion of the project; assistance for the expansion of the project to involve more hospitals at country level; and; national capacity-building and exploration of alternative tools.

4.7

Yemen

The team in Yemen described the data entry phase of the project as problematic and a major challenge particularly in light of weak Internet connections in the country. The team has thus far had to use dial-up connection. Data missing from files after analysis (e.g. certain fields in the patient identification page) could not be accounted for and warranted investigation. A second challenge for the Yemeni team has been the incompleteness of data in medical records. Finally, obtaining funds for the project has proved to be a slow process. These funds are critical to maintain the project's momentum and speed up progress. The Yemeni team described their most outstanding needs as: facilitation of data entry; time required for completion of the two remaining hospitals; and obtaining funds in a timely fashion.

5. 5.1

CONCLUSIONS Improving medical records

One of the most important aspects of this project is how it can be used to drive change towards an improved national medical record archive system. Furthermore, the outcomes of this study will likely underscore the importance of documentation by health care providers (nurses and doctors) in a legible, conscientious and meticulous manner. Indeed, medical records should be accurately representative of the patient's journey through the health care system. Standards for systemic record keeping systems need to be put in place. The fact that departments can keep their own records is historically documented, but it is faulty. The use of a system that has terminal digit hospital filing is crucial. Furthermore, all results need to be kept in the records. Nursing progress notes are an essential item. In fact, they are the best way of documenting what happens to a patient

WHO-EM/HCD/084/E Page 9 on a daily basis. Exploiting the nurse's expertise in a chronological system is essential. The medical record archive system does not necessarily need to be computerized but it should be coded. In some countries, the systematic standardization of a medical record system is part of the general accreditation process. This may encourage the employment of such methods in hospitals. The results of the current research should act as an agent for change, highlighting errors in the medical record keeping system as it stands, underscoring the significance of using an adequate system and instigating the enthusiasm to improve record systems at national level. 5.2

Use of the medical record review methodology

The use of the medical record review methodology for determination of adverse events is not flawless and has limitations. However, several research studies have provided evidence that medical record review is relatively efficient in capturing incidents of patient harm.2 Indeed, recent studies comparing retrospective record review with other tools for adverse event detection such as incident reporting systems, have shown that the latter may not provide an accurate picture of the extent and severity of patient safety incidents, particularly those resulting in harm to patients.3 The current studies have demonstrated that medical records review methodology can indeed be used in developing countries as a tool for measurement of patient h m . Although this method may underestimate error, it will document measurable harm. Thus, if a patient requires a second surgery inadvertently, it will be recorded in the medical record. Likewise, if a patient's death is an outcome of an admission, it cannot be hidden. It will appear glaringly on the record and will be investigated if unanticipated as a natural consequence of the disease process. Thus, while documentation of error will likely be deficient in medical records, the consequential outcome of the error will likely be present in the medical record. A detailed description of alternative methodology is presented in Annex 7 of this report. 5.3

Ensuring a strong project structure

The structural aspects of the project management process are key to the success of the project. This includes items such as hospital selection, reviewer training, ongoing reviewer monitoring, ensuring record randomization and appropriate case mix. Consistency in training is another must. In that regard, continuous communication/consultation with the expert training team is essential for provision of technical support along short time intervals to compensate for the possibility of key elements getting lost along the way.

5.4

Use of the data at country level

The purpose of this section is to highlight that the data provided by countries will be analysed using international software (at NCHI) to ensure consistency of results. Data will be presented in an aggregated form with confidentiality secured. The

WHO-EM/HCD/084/E Page 10 cleaned data will then be provided to participating countries (each country only receiving its own set of data) on excel files for further use at country level. It is worthy of note that it will not be possible to review enough cases from any one country to be able to generalize the results nationally. If patient safety is to be prioritized on the health care policy agenda, data is needed in an effective package. Such a package has been (and will further be) provided by the power of collaboration between many countries of the Region in the project. The results will likely provide good evidence of the need for improvement and will set the stage for many other patient safety activities. 5.5

Low RFl+ rates

One of the most important outcomes of the current workshop is the emphasis on the significance of correct RF1 reviewing in increasing the credibility of the data. Several lessons have been learnt. The initial data analysis phase has shown that RF1 rates are generally low. Some results (Jordan) were overwhelmingly dominated by obstetrics records, known to involve normal deliveries for the most part. This may partially explain the low RF1 rate. The initial data entered by some countries (Tunisia) have been the records known to be RF1 for convenience and to speed up the process of data entry (because RF1 files will not require too much time for data entry). In addition, the replacement of nurses who conventionally perform RFl review with medical students may have negatively impacted the RFl rate. The medical records reviewed (and entered in the database) at the initiation of the study will have probably reflected the lower efficiency and suboptimal competence of reviewers at the initial phases of the project (Sudan). The learning curve for reviewers is known to become gradually steeper as experience with the methodology and acquaintance with the record system advances. In some countries (Egypt) a small number of records have been entered to date and may not be reflective of the actual status. Inclusion of some day-only cases and exclusion of some records where adverse events were detected prior to index admission may be additional factors contributing to the low RF1+ rates. In several countries, doctor bias may present a concern (although this reflects mainly on adverse event rates and not on the RF1 rate). For example, doctors reviewing in their own field tend to over-report preventability (personal communication, Dr R McL. Wilson).

6.

NEAR FUTURE PLANS OF ACTION

Country presentations flagged many issues for discussion. Several items were clarified by the facilitators. Consequently, country teams were encouraged to take back lessons learnt from the discussion and sharing of results to apply to near future plans of action. For some country teams, such application involves better management

WHO-EM!HCD/084/E Page 11 of hospitals remaining on the list of proposed selected hospitals. For others where data entry from proposed hospitals has been completed, inclusion of extra hospitals was potentially proposed. The following section provides input on the near future plans of action for the participating country teams.

6.1

Egypt

The Egyptian team has one more hospital to review on their list of previously selected hospitals. The team will ensure accurate randomization because the remaining hospital is the largest and is multidisciplinary. It is therefore important to make sure that all departments are appropriately represented. There will also be more accurate monitoring of the time taken for reviewing and of the overall time spent on different managerial aspects of the project, such as record acquisition. More intense training will be provided for screeners (RFl) and more screeners will be recruited. On-site supervision will be performed to ensure consistency of reviewing and to provide technical support as required and the interval between training and reviewing will be made as short as possible. The element of "sensitivity" rather than "specificity" will be emphasized in the RF1 review phase to ensure that adverse events are not missed. One-day admissions will be excluded. More exhaustive training on determination of preventability will be used and reliability testing will be appropriately enforced.

6.2 Jordan The Jordanian team has one more hospital to review. The team proposed to increase the training of reviewers for criteria (RF1) and adverse event (RF2) detection in accordance with taught rules and regulations. Reliability testing will be monitored and both RF1 and RF2 duplicate reviews will be performed on the same set of records. Data entry will be performed on site and record mobilization will be restricted. Less time will probably be required for reviewing. Finally, mobilization of funds from WHO Regional Office has proved to be a long process. The team has therefore proposed to start the process of requesting funds earlier.

6.3

Morocco

The team will include a private hospital in the study. Nurses accompanied by residents will perform the RF1 review. Medical files will not be moved and files that are rejected or not located will not be counted in the study.

6.4

South Africa

The South African team has proposed the inclusion of two more non-academic hospitals in the study. The team will ensure that only nurses perform RF1 reviews and that one team of nurses and doctors will be mobilized to review both hospitals. Doctors who will be involved in RF2 reviewing will not be involved in RF1 reviewing.

WHO-EM./HCD/O84/E Page 12

6.5 Sudan The team in Sudan proposed to start as soon as is feasible with the next (1 extra) hospital. The rule to be more "sensitive" than specific will be strictly applied to RF1 reviews. Finally, the team has vowed to increase administrative supervision through (on-site) nurse management.

6.6 Tunisia

A general (non-academic) hospital in Central Tunisia has been proposed for inclusion in the project by the Tunisian team. Records are generally known to be of poorer quality in this hospital compared to those previously tested in the project. The team proposed to use nurses for RFl review and to ensure that reviewers (RF1 and RF2) are external (from outside the tested hospital). The inclusion of nurses to replace medical students for RFl reviewing will have to be endorsed by the General Director. Examination of nursing notes will be conducted thoroughly along with the review process.

6.7

Yemen

The team in Yemen has two more hospitals to review before completion of the current research project. The team proposes to electronically randomize their records to ensure true random sampling. They will also exclude one-day admissions and will resolve the problem of Internet connection to ensure faster data entry. Constant communication with NCHI for on-going technical support is proposed.

7.

REPORTING

The reporting back of the data to pertinent recipients in a calculated manner is a critical step in the project. A multitude of parties will be interested in the results that stem out of this study. Each party will require presentation of the data in a different manner and with different levels of detail. For example, individuals in the research field will require a technically detailed presentation method to ensure that the project can be duplicated, whereas stakeholders will require take home messages, conclusions and recommendations. The purpose of this project is to instigate change towards improvement. With that in mind, the data should be presented to each party as efficiently as possible to ensure that the information is appropriately utilized in improvement. Several discussions were conducted on the topic and culminated in a set of recommendations providing the broad guidelines for reporting at different levels.

7.1

National (central) level

The initial national report on the results of the studies on detection of the magnitude and nature of patient harm at health care facilities should target the Ministry of Health (Minister of Health, Undersecretary) and other outstanding national health-concerned authorities, e.g. Medical Syndicate, Health Committee in

WHO-EiWHCDIO84E Page 13 Parliament, etc. A personal communication with the Minister of Health, possibly through an executive summary with brief information, should be followed by delivery of the information to stakeholders (including the Minister). This initial communication with the Minister should include pre-information about the study's results and clarification of its international context. The information vehicle to stakeholders should optimally involve a meeting endorsed and attended by the Minister of Health. It is important that such a meeting be short and succinct with key messages clearly conveyed at the initiation of the presentation. After delivery of the key messages, it is important to entice the Minister and other officials into listening to the remaining aspects of the presentation. It is this key message that the Minister can use and it should be clear and precise. Following the introduction of the main take home message, the presenter should provide the Minister with a solution to work with and reasons why helshe should endorse the study. Presenting suggestions to launch a national event in relation to the results would also strengthen the presentation. The meeting with the stakeholders requires an exceptionally good chairman. It will likely involve politicians, managers, statisticians, etc. A good chairman, who is respected and content-rich, can keep the discussions on track with a carefully-planned agenda. It is essential that the content of the presentation is short with less presentation time and more time for panel discussion and recommendations. The presentation must include figures. The major pieces of data in the presentation should include a figure stating the adverse event rate detected, highlighting a classification of major causes, contributing factors and emphasizing areas of priority. Recommendations and strategies for improvement should be the final outcome of this meeting. Worthy of note, the main message sent through this meeting should revolve around the presented figures, emphasizing that change needs to be driven, but avoiding excessive criticism of the current situation. The desired outcome of this work is to drive change, but not fear. Presenting the resulting data is a subtle art. Intercountry comparisons should be avoided. The use of external experts, who are neutral and content-rich, to provide specific commentary, may add credibility.

7.2 Facility (peripheral) level At the facility level, hospital directors, hospital personnel, clinicians and quality directors need to be informed of the results. In addition to the framework provided to stakeholders (which should include the adverse event rate, major causation categories, areas requiring prioritization), more detail will be required. For example, the facility may need to know about disability and death due to adverse events and figures pertaining to preventability. The rate of adverse events that occur prior to, or during, hospital admissions may be an important item for facilities. Extrapolations, e.g.

WHO-EMIHCD1084IE Page 14 wasted bed days and dollar implications, may be additionally useful. A review flow and a table of patient data followed by review of a few cases would also be useful. The conversation at the level of the facility may have to be different from that with the clinicians. Clinicians are intrinsically competitive and comparison with peer organizations may induce motivation. Also, while collective data may be provided at the facility level, clinicians are probably better presented with individual cases. Individual ward result discussions, on the other hand, may not be feasible in this study. It is strongly recommended, however, that information should be collective at facility level and should be delivered to both clinicians and managers simultaneously to avoid secrecy and to endorse teamwork between the two parties. The material provided may be a one page summary for each hospital or for a group of hospitals and should provide a brief description of the project as well as recommendations. Comparing facilities is best avoided. If, however, it is unavoidable, then it could be accompanied by possible explanation, e.g. that the adverse event rate is lower in one facility due to the predominance of obstetrics admissions.

7.3

Regional and international level: Publishing data

The importance of publishing the resulting data was unequivocally acknowledged. Publication increases credibility of results and is imperative for change. It also helps in prioritization and creates focus on what needs to be improved. The consensus was that for the most part there should not be any need for Ministry of Health approval for publication or that such approval is feasible in most countries. (The team in Yemen pointed out that the Minister of Public Health and Population would have to approve publication, but that there were no foreseeable problems in that regard.) Furthermore, WHO can help communicate with ministers of health if required to facilitate the issue. 7.4

Lessons learnt, challenges and needs

Country teams presented progress to date on the project. For the purpose of this report, country presentations were divided into two major components: project core and lessons learnt, challenges and needs. The first part (project core) was presented in tabular form (Annex 4) for convenience and to enable side-by-side comparison of different country approaches. Individual hospitals will not be identified by name in this report to ensure confidentiality.

8.

RECOMMENDATIONS

To Member States 1. 2.

A deadline should be set for the project within a realistic time frame, Each country team should provide information completing a project flowchart.

WHO-EM/HCD/O84/E Page 15 Alternative tools should be pilot-tested by countries. (Technical support on implementation is to be provided by NCHI). To WHO Regional Ofice

Continuous technical support should be provided via regular teleconferencing

A standardized format for reporting should be generated to ensure consistency of the message sent out by country teams. (NCHI support will help in standardizing the reporting format.) Country teams should be provided with country data on Microsoft Excel sheets after cleaning the data. To the Northenl Centre for Healthcare Improvement

The NCHI should provide a template for reporting to be used by each country for result dissemination. Technical support should be provided by the NCHI to ensure pilot testing of alternative research tools for adverse event determination by country teams. After pilot testing, the NCHI should provide guidance to ensure appropriate implementation at country level. The NCHI should provide each country team with an individualized report, detailing country results. In addition, aggregated results should be discussed with all participating teams in the next meeting. The NCHI should provide guidance on mechanisms of result reporting to various involved parties, including national stakeholders and ministries of health.

WHO-EM/HCD/084/E Page 16

Annex 1

AGENDA 1. Inauguration

2. Opening message 3. Adoption of agenda of the workshop

4. Presentation of the objectives, expected outcomes of the workshop

5. Country presentations on progress made according to previously agreed upon workplans

6. Overall summary of outcomes to date 7. Discussion and agreement on the use and distribution of the results 8. Workshop of draft tools for use when medical records are poor or absent. Details of further support and follow-up 9. Conclusions and recommendations

WHO-EM/HCD/O84/E Page 17

Annex 2

PROGRAMME Monday, 12 February 2007 08:30-09:OO 09:OO-1O:OO Registration Opening session Message of Dr Hussein A. Gezairy, WHO Regional Director for the Eastern Mediterranean Introduction of participants and resource persons Adoption of agenda and programme: Dr Ahmed Abdellatif Welcome speech by Dr Ross Wilson Summary of the experience Country presentations Summary

10:30-12:30 13:30-16:30 16:30-17:OO

Tuesday, 13 February 2007 08:30-13:30 14:OO-15:30 16:OO-17:OO Country presentations continued Presentation of summary of results to date Addressing challenges and needs identified by presentations

Wednesday, 14 February 2007 08:30-10:30 Workshop: How should we use the research results to improve patient safety inside your country and internationally? Practical guidance on data handling, ownership, report generation and publication: next steps towards national capacity-building Plenary sessions on different aspects of the challenge Reporting from plenary sessions and discussion of next steps towards national capacity-building Finalizing agreement on how the results will be used

11:OO-13:OO 13:30-15:OO 15:30-17:OO

WHO-EMIHCDl084IE Page 18

Thursday, 15 February 2007 08:30-10:30 13:30-15:30 15:30 Workshop: New draft tools to measure patient harm for use in datapoor environnlents Agreement to next steps for testing new measurement tools: details on further support and follow-up Close

WHO-EMIHCDlO84E Page 19

Annex 3 LIST OF PARTICIPANTS EGYPT Professor Ossama S.E. Rasslan Professor and Head of Medical Microbiology and Immunology Department Ain Shams University Cairo Dr Atef Badran Clinical data Manager National Cancer Institute (NCI) Cairo Dr Aisha Abou El Fotouh Ain Shams University Cairo Dr Nashwa Nazmi AbdelAziz Ain Shams University Cairo

JORDAN Dr Safa Qusous Quality Director Ministry of Health Amman Dr Mai Rahahleh Head of Quality Monitoring Department Amman Mr Wael Kildani El Nadeem Hospital Madba

WHO-EIWHCD/084/E Page 20

MOROCCO Dr Amina Sahel Chef de Service des Soins Ambulatoires Direction des HBpitaux et des Soins Ambulatoires (DHSA) Ministkre de la Santi Rabat

SUDAN Dr Malik Abdo Ali Focal point for Patient Safety Federal Ministry of Health Khartoum Dr Zahir Mohammed Khier Allah Deputy Director of Hospital Department Curative Medicine Directorate Federal Ministry of Health Khartoum Dr Sara Hassan Mosatafa Planning and Research Office Federal Ministry of Health Khartoum Dr Alkhansaa Awad Elkarim Elias Federal Ministry of Health Khartoum

TUNISIA Professor Mohamed Salah Ben Ammar Professor of Anesthesia and Intensive Care CHV-Mougi Slim Tunis Professor Mondher Letaeif Preventive Medicine and Epidemiology unit University Hospital Tunis

WHO-EM/HCD/084/E Page 21

YEMEN Dr Adel Al-Moayed Director General Hajja General Hospital Sana'a Mr Nasr Ali Ahmed Head of Continuing Studies Nursing Department Ministry of Public Health and Population Sana'a

Regional Office for Africa SOUTH AFRICA Professor Stuart Whittaker Managing Director Council for Health Service Accreditation of South Africa (COHSASA) Dr Saskia Blakeway Programme Manager Council for Health Service Accreditation of South Africa (COHSASA)

KENYA Dr Janet Butage Medical records officer Member of Africa Advisory Committee on Health Research and Development and the focal point for patient safety research Regional Office for Europe SPAIN Ram6n Lim6n Ramirez Oficina de Planificaci6n Sanitaria y Calidad Agencia de Calidad Depacho 3043 Ministerio de Sanidad y Consumo Madrid

WHO-EM/HCD/084/E Page 22 Jeslis Aranaz Andr6s Departamento de Salud Phblica Facultad de Medicina Universidad Miguel Henandez Campus de San Juna Ctra. Valencia-Alicante, s/n Madrid Carlos Aibar Remon Servicio de Medicina Preventina Hospital Clinico Lozana Blesa Madrid

Short-term Consultants AUSTRALIA Dr Ross Wilson Director Center for Healthcare Improvement Royal North Coast Hospital St Leonards

UNITED KINGDOM Ms Sisse Bjom Olsen MRCS Clinical Research Fellow Department of Biosurgery and Surgical Technology Faculty of Medicine Imperial College London London

FRANCE Professor Phillip Michel Director, Health Evaluation Strategy Public Health Programme and Disease management French National Authority for Health

WHO headquarters Dr Itziar Lanzgoitia Jauregui Research Lead, Patient Safety Secretariat WHO World Alliance for Patient Safety WHOEIQ Geneva

WHO-EMlHCDl084E Page 23

WHO Secretariat Dr Ahmed Abdellatif, Coordinator Health Systems and Regional Adviser, Health Care Delivery, WHOEMRO Dr Riham Elasady, Technical OfficerIHCD, WHOIEMRO Dr Ingy Khourshid, Technical OfficerIHCD, WHOIEMRO Mr Kareem Elhadary, Technical Support, WHOEMRO Mrs Hoda Elsabbahy, Secretary, HCDDHS, WHOEMRO Mrs Naglaa Salah, Secretary, HECDHS, WHOEMRO

Table 1. Comparison of country approaches South Africa Local leadership Permission to conduct the study obtained from the Western Cape Provincial Health Service and the National Department of Health

Sudan Project Management stationed within MOH in close proximity to policy-makers

Yemen Several components have been made aware of project: Deputy Minister for the Curative medicine, Member of the Executive board of GCC, General director of the medical services, General directors of health offices in the governorates, and the targeted hospital Directors, Research Center, Quality office MOH

Tunisia MOH, General Director, Quality Department approached Permission to conduct the study was taken from the MOH Steering committee: MOH, General Director, Quality Unit responsible, Hospital directors, Head of Medical Committee, Established communication system through which local leadership are regularly briefed on the progress of the studies

Briefing meetings with the Minister of Health and

Egypt Communication with MOH, and First Undersecretary of MOH (the President of steering committee) General Directors of selected hospitals Professor of Public health Dept. in Universities Notifying the Minister of Health and Population

Morocco Minister of Health is made aware of the project and is continuously briefed on progress. The project was presented at the level of 2 major universities (2 meetings in Fes and 2 in Rabat)

Jordan The minister of health is the head of the steering committee (general secretary, assistant for hospitals, director of one hospital, chief of nursing speciality. director of international and public relation directorate, director of Internal auditing unit)

EC approval of study protocol

try approaches (cc lt.) Sudan National group Health Steering committee headed by Undersecretary

Yemen Deputy Minister is the chairperson of the Steering Committee. G. directorate for medical services. Patient safety progress and key universities

Tunisia General Health director Medical Committee of both hospitals Hospital managers

Egypt Minister of Health and Population

Morocco Steering :ommittee :omposed of Directors of Hospitals and ~mbulatory services at the MOH and iirectors of university hospitals Letter was sent by MOH to iirectors of selected hospitals for support.

Jordan Project management team within the quality directorate at the Ministry of health, expert panel composed of 12 consultants

First undersecretary of MOH Hospital Directors

Hospital engagement

Obtained by negotiations between COHSASA and hosp management

Orientation meeting with the Higher Council of Hospitals:Hospital directors, heads of clinical specialities, senior clinicians. Presentation in the "Patient Safety Conference": Directors of public, university and armed forced hospitals, clinicians, journalists, civil organizations

Targeted hospital directors approached, made aware of project and invited for participation.

Formal correspondence to Hospital Directors to facilitate and conduct the PS study

Through personal communication, arranged meetings with selected hospital directors Formal letter was generated to the hospitals signed by the First undersecretary. Site meeting in the selected hospital with quality control staff and head of medical Dept. and medical records personnel ... .. . . . ..... . . ... .. . ....... .

Meetings with hospital directors Meeting and training of heads of medical records departments. Continuous updating on the progress and findings to the minister and the general secretary

... . . . . ... . . . . . . . . ...

ltry approaches (cc nt.) Sudan Yemen

Tunisia

/ Egypt

Morocco

Jordan national day of quality

Directors approached by project manager Medical record acquisition Hospitals set room aside for audits Record staff provided required records and requested their return by the end of the day. Hospital selection criteria Convenient location Excess of 200 beds Cooperative, interested staff Reasonable records Two nurses from the team worked with the statistic hospital departments to pull the selected files Records were kept in individual departments: no central archive By statistician in Rabat and Professor of Public Health in Fes

Large fed hospitals Input from all over the country Not the best but leading hospitals

Wide geographic distribution Large hospitals General

Representative of North and central Tunisia Multiple activities Previous cooperation in terms of quality assessment initiatives as well as non formal willingness to carry on the patient safety study ~~~~~.~ ~~~~~

Geographically inside Cairo for the pilot project. Cooperative hospital managers Available medical records General hospital with different specialities Then three hospitals were chosen:

The availability of data on medical files Teaching hospitals (should he role models for district hospitals) Involvement of quality focal point with selected hospitals.

Good reputation. Good medical record keeping system. Cooperative hospital directors Middle region (three governorates)

Table 1. Comparison of country approaches (cont.) Experience and success of implementing maternal mortality surveillance system Good medical records Ministry of Health hospital University teaching hospital Private hospital

WHO-EMlHCDl084IE Page 28

4 pub 3 pub 129 130 811 1050 no obigy, neur, ped, Gen Gen 12000 18000 6395 1 17228 are nbs 15254 Gen 6332nhs 10731 Gen 4935 9368

91

Gen 3558 ohs

9368

2 pub 1 pubiteach

2 univlpub ophthOLR -

1 univi Pub 400 115 3

V

teach 1 pub1 teach

I I

I

I

906

Gen

44161

Table 3. Record classifications South Africa Sudan Yemen Tunisia Egypt Morocco Jordan

Completeoess Adequacy to answer Qs No of records

I

700 selected 500 used True randomization employed

800 sampled All entered in database* Systematic random sample Discharge log used as sampling frame**

systematic random

department had its own separate record archive system. Records were weighted according to de~artment

500 from each hospital Systematic random sampling Sampled a little over 500 (534 from H2,507 from H2, got 500 complete records

Systematic Random Sampling (600-650 records) from one main list of recurds =

Systematic Randoin Sanlplilig 800 records Z

ICD 10 coding

I Not used

/ Not used

I

I

I

I

I

Reviewing scale 1-10 10 = Excellent

WHO-EM/HCD/084/E Page 30

WHO-EM/HCDIO84IE Page 3 1

Table 5. W2flr !viewers South Africa Sudan Recommended by the Hospital Administration Directorate Yemen As for RFI reviewers and working in a variety of specialties Tunisia Academic doctors with vast experience in clinical practice from the SAME hospitals 4 sessions

Recruitment

#

70HSASA taff

Egypt Ti me dedication At least 10 years background

Morocco Interest in patient safety Time availability Competence

Jordan [nterest availability Physicia~~ recruitment Found to be difficult With nurses

Training

1 day session ~tCOHSASA

2 3-day training workshops Nurses separated from docs

13-day training workshop for nurses and doctors' 4 doctors

2-day lectures followed by assisted field work I surgeon and 1 internist

2 days

Background and number

L gynecologist,

L surgeon

5 doctors One internist,One surgeon,Two Paediatricians,On e Obstetrician Worked separately according to availability

Surgeon, GP. internist

2 doctors per site

2 groups of 4 senior doctors each Neurosurg, medicine, Intensive care, emergency med, surgery

2 doctors per slte Two lnternlst ,two surgeons, one anaesthes~as ,one ObsIGyn Duplicate reviews on 175 and 9 3 records for RF1 and RF2 respectively

Quality contra

1/10RFI and iF2 records luplicate .eviewed with ligh degree of :oncordance

For quality control every 10th record is double reviewed

Table 6. Data classification South A f r i c a Ease of finding information in medical records Ease of using reviewing tools Confidence of data in records Co~ifideoce of data presented by reviewers RFI

Sudan RFI

Yemen

Tunisia

RF2 6

RF2 9

RF1 H1=9

RF2 HI=6

RFI 8

RF2

Egypt RF1 RF2 8 8

Morocco

RFI 9

RF2

Jordan RF2 RF1 9-10 8

7

9

7

7"

H2=8 9

H2=5 8 RFI

9

9

7

9

8

9

8

9

7

10

9

8

7

8

9

8

9

7

8

8

9

Records scale 1-10 1 0 =Excellent * Sudan: For first 2 hospitals 800 records were samplcd and all were entered in database. For the rcmaining hospitals, some records have been discarded due to deficient infunnation. ** Sudan: In some hospitals, different discharge books were available for different specialties, but a good case mix was obtained by the team. 'Sudan: Some hospitals offered reviewers financial incentives to perform the work, thus helping in the process of recruitment. ' Yemen: Training workshop was followed immediately by field reviewing. Yemen: Extra incentives were given for travel. Yemen: 700 samples were selected for review, 500 were actually taken, reviewed and recorded. Any incomplete/ missing files (10-20) were replaced from the remaining 200 selected. The rest of the 200 files were not used. a ~ u n i s i aAlthough : different records were kept in individual departments, a systematic random sample approach was used to select records from EACH dcparttnent separately in the interest of randomization. Tunisia: Med, students who performed the RFI review may have attended tested hospitals, but were not affiliated with them at the time of review. Also, med students were NOT involved in data entry. ee Egypt: Hospital 2 had 6500 records and had a computerized admission data system. However, mortality records and records for economic (paying) patients were kept separately. Medical records were good as hospital is included in quality control hospitals project. Hospital 3 had an Electronic archiving system. Files wcrc well organized and almost complete. 507 records were sclcctcd as 7 represented one day hospital stays.

"

NB. Completion of record review for the first site (HZ) took 2 weeks by the Egyptian team. = Morocco: Einergency records were not registered in the central archive. ObIGyn department archived its records by pathology. Once a selected rccord is identified as missing. the team would go a n to the record that follows it chronologically. If that too is not found, the following rccord on thc admission log is pursued elc. In some cases, up to 10 records in a row would be pursued before anc is found for selection and reviewing. One hospital was in the process of moving to a new location, which may explain why so many folders were missing. Below is a flowchart describing record sampling and acquisition by the Moroccan team in each of thc 2 selected hospitals. Y Morocco: Nurses who were not viewed as highly competent in the review process were assisted by others and had to work in pairs. Thus, a reviewer ID on the data entry form may take the form : 432, indicating the reviewers # 4.3, and 2 reviewed the record. NB. It was noted that the RFI+ rato was higher in hospitals reviewed by nurses compared to that reviewed by a mix of nurses and residents. N.B. 2 Morocco was thc only counrry where data ently was panially performed on site. X X Morocco: Some details could not be easily found in files, c.g. contributory factors, patient journey before hospitalization. RFI reviewing became easier once the rule " 1 is bettcr to be more sensitive than specific" was employed. That is, an RF1 screener is bettcr off scoring positive a criterion on RFI if in doubt. The file will thcn undergo further scrutiny during RP2 review. This ensures that no AE are missed. Judgment of causation vs. preventability a n RF2 was not an easy task. It was also frequenlly difficult to judge thc degree of disability caused by an AE. X Jordan 79 medical records were missing and replaced. 69 medical records with inadequate and incomplete data were replaced too. 175 and 93 records were dupliantc reviewed for reliability testing for RFI and RF2 reviews respectively.

z 0 %

<

2 W +

1 0

? s 2

WHO-EM/HCD/084/E Page 35

Annex 5 DATA ENTRY The expert training team had provided participants with login access to a database in which country data was entered. The entered data was analysed at the Northern Centre for Healthcare Improvement (Sydney, Australia). Technical support was provided during the second training workshop at WHO Regional Office in May, 2006, and was followed by ongoing support by expert trainers via field visits and teleconferences to facilitate data entry. The Internet-based database was designed to work with both DSL and dial-up Internet connections. It was also designed to retain most information if the connection is lost and to accurately mirror W1 and RF2 forms used in the current studies. The system is extremely secure. The data is retained on a special server in a firewall. Three dedicated servers are separately firewalled to avoid hacking and to ensure security. The system was designed so that data can be entered on-site, a process that has not been feasible for most countries due to time constraints and/or Internet access. Finally, the system has been provided by a backup error prevention system via several forcing functions. While there is a consensus that the data entry system has delivered most of the goals it was designed to deliver, however, some concerns about data entry were raised during the current meeting and are highlighted below. If the Internet connection is lost during login (after entering login, password and country code), an error message pops up on re-logging requesting that the country code is re-entered. Thus, although the initially entered country code may be retained on the screen, it has to be reentered (Figure 1).

Figure 1. Country code error message When creating a new record, the user is required to enter the medical record number and the reviewer ID number, which are located on separate pages on the paper review forms. This is an inconvenience and results in some time loss. Also, the user is prompted to search for a

WHO-EM/HCD/084/E Page 36 record using either the family name or the case ID, family name, however, is not actually used in the database to locate the files. When creating a new record, the search drop down list is empty. Thus one cannot tell if the record number has been already used until a duplicate record error message appears (Figure 2). Also, an individual entering data cannot search for previous files. (N.B. This concern has already been resolved by the NCHI support team).

Figure 2. Creating a new medical record When a new record is created, a warning bar pops up and can only be removed if one clicks OK (Figure 3).

WHO-EM/HCD/O84/E Page 37

Figure 3. Warning bar pop-up

WHO-EMlHCD1084E Page 38

The size of the text on the interface is rather small. On zooming in, a large part of the screen is lost (Figures 4 a and b).

Figure 4 a. Drop down menu for zoom

WHO-EM/HCD/O84/E Page 39

Paticnt Fin1 N

Dale of .\i'irii\,.i:tit: Date af Re\*iew:

:

owing clc~nenrs present'?

Figure 4 b. Case ID screen The order of the fields on the data entry form is different from that on the paper forms, thus creating some confusion (Figure 5). Many medical abbreviations that could only be identified by doctors are used, e.g. DM (diabetes mellitus), IHD (ischemic heart disease). This restricts the process of data entry to doctors or highly qualified nurses. It often takes time for the hand tool to change to a pointer (Figure 6). Such time is precious when entering a large number of records.

WHO-EMIHCDI084IE Page 40

Figure 5. Order of fields on data entry form

Figure 6. Choosing tools on screen

WHO-EM/HCD/084/E Page 41 ICD-10 coding is used in the data entry system. Many physicians are not familiar with the coding system and may inadvertently choose the wrong code (Figure 7).

Figure 7. ICD coding In Section B of RF2, where adverse event determination is to be judged, the questionnaire should stop if no adverse event is determined. However, the software allows the access to the following screen despite that the message to stop the questionnaire is clearly stated (Figure 8).

WHO-EIWHCDI084IE Page 42

Figure 8. Adverse event determination If a new record is to be created after completion of data entry for one case, the individual entering the data has to go back through previous screens in order to access a new form. Determination of an AE is contingent upon detection of injury, disability and causation by health care management. It might be more convenient to have all three items on one web page (Figure 9).

WHO-EM/HCD/084/E Page 43

Figure 9. Adverse event determination error

If the answer to the occurrence of an injury on RF2 is "NO" (as in Figure 5.8), the system does not allow for the creation of a new RF1 for a new record. If no injury is determined, in which case the questionnaire should automatically stop, one can still go on to answer the question on disability (that normally follows the question on injury) (Figure 10). Reliability testing by duplicate review of a medical record could only be allowed if the reviewer ID number is different from that of the individual who initially reviewed the record. This, however, is an intended forcing function enforced by the data entry software to ensure that the same record is only duplicate reviewed by a second reviewer.

WHO-EM/HCD/084/E Page 44

Figure 10. Question on disability A concern was raised that patient name has to be identified by entry on the first page of the data entry form. Participants were reassured, however, that this information is not stored in the database after data cleaning. It is required to ensure that each patient is uniquely identified and will consequently be destroyed.

Once there is disagreement between RF1 and RF2 reviewer on the criteria in a file, the data entry process will automatically stop. To avoid this, some teams had to inappropriately indicate agreement of RF2 reviewer with criteria detected by RFI reviewer to enable further data entry. (NB This concern has already been resolved by the NCHI support team). Once a file has been completed, the file cannot be re-entered. This was originally designed to protect against over-writing. Access, however, can be requested by contacting NCHI support team who have a trail of passwords and thus know who accessed the record. When entering a new record with the same medical record number (MRN) and reviewer number (RN) but a different hospital number, an error message appears warning that the record cannot be saved because the MRN and RN already exist. If no discharge date is written on the medical record, a function in the software will not allow continuation of the data entry unless a discharge date is filled. (This can be resolved by placing in the "discharge date" field the last date seen on the nursing notes in the file followed

WHO-EM/HCD/084/E Page 45 by an explanation in the text field of the fact that the discharge date was missing and is thus not accurate. If there is absolutely no indication of the discharge date, the admission date can be put in the "discharge date" field. However, the whole record will be taken out of the system in that event because it will be regarded as a day-only admission.)

WHO-EM/HCD/084/E Page 46

Annex 6 REQUESTED FORMATS FOR DATA PRESENTATION Based on a review of the Quality of Australian Healthcare study1,the following formats for data presentation were requested by country teams (to be provided by NCHI support team). 1. Table showing the percentage of medical records positive for each of the 18 criteria (RF1) and its odds ratio for association with an adverse event (AE) (Table 1). Table describing adequacy of medical records (nursing notes, medical progress notes, procedure documentation etc.) (p. 464). Comparison of study patients (e.g. mean age, gender distribution, private hospital admission, etc.) (Table 2). Percentage of AE rated by preventability (no, high and low) (Table 3). Bed-days related to AE on index admission (Table 4) The correlation between age and gender with AE and categories of disability (Table 5). Table 6 (Admissions by MDC) may not not be feasible because it requires the entry of MDC which is not used by all country teams. AE by specialty of attribution (Table 7) AE by clinical category (Table 8) AE by location of occurrence (Table 9). Table 10 (areas for effort to prevent recurrence) may not be provided because it might be misleading. Clinical investigation may be required before the table can be drawn to ensure accuracy of conclusions. Other tables will be provided if supported by data entered Inclusion of tabulated data on the timing of the AE in relation to IA was suggested (Sudan). Inclusion of data on AE that occurred in an outpatient setting as well as AE which occurred during IA was suggested (Morocco).

2.

3.

4. 5. 6.

7.

8. 9. 10. 11.

12. 13.

14.

WHO-EM/HCD/084/E Page 47

15.

A list of AE that were rated Preventability 6 and Causation 6 (i.e. those that could have easily been avoided and are unequivocally the result of health care management) was suggested (Dr R McL Wilson).

Each country team is to provide information by completing a project flowchart (Figure 1).

Figure 1. Flowchart for country information

WHO-EIWHCD1084E Page 48 Annex 7 ALTERNATIVE RESEARCH METHODS 1.

SOURCES OF PATIENT HARM Data indicative of patient harm may be derived from various sources (Figure 1). Large data sets which are not always available and are not an option in many countries. Reporting systems including complaints, sentinel reports, litigation proceedings. Retrospective record review which is currently the only validated tool. Although not sophisticated, it represents the best available and most rigorously tested tool. Observation/interview which is important but underdevelo~ed.

Sources of Patient Harm Data Medical Record Review

k"S

i r f f

-'

Observationlinterview

Figure 1. Sources of patient harm data Medical record review (MRR) has been the gold standard for measuring adverse events despite its limitations. For example, causal factors are difficult to obtain by MRR. Thus, consideration of alternative research methods is important where medical records are lacking or inefficient. Also, alternative research tools are of utmost significance if more information is required than mere prevalence. This section will overview alternative methodology and will focus on some methods which could be applied in data-poor environments.

WHO-EMIHCDl084E Page 49 The following strategies will be considered: combining medical record review with structured interview with clinical staff; using nominal group methods to obtain information from a clinical focus group; using observation of a clinical interaction against agreed standards.

1.1

Reporting systems Among its major disadvantages, voluntary anonymous reporting: does not give prevalence or incidence data and hence does not allow prioritization of improvement activities; requires new infrastructure; is heavily dependent on organizational culture (requires a culture of willingness to promote openness); is heavily biased toward errors of commission with a very short latent interval (where the effect of the AE is seen within a short time interval from the occurrence of the AE).

WHO has drafted guidelines for adverse event reporting and learning systems.4 A set of these document have been generated through the World Alliance for Patient Safety under the guidance of Dr Lucian Leape. These documents represent an attempt to work out where the reporting system lives and what should be done in the field of patient safety, highlighting the current state of knowledge that needs to be tested. A study performed in the UK in 2005 (Shaw R et all5 analysed approximately 30 000 reports. The study demonstrated that reporting systems may not necessarily be accurately reflective of medical record reviewing. For example, the latter study demonstrated that over 40% of reported adverse events (AE) were slips, trips and falls, which are underestimated in the record documentation. This study highlighted that reporting and investigational systems go hand-in-hand to provide an accurate perception of AE occurrence. The Australian study performed in 1995 (Wilson, R et al.)' demonstrated that the majority (-40%) of AE were detected in association with operations, rather than, for example, in the area of slips, trips and falls (1.7%). The findings of the latter study explicitly demonstrated that the data will determine the aspects of patient safety that require attention. Furthermore, the study highlighted that simple basic fundamental medical issues (such as surgery-related AE) are superior to isolated events in order of priority and warrant attention. Thus, MRR is about harm, rather than an event. Indeed, a slip may be a reported event, but may not have resulted in harm and would thus probably not be documented by the MRR method.

1.1.I Cultural constraints The main limitations to the application of reporting systems in countries of the Eastern Mediterranean Region is that one will need to ensure that there is no conflict of interests, e.g. a nurse may fail to report an incident committed by her direct boss. Also, there has to be a forcing function that ensures that reported events are genuine and do not represent, for example, personal issues.

WHO-EM/HCD/084/E Page 50

1.2 Interview only The interview method is structured with a single person (usually the nurse and requires about 30 minutes) or may involve multiple individuals. 2-5 different interviews may be required if one wants to examine AE in a whole ward. Interview may also involve a focus group. In the latter scenario, about 10 people from one clinical area are pulled together for a 90 minute interview and are asked to describe an AE that they know about using the nominal group technique. The group is usually comprised of a mixture of doctors and nurses from 1 or 2 wards. There are no review forms involved in this technique (RF1 and RF2) and it is not structured. It can therefore be used where medical records are inefficient. It usually involves a discussion of an event, ideally a relatively recent one, its causes and prevention strategies. The moderator of this discussion should not be involved in the ward where the discussion is centred on and should be involved in patient safety. If the moderator is from the same hospital as the group, it is preferable that he does not hold a managerial position. The aims of this technique are to find out about adverse events (descriptive) and to use nominal group techniques to prioritize on causes of harm. The quality of the information that comes out of the latter technique is dependent upon the competence of the participants. Thus if they possess a good fundamental knowledge, there will be good input. The technique generally involves an introduction by the moderator, a silent brainstorming session, followed by discussions (possibly using a whiteboard or flipchart to generate a priority list). The output of this technique is a prioritized list or a pareto chart. The intangible benefit of this technique is that it brings people together to discuss how improvement can be made. In addition, this method is quite cheap. A matrix can be established using this method, e.g. the Veteran's Administration matrix to detect the size of harm and the likelihood of it happening. Disadvantages: Although simple in idea, this technique is usually complicated in practice. It usually depends on the skill in conducting the technique, although it is not too difficult to train for. The technique works well in some clinical fields but may be more complicated in others. One of its main limitations is that this method does not involve following of individual patients. Also, issues may come out of the discussion that may be related to other wards, not represented in the group. Finally, this method cannot be used to calculate the number of adverse events. 1.2.1 Cultural constraints The main limitation of application of the nominal group technique in countries of the Eastern Mediterranean Region seems to be getting doctors to participate, setting aside defensiveness and sensitivities. Certain groups, particularly nurses, commonly considered inferior to doctors in many countries, may feel reluctant to voice their thoughts. Finally, it would be more convenient (especially for nurses) to conduct the discussion in Arabic. Nonetheless, there is a consensus among participating countries that this technique is feasible with the right moderator and an appropriate, non-punitive approach.

WHO-EM/HCD/084/E Page 51

1.3 Staff interview: The French experience This method was developed in France in 2002 by the French national working group in charge of defining the methodology of a national survey. Because research on AE is now leaning towards more analytical and evaluative goals, it was decided to move away from the traditional medical record review and to base data collection on both the information declared by the ward staff and that collected from the notes with the ultimate goal of not just describing adverse events, but also introducing the potential to take action in response to such events. Its objectives are to: increase validity of AE assessment; better investigate the main contributory factors; identify types of AE rarely recorded, such as pain. 1.3.1 Principles This method is thought to be the only method which efficiently provides epidemiological estimates on all AE in data-poor environments. It shares many similarities with record review, e.g. external investigators; two-phase data collection; similar forms. Sources of information, however, are the main difference (interview with the ward staff, complemented by review of notes). The investigators (nurse and doctor) should preferably be external to the hospital, never from the participating ward. A good ability to explain and conduct the interview is required along with a good understanding of the hospital organization.

WHO-EM/HCD/084/E Page 52

1.3.2 Design There are three possible designs for this study:

. . . .

cross-sectional data collection, one given day assessment of AE in patients present the day of review; longitudinal data collection prospective follow-up of occurrence of AE in patients present during a given period (seven days in the French survey), retrospective data collection for patients recently discharged (the week before in the Spanish study).

1.3.3 Objectives of the French pilot study To compare epidemiological methods fop estimating risk of AE (Figure 2): cross-sectional (data gathered in one day, prevalence data); prospective (data gathered during hospital stay); traditional record review (looking back after patient discharge). On effectiveness, reliability, cost and acceptability criteria. 1.3.4 Methodology A representative sample (involving 7 hospitals, 37 units and 800 patients) was used in the French study. Independent assessment of the three methods applied to one sample was employed. For the longitudinal and cross-sectional study, a nurse interviewed the head nurse and explained the objectives of the study. The nurse investigator then interviewed the head nurse on all criteria (18 criteria structured similar to the record review method). At the end of the interview, which took about 3 hours for 30 cases, a list of criteria +ve and -ve patient cases was generated. The list was then sent out to a doctor for examination along with corresponding medical records. The examining (investigator) doctor then interviewed the doctor responsible for each case using W 2 criteria (similar to form used in MRR). Interviewing the doctor serves the purpose of finding about AE that are not in the record and helps in obtaining more information about AE.

WHO-EM!HCD/O84/E Page 53

,

.....

Patient's hospital stay

7

Time

I

?

Michel, P. et al. BMJ 2004;328:199

Figure 2. Epidemiologic methods used in French study

1.4

Main outcome measures

The main outcome measures were the proportion of cases (patients with at least one adverse event) identifiedby each method compared with a reference list of cases confirmed by ward staff and the proportion of preventable cases (patients with at least one preventable adverse event). Effectiveness of the methods was determined by the proportion of cases identified in relation to a reference list. Reliability of the prospective method was assessed from cases detected on the first day of the study and confirmed twice-by the doctors performing the cross sectional and prospective methods. 1.4.1 Obstacles met

Doctors are not always cooperative in this study. They do tend to be comforted once ensured of confidentiality and that this study is non-punitive. Doctors also do not always understand the definition of adverse events. Sometimes, the major controversy is around preventability. The final decision, however, always belongs to the investigating doctor. The main disadvantage of the cross-sectional method is that it does not allow information about prolonged hospital stay to be extracted. Disability has to be assessed with the treating doctor. More information is obtained if the longitudinal study is used. 1.5

Results and conclusions Medical record review was found to be appropriate for estimating AE rate.

WHO-EM/HCD/084/E Page 54 Prospective design is: more effective and reliable; higher face validity; acceptable increase of cost; acceptable workload for hospital house staff. The interview data was found to be more credible to the staff because they were involved in AE determination. There is no single gold standard method for AE determination. More AE in medicine were detected using the prospective method (Figure 3), while more are detected in surgery using the retrospective method.

15 units

278 patients

In brackets: Number of cases having occurred after the first day of data collection (thus non identifiable by the cross-sectional method

Figure 3. Results of AE detection in medicine in the French study using different research methodologies

1.6 Proposal: Cross-sectional study on AE detection using a cross-sectional interview method in selected hospitals in the Eastern Mediterranean and African Region Rationale: It is thought to be the only one which efficiently provides epidemiological estimates on all AE in data-poor environments. For logistical reasons, we propose the crosssectional design. Objective: to study AE using the information gathered by an investigator from the health care setting staff directly in charge of the patients. Outline

Cross-sectional data collection One single investigator - Reviews existing notes

WHO-EMIHCDIO84lE Page 55 Interviews nurseldoctor Analyses all AE present in patients hospitalized the day of data collection Screening -

The investigator interviews the head nurse and consults the patient's medical records Expected results: prevalence estimate of AE; distribution of types of events; their preventability; the main contributory factors. Pilot testing: will be necessary to determine acceptability, validity and reliability. Ideally, it should be performed in 10 wards in district and local hospitalsldispensaries. One investigator should suffice and, if possible, comparison with record review could be done.

Feasibility When asked on the question of feasibility of application of a cross-sectional interview study in the Region, most country teams predicted that it would be feasible. Some countries (Sudan) proposed that a secondary, rather than a teaching, hospital could be used for that purpose while for other countries (Tunisia) resistance on the part of doctors was anticipated. In some cases (Yemen), acceptability was thought to vary from hospital to hospital and from doctor to doctor. A reward system was suggested (Egypt) for participation in the study.

A summary of the advantages and disadvantages of alternative research tools for the French study and in general are provided below.

WHO-EM/HCD/084/E Page 56 1.6.1 Advantages and disadvantages of three methods used to estimate adverse event rates

Prospective method (data collected during hospital stay) Advantages

Best effectiveness for identifying preventable events Good reliability of judgment of iatrogenic nature of events Staff sufficiently involved to understandnotion of iatrogenic risk and search for causes Preferred because of their pedagogical and communicative virtues Good appreciation of chain of events and their consequences Possible role as "red flag" for care providers during data collection Disadvantages Most expensive Heaviest workload, although perceived as acceptable: Several visits for investigators Staff must be available

Cross-sectional method (data gathered on given day) Advantages

Least expensive Seamless continuation of former methodological approaches to iatrogenic risk Methodological approach fully understoodby professionals and appreciated because it is rapid and easily renewed May be sufficient to justify implementation of a risk reduction policy and to define priorities Good reliability of judgment of iatrogenic nature of events; possible role as "red flag" for care providers during data collection Disadvantages Consequences of lack of follow up during patient's hospital stay: Lowest effectiveness Lack of validity due to measurement errors (false positives and false negatives) Prevalence biased by underestimation of frequency, particularly of deaths, and by overrepresentation of short stays Believed by unit staff to involve an excessive workload for obtaining imprecise estimations Inadequate to serve as initial estimation when evaluating impact of risk reduction policy

WHO-EM/HCD/084/E Page 57

Retrospective (review of medical records) Advantages Good effectiveness, even superior in surgery for estimating adverse event incidence Almost no workload for staff Data collection easily planned Method sometimes favoured by surgical teams and centres Disadvantages Difficulty to judge iatrogenic and preventable nature on basis of sometimes piecemeal data. Therefore: Measurement errors due to quality of medical records and to lesser reliability of judgment of iatrogenic nature Underestimation of preventable events Lower face validity of results, especially for preventability judgment (no involvement of staff. Table 1 shows the strengths and limitations of alternative methods.

Table 1. Strengths and limitations of alternative methods Logistics, needs Cost Availability Study of records time needed Trailling Information in of facilitators participating settings Culture Ethics Finding solutions factors

Record review

+++ +++

+++ ++ +

++

+ +++ ++

+ +++ +++

Staff interview Nomilla1 group Observation

+++ +++

++ + +

++ +

+++ + +

+++ + +++

++ ++ +

++ +++ In limited care processes

+ ++

+ ++

+

+

++

+++

Source: Di- Philippe Michel

WHO-EMlHCDl084E Page 59

1.7

Observation

This technique is generally useful for a defined clinical process where there are already agreed standards for the process, e.g. injection, blood transfusion, IV cannulation, delivery, medication administration. Obtaining the data is a low cost process and low technology and training needs are required. The technique is relatively simple. Also, records are not required and the output data can be qualitative and quantitative. This methodology is usually well tolerated by staff and the general impression is that it is about improvement rather than judgement. The main disadvantages of this methodology, however, include the fact that analysis can be challenging. This technique tells about a process error, but does not measure harm and there is no patient follow up. Most of the errors encountered are near misses. Worthy of note is that the observer should preferably be from the same field as the observed individual. It is also important that the observer does not initiate a discussion with the staff member under observation on why a procedure (or part of a procedure) was or was not done. The final outcome of observation is usually a report with aggregated results. A nonjudgmental conversation can then be conducted the end result of which are possible solutions.

1.7.1

Cultural constraints

There is a general consensus that observation is a relatively feasible methodology. One concern raised was that in many hospitals, there are no fixed policieslprocedures in place to use as standards in general procedures. In such case, one should measure against external standards and use this as an improvement tool. Another concern was that this methodology may not be applicable to many procedures, e.g. surgical procedures. Indeed, this methodology is not suitable for cognitive procedures. Otherwise, this technique has been scored as "feasible" for the most part in all participating countries as long as it is manageable and the process is clearly defined.

1.8

Incident Monitoring (IM) System

To ensure confidentiality of the study there will be no reference to specific results and data sources will not be identified. In addition to the MRR project, the South African team has been working on an incident reporting system to determine its advantages relative to MRR and to promote the combination of more than one methodology for AE determination The main aim in implementing the I M and response programme in South Africa were: to review the capacity of IM; identify incidents as they happen, it is necessary to: determine which have to be dealt with urgently, deal with them and prevent them from happening again;

WHO-EM/HCD/084/E Page 60 identify the less serious incidents that have the potential to seriously harm patients and work towards understanding their underlying causes so that they can be prevented. 1.8.1 Structure

An IM programme was implemented in an under-resourced hospital with high patient load in a poor area with high HIV+ prevalence and trauma (violence related) rates. A computerized system for monitoring, analysing, reporting and managing problems ranging from near misses to sentinel events across the entire spectrum of health care (Advanced Incident Management System (AIMS)) was used. During the first phase of implementation: a call centre was established; staff were trained to understand incidents and to use the call centre to report as many incidents as possible; the call centre recorded and classified the reported incidents according to the AIMS Health care Incident Types (HITS) classification; COHSASA analysed the incidents and reported them to the hospital. Staff then reported incidents as they occurred to the COHSASA call centre. Contributing factors, health care outcomes for the subject, as well as consequences for the organization were captured through the system. If appropriate, this was followed by the hospital management team calling the staff with whom the incident occurred for investigation and possible root cause analysis guided by AIMS. The severity of the incidents was also classified during the reporting call, according to the following severity ratings. extreme risk: immediate action required through a root cause analysis; major risk: senior management attention needed - a root cause analysis will be undertaken at the discretion of management; moderate risk: management responsibility must be specified eg aggregated data and then undertake a practice improvement project; minor risk: managed by routine practice. Advantages of the system include: single point of entry for all departments; staff are guided through a series of simple questions by highly trained interviewers using a process that takes 7 to 10 minutes to capture detailed data specific to the incident; reporting facilities do not have to acquire and maintain complicated computer systems and software, nor the staff to run them;

WHO-EM/HCD/084/E Page 6 1 incidents are immediately sent to the hospital where they are addressed by staff assigned to patient safety duties. The hospital has access to the database and uses it as an aid to address the incident; the call centre provides summary reports of all incidents that occurred during the previous week to the hospital; in addition, immediate action is taken in response to catastrophic events.

1.8.2

Results

The main findings of the initial phase include that the system was quite efficient at picking up incidents that may not have been picked up by other methodologies. In addition, the study team concluded that the methodology was not time or effort-consuming and engaged the hospital management team. In the second phase of the study, currently in the phase of implementation, the COHSASA quality improvement and accreditation program is being used to improve systems and processes in the hospital's services and departments. Thus, various quality standards are being assessed according to performance criteria. The COHSASA information and reporting system is web-based and is accessible to hospitals and provincial management. Thus, organizations can obtain information n their performance through the Internet. In an experiment to compare MRR and IM systems, research was conducted comparing the efficiency of either one of the two systems in capturing AE.More AE were picked up by IM. Also, IM gave some idea about contributing factors. Whereas MRR mainly detected AE related to clinical management, IM mostly reported AE that wee related to organization management services. The team made the following conclusions:

MRR picks up mostly the serious AE; importantly, MRR gives an incident rate; IM picks up AE's and near misses but does not give an incident rate; IM does however facilitate rapid intervention. Integration of the COHSASA and AIMS and MRR systems is proposed by the South African team in future work.

1.9 Alternative methods: Plans of action Country teams were asked about the possibility of implementing one or more of the alternative research tools, building on the platform created by the participating country teams with the on-going study. Teams were requested to provide information as to what methods they believe are feasible for application at country level and what they anticipate can be done in the near future (within the next three months). The following summarizes the responses: Tunisia: An additional hospital in Central Tunisia will be included in the current study. The team will also conduct a prospective staff interview study in two hospitals.

WHO-EM/HCD/084E Page 62 South Africa: Will perform a cross-sectional study within the next month. Another hospital will be included in the next three months. Egypt: Already has one hospital remaining for completion of the record review study. The team will also perform an observation (on injection safety) in two or three hospitals. Jordan: The team believes that all of the above mentioned methods are feasible in Jordan. The team proposed to perform a staff interview and observation study within the next three months. Sudan: The team will try to include an extra (sixth) hospital for record review depending on time availability following entry of remaining data. The team also proposed to conduct a nominal group interview and an observation (one hospital to be included in each study). Yemen: The team has two rnore hospitals to complete with the MRR methodology. The team also proposed to start an observation study after completion of data entry. Morocco: The team in Morocco is currently busy with a quality competition at a national level and were therefore unable to propose extra work within the next three months. However, the team will complete. MRR in two hospitals and will then start a cross-sectional study in one hospital as well as a nominal group interview study possibly for implementation in hospitals included in the quality competition.

WHO-EMJHCD/O84/E Page 63

Annex 8

REFERENCES 1. Wilson R et al. The Quality in Australian Health Care Study. Med. J. Aust. 1995;163: 458471. 2. Vincent C. Neale G, Woloshynowych M. Adverse events i n B~itishhospitals: preliminary retrospective record review. British A4edicul J0~ina1.2001,;322:5 17-9. 3. Sari A et al. Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. British Medical Jour71a1, 2007;334:79.

4. World Alliance for Patient Safety: WHO Draft Guidelines for adverse event reporting and learning systems: From information to action. Leape, LL. WHO/EIP/SPO/QPS/05.3. Available online at http://www.who.int/patientsafetv/events/O5/Reuo~ing Guidelines.pdf, accessed 13 November, 2007. 5. Shaw R et al. Adverse events and near miss reporting in the NHS. Quality Safe Health Care 2005, August; 14 (4): 279-83.

6. Michel P et al.Comparison of three methods for estimating rates of adverse events and rates of preventable adverse events in acute care hospitals. British Medical Journal, 2004 January, 24;328(7433): 199.