Pennsylvania mandates that any medical event, involving the clinical care of a patient in a licensed acute care facility, that either results in an unanticipated injury (serious event) or could have injured the patient (near-miss incident) be reported to the Pennsylvania Patient Safety Authority.1 Pennsylvania is the only state mandating the reporting of near-miss incidents. Under the state statute, all individual reports are protected by confidentiality; they are not legally discoverable or admissible as evidence or subject to disclosure under right-to-know laws. Information identifying the patient or the healthcare personnel must be not reported. The Patient Safety Authority supervises the analysis of over 235,000 reports per year from over 550 hospitals, ambulatory surgical centres, and other facilities.2 The reports are classified by over 200 event types. Over 96% of the reported events are near-miss incidents.
Analyses of the causes of the medical errors and evidence-based best practices to prevent the errors are published in the Pennsylvania Patient Safety Advisory (Advisory),3 a quarterly peer-reviewed online journal indexed in the NLM Catalogue and CINAHL® Plus. Subscription to this online journal is free. The journal is distributed to over 4650 healthcare providers in Pennsylvania, in addition to over 4000 subscribers in all 50 states and 37 other countries (unpublished data). The Patient Safety Authority permits reproduction of Advisory articles provided the source is clearly attributed. Analytic reviews are accompanied by learning objectives and self-assessment questions that are linked to continuing medical education credits and nursing continuing education credits from the Pennsylvania Medical Society and Pennsylvania State Nurses Association, respectively. Over 440 articles have been published in over 40 issues of the Advisory.
The process of reporting and analysing medical errors has provided an infrastructure for identifying and correcting system weaknesses with some successes, such as the standardization of wristband colours to unambiguously communicate important patient information at the bedside and in transit.4 Aggregation of reports for analysis has also permitted detection of multiple weakness that can lead to rare events, such as fires on the surgical field, which occur an average of only once every 300,000 operations, but are reported ten times per year across the state.5 Dissemination of information from the analysis of reports also, anecdotally, encourages reporting, which increases awareness of system weaknesses.
However, the experience of the Patient Safety Authority has been that solely describing a patient safety problem and warning people to be more careful is – as might be expected – ineffective. Proposing a best practice without proposing how to implement and disseminate the practice in an institution’s healthcare delivery system does not produce the desired reduction in errors. An excellent example is the Patient Safety Authority’s efforts to prevent wrong-site surgery.6
The Pennsylvania Patient Safety Authority’s program to prevent wrong-site surgery
In mid-2007, the Patient Safety Authority began a program to prevent wrong-site surgery with an analysis of all wrong-site events and near-misses in Pennsylvania operating rooms (ORs).7 Eventually 21 evidence based best practices were identified for preventing wrong-site surgery, from identifying the correct site of the operation when scheduling the procedure to using intra-operative radiological confirmation to verify the correct vertebral level during spinal surgery.8 Articles on preventing wrong-site surgery have been published in every quarterly issue of the Advisory since June, 2007.9 However, the number of events for each quarter of the academic year 2009-2010 (16) was the same as for the first quarter of data collection in 2004(16).10 This lack of effect might have been predicted. McGlynn has shown that patients receive, on average, 55%, of recommended practices.11 Because of the lack of progress in eliminating wrong-site surgery through the presentation of evidence-based best practices,8 the Patient Safety Authority added collaborative learning in an attempt to improve compliance with those practices.
Dissemination of innovation in healthcare have been well summarized by Berwick.12 Successful dissemination through the use of collaborative learning has been nicely described by Dixon-Woods et al.13 Collaborative learning is the process of learning from each other. The Patient Safety Authority found the following principles useful in developing its collaborative learning module to implement best practices: i) adaption of recommended practices is a form of learning how to implement them; ii) all innovation is adapted, not adopted, for the local environment; iii) new ideas may come from the outside, but new processes come from within; iv) innovations need to be adapted locally in order to be adopted locally; v) attempts to improve recommendations do not represent resistance.
The Patient Safety Authority collaborative learning efforts benefitted from having key infrastructure elements already in place. The evidence-based best practices were already available and did not have to emerge as part of the collaborative learning process.8 A secure website for collaborating institutions to confidentially share experiences, called PassKey (for Pennsylvania Patient Safety Knowledge Exchange), already existed.14 The Patient Safety Authority has a staff of eight patient safety liaisons, primarily nurses and healthcare quality professionals who work directly with healthcare institutions in different geographic areas of the state to coordinate patient safety activities between the Patient Safety Authority and the institutions.
Prior to starting the collaborative learning module, the Patient Safety Authority interviewed patient safety officers at eight hospitals.15 Each had independently reduced its incidence of wrong-site surgery reports from an average of one wrong-site surgery every 24 weeks for a minimum of 3½ years to none for a minimum of 64 weeks. The themes of these individual successes were: i) leadership – the initial commitment and the on-going support of hospital leaders, including empowerment of the staff to stop procedures to resolve concerns and mentor providers when their performance did not comply with best practices; ii) education about best practices and outcomes from high-risk behaviour; iii) involving the providers in the detailed improvement of the policies and procedures, described by one as commitment from the top down, process from the bottom up; iv) standardization of practices – throughout an institution’s system, if possible; v) development or improvement of checklists to reinforce best practices; vi) on-going monitoring of compliance; vii) strict enforcement, including the empowerment of the staff to stop procedures and the counselling of non-compliant providers. These experiences were used to guide the focal points of the collaborative learning module. The Patient Safety Authority recognized that institutions had to volunteer willingly to join a collaboration, rather than be drafted. The incentives for institutions to devote time and attention to this collaborative program, in lieu of other priorities within their institutions, proved to be the opportunities for external expert consultation, confidential and high-quality aggregate process and outcome data, and a forum for collaboration and standardization across a network. The Patient Safety Authority has completed two collaborations to prevent wrong-site surgery, one with 30 institutions and one with 19.16,17 The collaborative learning process was similar for both. The collaborations began with getting the commitment of the executive leaders of the institutions to identify domain-expert opinion leaders to lead the project within their institutions, provide them with adequate resources, and support means of compliance with redesigned systems. The opinion leaders in each institution were three champions – from the departments of surgery, anaesthesia, and perioperative nursing, respectively.
A baseline assessment was done by these project leaders. A self-assessment checklist was used to compare institutions policies and procedures with the Patient Safety Authorities evidence-based principles for reliable performance of correct-site surgery.8,18 A compliance monitoring tool (since modified) was used to record compliance with those principles for 10 patients at each institution.19
An initial all-day conference started with a morning educational conference, during which the evidence for the principles for reliable performance was presented,8 and the results of the gap analysis of policies and compliance for the group as a whole, were discussed. The afternoon was devoted to discussions of strengths and weaknesses in the systems of each institution using contextually de-identified events from the Pennsylvania Patient Safety Authority’s wrong-site surgery database as the starting points for the conversations.
The subsequent efforts of the individual project teams to improve their systems to reliably comply with evidence-based best practices was collaboratively supported by the confidential and secure PassKey website restricted to the collaborating institutions and by conference calls to discuss progress and problems. After six months, follow-up assessments of policies and procedures and compliance with evidence-based principles were done by the project leaders in the same manner as the baseline assessments. A follow-up half-day conference presented the results of those assessments, including improvements and remaining gaps. Noteworthy successes and recalcitrant problems were discussed. Plans were made for periodic monitoring of processes within institutions and continued monitoring for wrong-site surgery by the Patient Safety Authority, with continued Passkey collaboration, open lines of communication with the Patient Safety Authority, and follow-up support as needed. In both collaborations, there was a lag of a few months before the system redesign took hold and provider compliance was achieved. The first collaboration of 30 institutions achieved a 73% decrease in the rolling average of wrong-site events over 18 months.16 The second collaboration of 19 hospitals experienced no wrong-site OR procedures over more than one year.17 The first collaboration provided documentation of a 7% improvement in the alignment of policies with evidence-based best practices (from a baseline score of 82 to a follow-up score of 88). The most dramatic improvements were in the designation of a hospital staff person to be responsible for verifying the accuracy of information when the request to schedule an operation is received, and clearly delineating his/her role in the verification process (26.5%), using a standard mechanism for verifying the accuracy of information when the request to schedule an operation is received, including verifying the exact description of the surgical procedure and specifying the surgical site (38.5%), doing a separate time out prior to each procedure when multiple independent procedures are performed (18.5%), and removing patient information material left from previous surgeries when cleaning the ORs (52.5%).16
The first collaboration also documented improved compliance with existing evidence-based best practices, e.g. including information from the OR schedule and from the history and physical examination in the pre-operative verification (19% and 12% improvement respectively), marking the site after reconciliation of all documents (16% improvement), conducting a timeout prior to regional or local anaesthesia (16% improvement), having the surgeon encourage members of the surgical team to speak up if any concerns during the time out (8% improvement).16 The collective efforts of institutions that worked together to implement best practices to prevent wrong-site surgery has resulted in a persistent overall yearly decrease in wrong-site surgeries across Pennsylvania from a record high of 76 in the academic year prior to the completion of the first collaboration to a record low of 46 this last academic year.20 Interestingly, an analysis showed that all the improvement came from institutions that had made a documented effort to implement best practices.21 The institutions that had not been surveyed or part of the collaborations actually had an increase in reports of wrong-site events over that time. Recently, a 48-month follow-up of compliance with best practices was completed for some of the institutions in the first collaboration and a 22-month follow-up was completed for the institutions in the second collaboration.22 Compliance with some best practices had actually increased over time, most were maintained, and some had decreased. Continued improvement in compliance with best practices was more common than decreased compliance.
Based on its initial experiences, the Patient Safety Authority concludes that collaborative learning is an effective way of achieving implementation of best practices. Educational communication with institutions about evidence-based best practices is necessary, but not sufficient to get compliance. It is also necessary to work with institutions to adapt and adopt best practices through system changes that make the best process the easiest to do and sub-optimal processes harder to do. Institutions, in the form of leaders and champions, must be motivated to change. The commitment and support of executive leadership is critical, as efforts will require personnel time. A gap analysis is important to capture providers’ attention and focus the efforts as productively as possible.
One of the greatest values of collaborative learning is sharing successes and failures with others on the same journey. Processes need to be monitored. It seems to take time to make and see improvement. Continued monitoring is necessary to hold the gain.
The Patient Safety Authority has not had sufficient experience with collaborative learning to determine the best way to present improvements in the correction of medical errors to the public. The keys to holding gains are not well understood. It is unknown if successes in implementation of best practices can be disseminated to institutions outside of the collaborations. It is also unknown if collaborative learning is necessary to implement all evidence-based best practices.
What is clear is that education about evidence-based best practice is necessary but insufficient to create change. Guided efforts to implement best practice within institutions are also needed to get reliable compliance with optimal process and the desired results.