Learning from past incidents?

I've been thinking about incident reporting in healthcare, in terms of what we can learn about the design of medical devices, based on both what is theoretically possible and also what actual incident reports show us.

Incident reporting systems (e.g., NRLS) are a potential source of information about poor usability and poor utility of interactive medical devices. However, because the health care culture typically focuses on outcomes rather than processes, instances of sub-optimal use typically pass unremarked. There is growing concern that there is under-reporting of incidents, but little firm evidence on the scale of that under-reporting. One study that compared observed errors against reported incidents involving intravenous medications identified 55 rate deviation and medication errors in nine hours of observation; 48 such incidents had been reported through the hospital incident reporting system over the previous two years, suggesting a reporting rate of about 0.1%. Firth-Cozens et al investigated causes for low reporting rates, even when clinicians had identified errors or examples of poor care. All groups of participants “considered that minor, commonplace or unintentional mistakes, ‘genuine or honest’ errors, one-off errors, or ones for which a subordinate is ‘obviously sorry’ or has insight, need not be reported”. Examples reported by their participants included incidents involving infusion pumps: problems for which the design or protocols for use of the pumps were contributing factors. Even when incidents are reported, those reports might not deliver insights into what went wrong. In our recent study of incident reports involving home use of infusion pumps, we found that reports gave much greater insight into how people detected and recovered from the device not working than into what had caused the device not to work properly in the first place.

While incident reporting systems might be one source of information on poor design or use of interactive devices in healthcare, this is not a reliable route for identifying instances of poor design. Once an incident is reported it is important that the role of device design in contributing to the incident be properly considered, and not simply dismissed with the common response that the device “performed as designed” and that it was therefore a user error.

The Digital Doctor

I’ve just finished reading The DigitalDoctor by Robert Wachter. It’s published this year, and gives great insight into the US developments in electronic health records, particularly over the past few years: Meaningful Use and the rise of EPIC. The book manages to steer a great course between being personal (about Wachter’s career and the experiences of people around him) and drawing out general themes, albeit from a US perspective. I’d love to see an equivalent book about the UK, but suspect there would be no-one qualified to write it.

The book is simultaneously fantastic and slightly frustrating. I'll deal with the frustrating first: although Wachter claims that a lot of the book is about usability (and indeed there are engaging and powerful examples of poor usability that have resulted in untoward incidents), he seems unaware that there’s an entire discipline devoted to understanding human factors and usability, and that people with that expertise could contribute to the debate: my frustration is not with Wachter, but with the fact that human factors is apparently still so invisible, and there still seems to be an assumption that the only qualification that is needed to be an expert in human factors is to be a human.

The core example (the overdose of a teenage patient with 38.5 times the intended dose of a common antibiotic) is told compellingly from the perspectives of several of the protagonists:

poor interface design leads to the doctor specifying the dose in mg, but the system defaulting to mg/kg and therefore multiplying the intended dose by the weight of the patient;

the system issues so many indistinguishable alerts (most very minor) that the staff become habituated to cancelling them without much thought – and one of the reasons for so many alerts is the EHR supplier covering themselves against liability for error;

the pharmacist who checked the order was overloaded and multitasking, using an overly complicated interface, and trusted the doctor;

the robot that issued the medication had no ‘common sense’ and did not query the order;

the nurse who administered the medication was new and didn’t have anyone more senior to quickly check the prescription with, so assumed that all the earlier checks would have caught any error, so the order must be correct;

the patient was expecting a lot of medication, so didn’t query how much “a lot” ought to be.

This is about design and culture. There is surprisingly little about safer design from the outset (it’s hardly as if “alert fatigue” is a new phenomenon, or as if the user interface design and confusability of units is surprising or new): while those involved in deploying new technology in healthcare should be able to learn from their own mistakes, there’s surely also room for learning from the mistakes (and the expertise!) of others.

The book covers a lot of other territory: from the potential for big data analytics to transform healthcare to the changing role of the patient (and the evolving clinician–patient relationship) and the cultural context within which all the changes are taking place. I hope that Wachter’s concluding optimism is well founded. It’s going to be a long, hard road from here to there that will require a significant cultural shift in healthcare, and across society. This book really brought home to me some of the limitations of “user centred design” in a world that is trying to achieve such transformational change in such a short period of time, with everyone having to just muddle through. This book should be read by everyone involved in the procurement and deployment of new electronic health record systems, and by their patients too... and of course by healthcare policy makers: we can all learn from the successes and struggles of the US health system.

When is a medical error a crime?

I've recently had Collateral Damage recommended to me. I'm afraid I can't face reading it: just the summary is enough. Having visited Johns Hopkins, and in particular the Armstrong Institute for Patient Safety, a couple of months ago, I'm pretty confident that the terrible experience of the Walter family isn't universal, even within that one hospital, never mind nationally or internationally. And therein lies a big challenge: that there is such a wide spectrum of experiences and practices in healthcare that it's very difficult to generalise.

There are clearly challenges:

• the demands of doing science and of providing the best quality patient care may pull in opposing directions: if we never try new things, relying on what is already known as best practice, we may not make discoveries that actually transform care.
• if clinicians are not involved in the design of future medical technologies then how can those technologies be well designed to support clinical practice? But if clinicians are involved in their design, and have a stake in their commercial success, how can they remain objective in their assessments of clinical effectiveness?

There are no easy answers to such challenges, but clearly they are cultural and societal challenges as well as being challenges for the individual clinician. They are about what a society values and what behaviours are acceptable and/or rewarded, whether through professional recognition or financially.

I know that I have a tendency to view things positively, to argue for a learning culture rather than a blame culture. Accounts like "Collateral Damage" might force one to question that position as being naive in the extreme. For me, though, the question is: what can society and the medical establishment learn from such an account? That's not an easy question to answer. Progress in changing healthcare culture is almost imperceptibly slow: reports such as "to err is human" and "an organisation with a memory", both published over a decade ago (and the UK report now officially 'archived'), haven't had much perceptible effect. Consider, for example, the recent inquiry into failings in Mid Staffordshire.

Bob Wachter poses the question "when is a medical error a crime?". He focuses on the idea of a 'just culture': that there is a spectrum of behaviours, from the kinds of errors that anyone could make (and for which learning is a much more constructive response than blaming), through 'at risk' behaviours to 'reckless' behaviours where major risks are knowingly ignored.

The Just Culture Community notes that "an organisation's mission defines its reason for being". From a patient's perspective, a hospital's "reason for being" is to provide the best possible healthcare when needed. Problems arise when the hospital's mission is "to generate a profit", to "advance science", or any other mission that might be at odds with providing the best possible care in the short term. The same applies to individual clinicians and clinical teams within the hospital.

I find the idea of a "just culture" compelling. It is not a simple agenda, because it involves balancing learning with blame, giving a sophisticated notion of accountability. It clearly places the onus for ensuring safety at an organisational / cultural level, within which the individual works, interacts and is accountable. But it does presuppose that the different people or groups broadly agree on the mission or values of healthcare. 'Collateral Damage' forces one to question whether that assumption is correct. It is surely a call for reflection and learning: what should the mission of any healthcare provider be? How is that mission agreed on by both providers and consumers? How are values propagated across stakeholders? Etc. Assuming that patient safety is indeed valued, we all need to learn from cases such as this.

The Hillsborough report 23 years on

I'm listening right now to the news report on the review of the Hillsborough disaster from 23 years. ago. I have heard terms including "betrayed", "dreadful mistakes were made", "lies" and "shift blame" (all BBC News at Ten). There is talk of "cover up", and people not admitting to mistakes made.

Families of the victims seem to be saying that they were never looking for compensation but that they wanted to be heard, and they want to know the truth. Being heard seems to be so important; if we do not hear then we do not learn; if we do not learn then we cannot change practices for the better. Maybe for some compensation is important, but for many others all that matters is that the tragedy should not have been in vain.

Earlier today, in a different context, a colleague was arguing that we need people to be "accountable" for their actions and decisions, that people need to be punished for mistakes. But we all make mistakes, repeatedly and often amusingly; for example, this evening, I phoned one daughter thinking I was phoning the other one, and because I was so sure I knew who I was talking to, and because we have a lot of "common ground", it took us both a while to realise my error. We could both laugh about it. Errordiary documents lots of equally amusing mistakes. But occasionally, mistakes have unfortunate consequences. Hillsborough is a stark reminder of this. Does unfortunate consequences automatically mean that the people who made mistakes should be punished for them? Surely covering up mistakes is even more serious than making errors in the first place. How much could we have learned (and how much easier would it have been for families to have recovered) if those responsible had not covered up and avoided being accountable? Here, I want to use the term "accountable" in a much more positive sense, meaning that they were able to account for the decisions that they made, based on the information and goals that they had at the time.

Being accountable currently seems to be about assigning blame; maybe this is sometimes appropriate – particularly if the individual or organisation in question has not learned from previous analogous incidents. But maybe sometimes learning from mistakes is of more long term value than punishing people for them. That implies a different understanding of "accountable". We need to find a better balance between blame and learning. Unless I am much mistaken.

Attitudes to error in healthcare: when will we learn?

In a recent pair of radio programmes, James Reason discusses the possibility of a change in attitude in the UK National Health Service regarding human error and patient safety. The first programme focuses on experiences in the US, where some hospitals have shifted their approach towards open disclosure, being very open about incidents with the affected patients and their families. It shouldn't really be a surprise that this has reduced litigation and the size of payouts, as families feel more listened to and recognise that their bad experience has at least had some good outcome in terms of learning, to reduce the likelihood of such an error happening again.

The second programme focuses more on the UK National Health Service, on the "duty of candour" and "mandatory disclosure", and the idea of an open relationship between healthcare professional and patients. It discusses the fact that the traditional secrecy and cover-ups lead to "secondary trauma", in which patients' families suffer from the silence and the frustration of not being able to get to the truth. There is of course also a negative effect on doctors and nurses who suffer the guilt of harming someone who had put their trust in them. It wasn't mentioned in the programme, but the suicide of Kim Hiatt is a case in point.

A shift in attitude requires a huge cultural shift. There is local learning (e.g. by an individual clinician or a clinical team) that probably takes effect even without disclosure, provided that there is a chance to reflect on the incident. But to have a broader impact, the learning needs to be disseminated more widely. This should lead to changes in practice, and also to changes in the design of technology and protocols for delivering clinical care. This requires incident reporting mechanisms that are open, thorough and clear. Rather than focusing on who is "responsible" (with a subtext that that individual is to blame), or on how to "manage" an incident (e.g. in terms of how it gets reported by the media), we will only make real progress on patient safety by emphasising learning. Reports of incidents that lay blame (e.g. the report on an unfortunate incident in which a baby received an overdose) will hardly encourage greater disclosure: if you fear blame then the natural reaction is to clam up. Conversely, though, if you clam up then that tends to encourage others to blame: it becomes a vicious cycle.

As I've argued in a recent CS4FN article, we need a changed attitude to reporting incidents that recognises the value of reporting for learning. We also need incident reporting mechanisms that are open and effective: that contain enough detail to facilitate learning (without compromising patient or clinician confidentiality), and that are available to view and to search, so that others can learn from every unfortunate error. It's not true that every cloud has a silver lining, but if learning is effective then it can be the silver lining in the cloud of each unfortunate incident.