What's in it for me? The challenges of designing interventions for others

"Uninvited guests" is an entertaining short video showing possible, compelling, responses to well-meaning digital interventions for wellbeing that an elderly relative is encouraged to use.

Recently, a friend (I'll call her Hanna) told me about her experience of something similar, and it highlighted to me just how challenging it is to design well to help others to live well, and how important it is to make new designs of direct value, and easy to understand.

Hanna's parents are elderly, and had been plagued by nuisance calls: some just irritating, but others that involved mis-selling, "fixing" a computer virus, or otherwise leaving her parents feeling unsettled and cheated. She wanted to work with them to help avoid these calls. They installed Truecall on the line. And for a couple of weeks, it seemed to be working really well: letting through trusted callers while blocking unknown callers. A couple of unrecognised callers contacted her to request access and she extended the list of trusted callers in response. All good!

Then things started to unravel. An elderly acquaintance who wasn't on the list tried calling, did not understand the 'blocking message' immediately, and promptly drove round to her parents' house to ask what was going on. They found this really embarrassing, and it undermined their trust in the system. Hanna worked with her parents to add every known acquaintance to the list of trusted callers. But their fear of missing even one 'real' call had been triggered. At least: that was the surface presentation; I suspect there was more going on.

Apparently, when adding names to the list of trusted callers, Hanna's parents talked about the data entry as if they would then be able to use the list as a phone book. That would have been useful to them. But of course it didn't have that functionality (it's a call blocker, not a call enabler). They had a poor mental model of how Truecall worked and what it did. I'm guessing that this lack of understanding made them feel alienated and disempowered.

Hanna showed her parents their own call log, highlighting all the nuisance calls that had been blocked, and that therefore had not been disruptive. But this was apparently not persuasive at all: they could not remember the occasions where they had been persuaded by mis-selling, and now the concern about missing genuine calls dominated completely. Indeed, Hanna's parents seemed to grow in confidence regarding their ability to manage nuisance calls with every day that passed, and Truecall seemed to become a device that questioned their competence.

They told her about one of their friends also using Truecall. But she tells me she couldn't work out whether this was a positive comment (this is catching on; we're ahead of the curve) or a negative comment (that friend is getting old and having difficulty screening nuisance calls).

At one level, Truecall is a technology that does one job and seems to do it very well. At another level, it is a social device. The fact that them using Truecall was visible to a few of their friends and acquaintances seems to have made it unacceptable, even "embarrassing". I'm guessing it is preferable to them to be autonomous, to feel in control, and not to be seen to be using a call blocker, than to avoid nuisance calls. 
 

We all use technologies that we don’t fully understand. But we need to understand them well enough to feel in control, and it seems as if Truecall went beyond that for these elderly people and their equally elderly friends.

Truecall has had rave reviews, and it really does seem to do its job very well. So it was a surprise to me when Hanna told me about her and her parents' experiences. Maybe, even though I'm pretty sure that Hanna's parents are in the target market segment for Truecall, for something to work for Hanna's parents it would have to be even easier to use, even more transparent. I'm guessing it would have needed the following features:

• everything accessible without obviously accessing the internet (so, visible on a dedicated display with the phone).
• offering the 'phone book' capability so that they could more easily make calls.
• having three call categories that are simultaneously enabled: trusted (come straight through); zapped (blocked, including all withheld numbers); and unknown (with a really easy way to move unknown numbers into trusted or zapped, whether before or after accepting the call).

I'm not sure that this is technically possible at the moment – or if it is, it might be prohibitively expensive to implement. But hopefully it will be possible in the near future. For me, the most important insight is that there are some very subtle emotional and social values that tip a technology from being something to engage with to being something that is rejected. In the uninvited guests video, the star of the show is technology savvy enough to subvert the best of intentions of his family and of the technology design; in Hanna's case, it seems that the only option for her parents was to reject the technology completely. We still have a lot to learn about how to design technology that is truly empowering.

The lifecourse and digital health

I've just been away for the weekend with a group of people of varying ages. Over breakfast, I was chatting with Diane (names have been changed), who surmised that she was the oldest person there. I looked quizzical: surely she's in her 70s and Edna is in her late 80s? But no: apparently, Diane is 88, and thinks that Edna is only 86. Appearances can be deceptive. Diane has a few health niggles (eyesight not as good as it once was, hip occasionally twinges) but she remains fit and active, physically and mentally. I hope I will age as well.

Meanwhile, last week I was at an Alan Turing Institute workshop on "Opportunities and Challenges for Data Intensive Healthcare". The starting point was that data sciences have always played a key role in healthcare provision and deployment of preventative interventions, and that we need novel mathematical and computational techniques to exploit the vast quantities of health and lifestyle data that are now being generated. Better computation is needed to deliver better health management and healthcare at lower cost. And of course people also need to be much more engaged in their own care for care provision to be sustainable.

There was widespread agreement at the meeting that healthcare delivery is in crisis, with rising costs and rising demands, and that there is a need for radical restructuring and rethinking. For me, one of the more telling points made (by a clinician) is that significant resources are expended to little good effect in the interests of keeping people alive, when perhaps they should be left to die peacefully. The phrase used was "torturing people to death". I don't imagine many of us want to die in intensive care or in an operating theatre. Health professionals could use better data analytics to make more informed decisions about when "caring" means intervening and when it means stepping back and letting nature take its course.

In principle, better data, better data analysis, and better personalised health information should help us all to be better manage our own health and wellbeing – not taking over our lives, but enabling us to live our lives to the full. My father-in-law's favorite phrase was "I'd like a bucket full of health please". But there's no suggestion that any of us will (or wants to) live forever. At the meeting, someone suggested that we should be aiming for the "Duracell bunny" approach to life: live well, live long, die quickly. Of course, that won't be possible for everyone (and different people have different conceptions of what it means to "live well").

This presents a real challenge for digital health and for society: to re-think how each and every one of us lives the best life we can, supported by appropriate technology. There's a widespread view that "data saves lives"; let's also try to ensure that the saved lives are worth living!

Experiences of home haemodialysis

An open letter to study participants:

You generously shared your experiences of managing your care on home haemodialysis (HHD), and of using the dialysis machine. You were one of 19 patients and families, or members of care teams, across four NHS Trusts who took part in the study. There were lots of common themes in what you told us:

• about the challenges of learning to do dialysis in the first place (and how scary it was in the first few weeks at home), but how it became routine (“like driving a car”) over time;
• about the challenges of troubleshooting, particularly when the problem was an unfamiliar one;
• about the ways your home care team support you – and might be able to support you better if you had more seamless data exchange with your team;
• about the common difficulties, such as clearing bubbles from the system and remembering to open and close all clamps at the right time; and
• about the various strategies you have discovered for keeping yourselves safe.

The project has now finished, and we’ve been reporting the findings as widely as possible:

• We’ve published a paper in a nephrology journal (http://www.biomedcentral.com/1471-2369/15/195/) on patients’ and carers’ experiences of interacting with home haemodialysis technology, and identified implications for quality and safety. A couple of short articles based on that paper have been published on the Shared Care website on home dialysis (https://shareddialysis-care.org.uk/home-therapies).
• A second paper has been published in a biomedical informatics journal (http://www.sciencedirect.com/science/article/pii/S1532046415001124). This focuses on a theoretical approach called Distributed Cognition. This helped to better understand the social structures that support people’s interactions with the dialysis machine.
• A further paper reflecting on how people cope with the demands of HHD has been accepted by the Journal of Renal Nursing. A draft can be seen at http://www.chi-med.ac.uk/research/bibdetail.php?PPnum=PP328 .
• We’ve given talks on the work at various events attended by clinicians, patients and manufacturers.

We hope that this project can ‘make visible’ your experiences and practices in managing care at home, and that this will help manufacturers to design next-generation systems, and nephrology services in planning effective home care support.

Thank you to everyone (patients, carers and professionals) who made the study possible.

The Digital Doctor

I’ve just finished reading The DigitalDoctor by Robert Wachter. It’s published this year, and gives great insight into the US developments in electronic health records, particularly over the past few years: Meaningful Use and the rise of EPIC. The book manages to steer a great course between being personal (about Wachter’s career and the experiences of people around him) and drawing out general themes, albeit from a US perspective. I’d love to see an equivalent book about the UK, but suspect there would be no-one qualified to write it.

The book is simultaneously fantastic and slightly frustrating. I'll deal with the frustrating first: although Wachter claims that a lot of the book is about usability (and indeed there are engaging and powerful examples of poor usability that have resulted in untoward incidents), he seems unaware that there’s an entire discipline devoted to understanding human factors and usability, and that people with that expertise could contribute to the debate: my frustration is not with Wachter, but with the fact that human factors is apparently still so invisible, and there still seems to be an assumption that the only qualification that is needed to be an expert in human factors is to be a human.

The core example (the overdose of a teenage patient with 38.5 times the intended dose of a common antibiotic) is told compellingly from the perspectives of several of the protagonists:

poor interface design leads to the doctor specifying the dose in mg, but the system defaulting to mg/kg and therefore multiplying the intended dose by the weight of the patient;

the system issues so many indistinguishable alerts (most very minor) that the staff become habituated to cancelling them without much thought – and one of the reasons for so many alerts is the EHR supplier covering themselves against liability for error;

the pharmacist who checked the order was overloaded and multitasking, using an overly complicated interface, and trusted the doctor;

the robot that issued the medication had no ‘common sense’ and did not query the order;

the nurse who administered the medication was new and didn’t have anyone more senior to quickly check the prescription with, so assumed that all the earlier checks would have caught any error, so the order must be correct;

the patient was expecting a lot of medication, so didn’t query how much “a lot” ought to be.

This is about design and culture. There is surprisingly little about safer design from the outset (it’s hardly as if “alert fatigue” is a new phenomenon, or as if the user interface design and confusability of units is surprising or new): while those involved in deploying new technology in healthcare should be able to learn from their own mistakes, there’s surely also room for learning from the mistakes (and the expertise!) of others.

The book covers a lot of other territory: from the potential for big data analytics to transform healthcare to the changing role of the patient (and the evolving clinician–patient relationship) and the cultural context within which all the changes are taking place. I hope that Wachter’s concluding optimism is well founded. It’s going to be a long, hard road from here to there that will require a significant cultural shift in healthcare, and across society. This book really brought home to me some of the limitations of “user centred design” in a world that is trying to achieve such transformational change in such a short period of time, with everyone having to just muddle through. This book should be read by everyone involved in the procurement and deployment of new electronic health record systems, and by their patients too... and of course by healthcare policy makers: we can all learn from the successes and struggles of the US health system.

On rigour, numbers and discovery

Recently, I was asked the following:

In talking to my psychology seminar group about their qualitative lab I ended up looking at Helene Joffe’s book chapter on thematic analysis.  She suggests including diagrammatic representations of the themes, together with quantitative data about how many participants mentioned the theme, and it’s subparts.  This appealed to the psychology students because it gives them quantitative data and helped them see how prevalent that theme was within the sample.

And then today I saw another paper “Supporting thinking on sample sizes for thematic analyses: a quantitative tool".  It argues that one should consider the power of the study when deciding on sample size – another concept I’d only seen in quantitative research. 

Both of these sources seem to be conducting qualitative analysis with at least a nod towards some of the benefits of quantitative data, which appears to make qualitative analysis have more rigor.  Of course, simply adding numbers doesn’t necessarily make something more rigorous but it does add more information to results of an analysis and this could influence the reader’s perception of the quality of the research.  However, I don’t recall seeing this is any HCI papers.  Why isn’t it used more often? 

The answer (or at least, my answer) hinges on nuances of research tradition that are not often discussed explicitly, at least in HCI:

Joffe, Fugard and Potts are all thinking and working in a positivist tradition that assumes an independent reality ‘out there’, that doesn’t take into account the role of the individual researcher in making sense of the data. Numbers are great when they are meaningful, but they can hide a lot of important complexity. For example in our study of people’s experience of home haemodialysis, we could report how many of the participants had a carer and how many had a helper. That’s a couple of numbers. But the really interesting understanding comes in how those people (whether trained as a carer or just acting as a helper) work with the patient to manage home haemodialysis, and how that impacts on their sense of being in control, how they stay safe, their experience of being on dialysis, and the implications for the design of both the technology and the broader system of care. Similarly, we could report how many of their participants reported feeling scared in the first weeks of dialysis, but that didn’t get at why they felt scared or how they got through that stage. We could now run a different kind of study to tease out the factors that contribute to people being scared (having established the phenomenon) and put numbers on them, but to get the larger (60-80) participants needed for this kind of analysis would involve scouring the entire country for willing HHD participants and getting permission to conduct the study from every NHS Trust separately; I’d say that’s a very high cost for a low return.

Numbers don’t give you explanatory power and they don’t give you insights into the design of future technology. You need an exploratory study to identify issues; then a quantitative analysis can give the scale of the problem, but it doesn’t give you insight into how to solve the problem. For HCI studies, most people are more interested in understanding the problem for design than in doing the basic science that’s closer to hypothesis testing. Neither is right or wrong, but they have different motivations and philosophical bases. And as Gray and Salzman argued, many years ago, using numbers to compare features that are not strictly comparable – in their case, features of different usability methods when used in practice – is 'damaged' (and potentially damaging).

Wolcott (p.36) quotes a biologist, Paul Weiss, as claiming, “Nobody who followed the scientific method ever discovered anything interesting.” The quantitative approach to thematic analysis doesn’t allow me to answer many of the questions I find interesting, so I’m not going to shift in that direction just to do studies that others consider more rigorous. Understanding the prevalence of phenomena is important, but so is understanding the phenomena, and the techniques you need for understanding aren’t always compatible with those you need for measuring prevalence. Unfortunately!

Innovation for innovation's sake?

As Director of the UCL Institute of Digital Health, my job is to envision the future. The future is fueled by innovation and vision. And there's plenty of that around. But the reality is much more challenging: as summarised in a recent blog post, most people aren't that interested in engaging with their health data (the ones who are most likely to be tracking their data are young, fit and wealthy), and most clinicians are struggling to even do their basic (reactive) jobs, without having much chance to think about the preventative (proactive) steps they might be taking to help people manage their health.

Why might this be? Innovation is creative and fun. It's also essential (without it, we'd still be wallowing around in the primordial soup). But there's a tendency for innovation to assume a world that is simpler than the real world: people who are engaged and compliant and have time to take up the innovation. Innovation tends not to engage with the inconvenient truths of real life, or to tackle the difficult and complex challenges that get in the way of simple visions.

We need a new approach to innovation: one that takes the really difficult challenges seriously, that accepts that the rate of progress may be slow, that recognises that it's much harder to change people and cultural practices than it is to change technology, but that these all need to be aligned for innovation to really work.

We need innovation that works with and for people. And we need to recognise that an important part of innovation is dealing with the inconvenient and difficult problems that seem to beset healthcare delivery, in all its forms.

Digital Health: tensions between hype and reality

There are many articles predicting an amazing future for digital technologies for healthcare: wearables, implantables, wirelessly enabled to gather vital signs information and collate it for future sensemaking, by the individual, clinicians and population health researchers. Two examples that have recently come to my attention are a video by the American Psychiatry Association and a report by Deloitte. The possibilities are truly transformational.

Meanwhile, I recently visited a friend who has Type II diabetes. On his floor, half hidden by a table, I spotted what I thought was a pen lid. It turned out to be the top of his new lancing kit. Although he had been doing blood glucose checks daily for well over a decade, he hadn't done one for over two weeks. Not just because he'd lost an essential part of the equipment, but because he'd been prescribed the new tool and hadn't been able to work out how to use it. So losing part of it wasn't a big deal: it was useless to him anyway. He told me that when he'd reported his difficulties to his clinician, he'd... been prescribed a second issue of exactly the same equipment. So now he has three sets of equipment: the original (AccuChek) lancing device and blood glucose meter, which he has used successfully for many years, but which he can't use now because he doesn't have spare consumables; and two lancing devices and meters (from a different manufacturer), with plenty of spare consumables, which he can't use because he finds the lancing device too difficult to use. And in trying to work out with him what the problem was, we managed to break one of them. Good thing he's got a spare!

If we think it's just isolated individuals who struggle, it's not: a recent report from Forbes reports similar issues at scale: poor usability of electronic health records and patient portals that are making work less efficient and effective rather than more.

So on the one hand we have the excitement of future possibilities that are gradually becoming reality for many people, and on the other hand we have the lived experiences of individuals. And for some people, change is not necessarily good. The real challenge is to design a future that can benefit all, not just the most technology-savvy in society.