Who's the boss? Time for a software update...

Last summer, I gave a lift to a couple of friends to a place I was unfamiliar with. So I used a SatNav to help with the navigation. It was, of course, completely socially unaware. It interrupted our conversation repeatedly, without any consideration for when it is and is not appropriate to interrupt. No waiting for pauses in the conversation. No sensitivity to the importance of the message it was imparting. No apology. Standard SatNav behaviour. And indeed it’s not obvious how one would design it any other way. We turned off the sound and relied solely on the visual guidance after a while.

More recently, a colleague started up his computer near the end of a meeting, and it went into a cycle of displays: don’t turn me off; downloading one of thirty three. I took a record of the beginning of this interaction, but gave up and left way before the downloading had finished.

It might have been fine to pull the plug on the downloading (who knows?) but it wasn’t going to be a graceful exit. The technology seemed to be saying: “You’ve got to wait for me. I am in control here.” Presumably, the design was acceptable for a desktop machine that could just be left to complete the task, but it wasn’t for a portable computer that had to be closed up to be taken from the meeting room.

I have many more examples, and I am sure that every reader does too, of situations where the design of technology is inappropriate because the technology is unaware of the social context in which it is placed, and the development team have been unwilling or unable to make the technology better fit that context.

"How to avoid mistakes in surgery": a summary and commentary

I've just returned from the US, and my one "must see" catch-up TV programme was "How to avoid mistakes in surgery" (now available on youtube). It's great to see human error in healthcare getting such prominent billing, and being dealt with in such an informative way. This is a very quick synopsis (of the parts I particularly noted).

The programme uses the case of Elaine Bromiley as the starting point and motivation for being concerned about human error in healthcare. The narrator, Kevin Fong, draws on experience from other domains including aviation, firefighting and formula one pit-stops to propose ways to make surgery and anaesthesia safer. Themes that emerge include:

• the importance of training, and the value of simulation suites (simlabs) for setting up challenging scenarios for practice. This is consistent with the literature on naturalistic decision making, though the programme focuses particularly on the importance of situational awareness (seeing the bigger picture).

• the value of checklists for ensuring that basic safety checks have been completed. This is based on the work of Atul Gawande, and is gaining recognition in UK hospitals. It is claimed that checklists help to change power relationships, particularly in the operating theatre. I don't know whether there is evidence to support this claim, but it is intuitively appealing. Certainly, it is important in operating theatres, just as it has been recognised as being important in aviation

• the criticality of handovers from the operating theatre to the intensive care unit. This is where the learning from F1 pitstops comes in. It's about having a system and clear roles and someone who's in charge. For me, the way that much of the essential technology gets piled on the bed around the patient raised a particular question: isn't there a better way to do this?

• dealing with extreme situations that are outside anything that has been trained for or anticipated. The example that was used for this was the Hudson River plane incident; ironically, on Thursday afternoon, about the time this programme first broadcast, Pete Doyle and I were discussing this incident as an example that isn't really that extreme, because the pilot has been explicitly trained in all the elements of the situation, though not in the particular combination of them that occurred that day. There is a spectrum of resilient behaviour, and this is an example of well executed behaviour, but it's not clear to me that it is really "extreme". The programme refers to the need to build a robust, resilient safety system. Who can disagree with this? It advocates an approach of "standardise until you have to improvise". This is true, but this could miss an important element: standardisation, done badly, reduces the professional expertise and skill of the individual, and it is essential to enhance that expertise if the individual is to be able to improvise effectively. I suspect that clinicians resist checklists precisely because it seems to reduce their professional expertise, when in fact it should be liberating them to develop their expertise at the "edges", to deal better with the extreme situations. But of course that demands that clinical professional development includes opportunities and challenges to develop that expertise. That is a challenge!

The programme finishes with a call to learn from mistakes, to have a positive attitude to errors. Captain Chesley 'Sully' Sullenberger talks about "lessons bought with blood", and about the "moral failure of forgetting these mistakes and having to re-learn them". `On the basis of our research to date, and of discussions with others in the US and Canada studying incident reporting and learning from mistakes, this remains a challenge for healthcare.

Ethics and informed consent: is "informed" always best?

I am in the US, visiting some of the leading research groups studying human factors, patient safety and interactive technologies. This feels like "coming home": not in the sense that I feel more at home in the US than the UK (I don't), but in that these groups care about the same things that we do – namely, the design, deployment and use of interactive medical devices. Talking about this feels like a constant uphill struggle in the UK, where mundane devices such as infusion pumps are effectively "invisible".

One of the issues that has exercised me today is the question of whether it is always ethical to obtain informed consent from the patients who are receiving drugs via infusion devices. The group I'm working with here in Boston have IRB (Institutional Review Board, aka Ethics Board) clearance to just obtain informed consent from the lead nurse on the ward where they are studying the use of devices. Not even from all the nurses, never mind the patients. In one of our studies, we were only allowed to observe a nurse programming a device in the middle of the night if we had obtained permission to observe from the patient before they had fallen asleep (which could have been several hours earlier). Even though we were not gathering any patient data or disturbing the patient in any way. In fact, we were probably disturbing the patient more by obtaining informed consent from them than we would have been just observing the programming of the pump without their explicit knowledge.

We recently discussed the design of a planned study of possible errors with infusion devices with patient representatives. Feedback we got from one of them was: "patients and relatives need to have complete confidence in the staff and equipment, almost blind faith in many instances." There are times when ensuring that patients are fully informed is less important than giving them reassurance. The same is true for all of us when we have no control over the situation.

On the flight on the way here, there was an area of turbulence where we all had to fasten our seatbelts. That's fine. What was less fine what the announcement from the pilot that we shouldn't be unduly worried about this (the implication being that we should be a little bit worried): as a passenger in seat 27F, what use was it for me to worry? No idea! It made the flight less comfortable for me, to no obvious benefit (to me or anyone else).

Similarly with patients: if we accept that studying the use of medical devices has potential long-term benefits, we also need to review how we engage patients in the study. Does obtaining informed consent give them benefits or whatever-is-the-opposite? Maybe there are times where the principle of "blind faith" should dominate.

The information journey and information ecosystems

Last year, I wrote a short piece for "Designing the search experience". But I didn't write it short enough (!) so it got edited down to a much more focused piece on serendipity. Which I won't reproduce here for copyright reasons (no, I don't get any royalties!). The theme that got cut was on information ecosystems: the recognition that people are encountering and working with information resources across multiple modalities the whole time. And that well designed information resources exploit that, rather than being stand-alone material. OK, so this blog is just digital, but it draws on and refers out to other information resources when relevant!

Here is the text from the cutting room floor...

The information journey presents an abstract view of information interaction from an individual’s perspective. We first developed this framework during work studying patients’ information seeking; the most important point that emerged from that study was the need for validation and interpretation. Finding information is not enough: people also need to be able to assess the reliability of the information (validation) and relate it to their personal situation and needs (interpretation).

This need for validation and interpretation had not been central to earlier information seeking models—possibly because earlier studies had not worked with user groups (such as patients) with limited domain knowledge, nor focused on the context surrounding information seeking. But we discerned these validation and interpretation steps in all of our studies: patients, journalists, lawyers and researchers alike.

The information journey starts when an individual either identifies a need (a gap in knowledge) or encounters information that addresses a latent need or interest. Once a need has been identified, a way to address that need must be determined and acted upon, such as asking the person at the next desk, going to a library, looking “in the world,” or accessing internet resources. On the web, that typically means searching, browsing, and follow trails of “information scent”. Often finding information involves several different resources and activities. These varied sources create an information ecosystem of digital, physical and social resources.

Information encountered during this journey needs to be validated and interpreted. Validation is often a loose assessment of the credibility of the information. Sillence and colleagues highlight important stages in the process: an early and rapid assessment—based on criteria such as the website’s design and whether it appears to be an advertising site—is typically followed by a more deliberate analysis of the information content, such as assessing whether it is consistent with other sources of information.
 
Interpretation is not usually straightforward. It often involves support from information intermediaries (an important part of the information ecosystem). This is one of the important roles of domain specialists (e.g. doctors and lawyers): working with lay people to interpret the “facts” in the context of the actual, situated needs. Even without help from intermediaries, Sillence & co. describe the lay users of health information in their study as acting like scientists, generating and testing hypotheses as they encountered new information resources, both online and offline. No one information resource is sufficient: online information fits in a broader ecology of information sources which are used together, albeit informally, to establish confidence and build understanding.
 
The interpretation of information can often highlight further gaps in understanding. So one information need often leads to others. For example, a colleague of mine was recently planning to buy a Bluetooth headset. His initial assumption was that there were only a few suitable headsets on the market, and his aim was simply to identify the cheapest; but it quickly became apparent that there were hundreds of possible headsets, and that he first needed to understand more about their technical specifications and performance characteristics to choose one that suited his needs. A simple information problem had turned into a complex, multi-faceted one. A known item search had turned into an exploratory search, and the activity had turned from fact-finding to sensemaking.

Information resources surround us. We are informavores, consuming and interpreting information across a range of channels. We are participants in huge information ecosystems, and new information interaction technologies need to be designed not just to work well on their own, but to be valuable components of those ecosystems.

The importance of context (even for recognising family!)

I've been using the face recognition feature in my photograph management software. It was coming up with some suggestions that were pretty impressive (e.g. finding several additional photos that featured my mother, when primed with a few) and some that felt a little spooky (e.g. suggesting that a photo of me was actually of my mother – something that probably none of us wants to admit to, however attractive the parent). But it was also making some inexplicably bizarre suggestions – e.g. that a male colleague might be one of my daughters, or that a wine glass might be a face at all. This recognition technology is getting very sophisticated, but it clearly does not recognise faces in a human-like way!

From a computational perspective, it does not account for context: it identifies and matches features that, in some low-level way, correspond to "face", and it gets that right a lot of the time, identifying real human faces and artificial faces (such as a doll). However, it does not have the background knowledge to do the gender- and age-based reasoning that people naturally do. This makes some of its suggestions seem bizarre. And the fact that it works with low-level features of an image is really exposed when it suggests that a wineglass should be named.

From a human perspective, context also matters in recognition. For most adult faces that were close friends or relations, recognition was generally straightforward, but for children or less familiar people, it was almost impossible to recognise people out of context. The particular software I was using did not allow me to switch easily between detail and context, so there are some faces that are, and will remain, unlabelled, meaning that I won't be able to find them again easily. For example, with context, it was instantly apparent who this small child was: she was sat on her (recognisable) mother's knee, with her big sister at her side. But without that context, she is a small (and slightly uncomfortable-looking) blonde toddler. Context matters.

When context really matters: entertainment, safety ... or neither?

Yesterday, Mark Handley drew my attention to a video of the recent evacuation of an All Nippon Airways Boeing 787 due to a battery problem: "Here's a video from inside the plane:  The inflight entertainment system has clearly just rebooted, and about half the screens are displaying the message "Please Wait" in large comforting letters. Maybe not the most appropriate message when you want people to evacuate quickly!"

Fortunately, it seems that passengers ignored the message asking them to wait, and did indeed evacuate instead. But did they do so as quickly as they might have done otherwise? We'll never know. They will have had many other sources of information available at the time, of which the most powerful were probably other people's behaviour and the appearance of the evacuation slides. The digital and physical contexts were providing different cues to action.

Brad Karp observed that : "Presumably when you activate the slides, you either want to kill the entertainment system or have it display "EVACUATE!"

Alan Cooper, in "The inmates are running the asylum", discusses many examples of interaction design. One he explores is the challenge of designing good in-flight entertainment systems. For example, he points out that the computer scientist's tendency to deal with only three numbers (0, 1, infinity) is inappropriate when choosing a maximum number of films to make available on a flight, and that choosing a reasonable (finite) number makes possible attractive interaction options that don't scale well to infinity. He also argues that the entertainment system needs two different interfaces: one for the passenger and a different one for the crew who need to manage it. But if you watch the video, you will see that half the screens on the plane are showing a reboot sequence. Who designed this as an interface for passengers? If the system developers don't even think to replace a basic reboot sequence by something more engaging or informative, what chance of them thinking about the bigger picture of how the entertainment system might be situated within, and interact with, the broader avionics system?

In-flight entertainment systems don't seem to be considered as part of the safety system of the aircraft. Surely, they should be. But that requires a broader "systems" perspective when designing, to give passengers more contextually relevant information that situates the digital more appropriately within the physical context.

Happy (entertaining, safe) flying!

Sometimes it just works!

Last week, we were in Spain, enjoying the sunshine and using the motorways. And I was really impressed by their toll machines: attractive, easy to use, and even delightful. Even though they were taking our money. My very amateur video of the machine in action isn't great (sorry!), but it shows the key features of the system.

1) The instructions screen presents a clear representation of what to do next, independent of language. This isn't essential, but provides backup for those who might not be able to interpret the main interface.

2) Each action is clearly illuminated in a timely way: insert ticket; pay (card, notes / coins); get change; optionally, get receipt. Sure, the action sequence is simple, and there's limited scope for error, but the device leaves little room for doubt. [Contrast this with the story a colleague told me of observing someone buying a ticket from a UK rail ticket machine who could not locate a notes slot, so folded up a £5 note and fed it carefully into the coin slot.]

3) The coin slot, in particular, is well designed, opening and closing smoothly to accept coins at just the right time.

I know it's simple, but that's surely the point: it's only as complicated as it needs to be, and no more, and it's easy even for someone who speaks no Spanish to use without help.

Human–Computer Interaction specialists like me tend to notice poor features of interactive systems; it's delightful to celebrate a system that really seems to work well, come rain or shine.