Qualitative research comes of age

For a long, long time, qualitative research has felt like a "poor relation" to quantitative: so much more subjective, so much harder to generalise, so much more reliant on the skills of the researcher to deliver quality.

I'm delighted to see that it's becoming more mainstream - at least based on the evidence of a couple of recent publications that appear in the mainstream research literature and that simply report on how to report qualitative research well. One is in the medical literature, and the other in the psychology literature. The questions of what constitutes high quality qualitative research, and how to report it are ones we have grappled with, particularly when the findings of a study don't align well with the original aims because you discover that those aims were based on incorrect assumptions about the situation. I still get the sense that there is asymmetry in the situation: that qualitative researchers have to justify their methods to quantitative researchers much more forcefully than the converse. But this seems like progress nevertheless.

Quantitative tells you about outcomes, but gives little (or no) insight into causes or processes. To improve outcomes, you really need to understand causes too...

Learning from past incidents?

I've been thinking about incident reporting in healthcare, in terms of what we can learn about the design of medical devices, based on both what is theoretically possible and also what actual incident reports show us.

Incident reporting systems (e.g., NRLS) are a potential source of information about poor usability and poor utility of interactive medical devices. However, because the health care culture typically focuses on outcomes rather than processes, instances of sub-optimal use typically pass unremarked. There is growing concern that there is under-reporting of incidents, but little firm evidence on the scale of that under-reporting. One study that compared observed errors against reported incidents involving intravenous medications identified 55 rate deviation and medication errors in nine hours of observation; 48 such incidents had been reported through the hospital incident reporting system over the previous two years, suggesting a reporting rate of about 0.1%. Firth-Cozens et al investigated causes for low reporting rates, even when clinicians had identified errors or examples of poor care. All groups of participants “considered that minor, commonplace or unintentional mistakes, ‘genuine or honest’ errors, one-off errors, or ones for which a subordinate is ‘obviously sorry’ or has insight, need not be reported”. Examples reported by their participants included incidents involving infusion pumps: problems for which the design or protocols for use of the pumps were contributing factors. Even when incidents are reported, those reports might not deliver insights into what went wrong. In our recent study of incident reports involving home use of infusion pumps, we found that reports gave much greater insight into how people detected and recovered from the device not working than into what had caused the device not to work properly in the first place.

While incident reporting systems might be one source of information on poor design or use of interactive devices in healthcare, this is not a reliable route for identifying instances of poor design. Once an incident is reported it is important that the role of device design in contributing to the incident be properly considered, and not simply dismissed with the common response that the device “performed as designed” and that it was therefore a user error.