Friday 28 October 2016

Guidance on creating, evaluating and implementing effective digital healthcare interventions

This is an unconventional blog post – essentially, a place to index a set of papers. Last year, I participated in a workshop: ‘How to create, evaluate and implement effective digital healthcare interventions: development of guidance’. 
The workshop was led by Susan Michie, and resulted in a set of articles discussing key issues facing the development and evaluation of digital behaviour change interventions. There were about 50 participants, from a variety of countries and disciplines. And we all had to work ... on delivering interdisciplinary papers as well as on discussion. The outcome has just been published.
Credits: The workshop was hosted in London by the Medical Research Council, with funding from the Medical Research Council (MRC)/National Institute for Health Research (NIHR) Methodology Research Program, the NIH Office of Behavioral and Social Sciences Research (OBSSR)  and the Robert Wood Johnson Foundation.The workshop papers are being made publicly available with the agreement of the publishers of the American Journal of Preventive Medicine.

Thursday 20 October 2016

If the user can't use it, it doesn't work: the invisible costs of bad software

This is a quick rant about unusable enterprise systems and turning visible costs into invisible costs. For an earlier, longer, discussions about different unusable systems, see my reviews of ResearchFish and an electronic healthcare system.

Yesterday, I was one of several people asked to use the Crown Commercial Services system to review some documents related to a bid for one of our public funding bodies. The use of this system is apparently mandated for that organisation.

I was sent instructions on how to do part of the process (which I could not have worked out from the user interface). I followed the instructions provided as far as they were relevant, and I then explored some more to try to locate the documents of the actual bids (which appeared to comprise 28 separate documents for eight bids). Then I tried to download them all in one file. 30 minutes later, the system timed out on me while still processing to create that file. When I logged back in I couldn’t locate the download window again without simply doing all the same actions a second time. And I ran out of time, energy or will to pursue this.

This is yet another example of a system where there is no evidence that the developers ever considered how the system would be used, by whom, under what circumstances, the learning curve to use it first time ... or anything else about the users. Susan Dray has a nice claim: "If the user can't use it, it doesn't work". This is yet another enterprise system that is absolutely not fit for purpose.

What this does is to shift costs from development (investing in making a system that is fit for purpose) to use (forcing every user of the system to waste time trying to achieve their goals despite the system). The former would be a visible cost to the developers and the people who commissioned the system while the latter is an invisible cost borne in all the stress and loss of productivity of the people who have to use the system. For the UK Research Evaluation Framework (REF), these invisible costs were estimated at almost £250 million. That was a one-off exercise; there should be a practice of estimating the annual costs of unusable enterprise systems. I'm pretty confident that the invisible costs would turn out to be significantly greater than the visible costs of creating a system that was fit for purpose in the first place. And we know how to do it. We have known how to do it for decades!