We are constantly told healthcare is “ripe” for disruptive innovation, a process of creative destruction enabled by the exciting new technologies of digital health. Yet, the robust demonstration of almost any substantive benefit (beyond perhaps transient delight and toxic self- absorption) has been slow in coming.
The challenges of moving a health-related technology from promise to impact are illustrated nicely by two recent attempts to carefully evaluate the benefits of intriguing new devices.
The Propeller Health Inhaler Monitor
The first study (NCT01509183) was sponsored by the non-profit California Healthcare Foundation (CHCF), and sought to examine whether use of an inhaler monitoring device made by Asthmapolis (now Propeller Health) resulted in improved asthma control, the study’s primary endpoint (CHCF is also an investor in Propeller Health). Control, measured by the Asthma Control Test (ACT) would be assessed at 4, 8, and 12 months.
The device senses inhaler use – in this study, the use of fast-acting “rescue” inhaler medications, specifically — and communicates data to patients (via an app) and to providers (via reports and change of status alerts), according toMedPage Today. In this study, asthmatic patients who had experienced a “healthcare utilization event” in the preceding year were randomized into either an active group , receiving the device and associated data, or a group receiving a deactivated device that provided no supplemental information.
As study co-author Bob Quade explains in an email, “the fundamental intervention is information.”
At a national meeting last November, interim results were presented. These data revealed slight improvements in the ACT in the active arm compared to the control arm in both adults and children, differences not statistically significant.
However, additional analysis found subjects in the actively monitored group used rescue inhalers significantly less often than the control group. Perhaps most strikingly, inpatient days were reduced from 0.225 per person per year in control group to 0.087 in active group; emergency room visits decreased marginally (from 0.141 per person per year in control group to 0.103 in active group) as well.
This reduction in utilization is claimed to be associated with a savings of $688.05 per patient (compared to baseline); subjects in the control group, notably, were found to save $281.95 per patient compared to baseline, suggesting a regression to the mean effect, not surprising given that subjects were selected based on experiencing a healthcare event in the previous year (most asthma patients don’t experience events in any given year). It’s also possible that simply participating in the study may have contributed to a change in utilization (the Hawthorne effect at work). Both possibilities were acknowledged by Quade.
However, the active group still exhibited a dramatic savings (~$400 per patient) associated with reduced utilization compared to the control group, despite the lack of significant changes in asthma control as measured by ACT scores. This presents a real puzzle.
“Their own numbers don’t add up,” he asserts in an email. “First, they say it’s not statistically a significant difference.”
“Even if it were significant,” Lewis continues, the difference reported on the ACT “is something like 2.2% improvement in the score. Then, they claim an inpatient reduction of 62%. This violates several of the rules in Why Nobody Believes. First, costs can’t decline that much and second, the cost reduction has to tie to the quality improvement.”
An alternative explanation, however, is that there are improvements in care not reflected in the ACT measure; for example, if the intervention doesn’t prevent exacerbation but catches them early enough to enable them to be managed as an outpatient rather than an inpatient, this could theoretically save significant dollars, yet be difficult to pick up on the ACT.
This study highlights the challenges of rigorously demonstrating the benefit of a digital health intervention that appears to make a world of sense. It seems logical, even obvious, that improved monitoring of rescue inhaler use, coupled with improved, immediate communication with patients and providers would improve disease management. As study principal investigator Rajan Merchant M.D., at Woodland Clinic Medical Group (a member of Dignity Health) points out, it enables the care of asthma patients to transition from episodic to continuous – one of the key ambitions of digital health.
I look forward to examining a peer-reviewed publication once the study is completed later this year, and all data are read out. (Presentation of additional data is planned for the American Thoracic Society meeting in May, according to Dr. Merchant.) My suspicion is that there may ultimately be a cost-savings achieved (vs the control group), and perhaps eventually a small improvement in asthma control. Whether this economically justifies the use of the intervention will need to be determined, given the cost of the devices and software, and the extra provider time potentially required.
While this sounds like a guarded assessment, the truth is I’m optimistic about the possibilities of this platform. A particularly attractive feature of digital health interventions (in contrast to more traditional medical products) is the opportunity for relatively rapid iterative improvements. It will be fairly easy for the company to apply its learnings from this initial study, and tweak its platform – monitoring the use of controller inhalers, say, to assess adherence (a feature now cited on the Propeller Health website).
Even at this point, the company has apparently refined the physician interface to the point where it’s extremely easy, Dr. Merchant says, to quickly view an entire population of asthmatic patients, and immediately determine, in real time, the small subset that seem to be heading towards trouble. It’s a compelling vision, one that seems like it could be generalized to the management of a range of other chronic conditions.
Even more compelling, of course, would be data conclusively demonstrating such population management tools resulted in measurable improvements in patient health.
The Steelcase Treadmill Desk
The second study, published last month in PLOS One and conducted by investigators at the University of Minnesota, the University of Texas, and the Mayo Clinic, examined the impact of one of today’s trendiest health devices – a treadmill desk (invented at the Mayo Clinic by study co-author James Levine) — on the productivity and total activity of a small number (~40) employees at a financial services firm who volunteered to participate. Subjects were provided with a treadmill desk during either the first two months of the year-long study, or during the seventh month. Total daily activity was measured with an accelerometer, and workplace productivity assessed by both self-report and supervisor surveys.
The authors found that compared to baseline, introduction of a treadmill resulted in a total daily increase in about 74 calories burned, and resulted in an initial large bump in productivity which decreased (but remained above baseline) for the next five months or so, then started to increase again.
These data, write the authors, “suggest that the introduction of treadmill workstations, as hypothesized, has a significantly favorable impact on both physical activity and work performance. “
The study’s corresponding author, Avner Ben-Ner, a Professor in the Carlson School of Management, notes in an email,
“for this type of office workers even a 2 or 3% increase in productivity (just to use some figures) is good, and there are long term health benefits associated with standing and moving about, which are hard to estimate and hard to quantify. But delay of onset of diabetes or osteoporosis may [lead to] health care cost reductions, among others. The participants were mostly inactive office workers so for them the gains are likely to be greater than for active people.”
In the publication, the authors write, “companies ought to consider making treadmill workstations available to their sedentary employees,” though they responsibly point out “several limitations to our study that may restrict their applicability to other situations,” including the Hawthorne effect (again) and the self-selection of subjects.
In contrast, a characteristically breathless TechCrunch article declared these results “convincingly show why it’s worth it to a company’s bottom dollar to buy walking stations for the entire office.”
The TechCrunch story continued,
“With any rudimentary back-of-the napkin calculations, outfitting the entire office with walking stations is a penny saver. As long as the new equipment satisfies this the [sic] simple Productivity Increase + Healthcare Savings > Cost of Treadmill Per Employee equation, it makes perfect sense.” (emphasis in original)
I’m not so sure. It would be remarkably naïve (and profoundly foolhardy) for an executive to purchase, on the basis of this small study, a treadmill for every employee, and expect to see a positive, measurable financial return.
First, it’s unlikely that slightly increased weekly activity (in the unlikely event this was even achieved were treadmills imposed rather than requested) would translate into a significant reduction in healthcare expenditures.
Second, it’s a large stretch to expect a company-wide treadmill policy would result in a significant boost in revenue, even in the event (again, fairly unlikely at scale) that employees report, and supervisors confirm, slight improvement on performance metrics of questionable predictive value; increased output may seem sensible in theory, but not when you’ve a realistic sense of the complexity of the path between tenuous metrics and hard revenue.
Not surprisingly, most wellness programs are targeted to the head of HR – the person in charge of recruitment and retention – rather than the CFO, who is responsible for the bottom line. It’s far more likely that a generous treadmill desk policy might help your company compete for talent than directly benefit your P&L.
This example effectively highlights our collective tendency to underestimate the translational gap, the space between a promising technology and the glorious health and business opportunity it’s hypothesized to represent. A gene sequence is not a drug, information is not a cure, and an early study is not recorded revenue.
To the extent that brilliant technologists are increasingly (thank goodness) looking towards healthcare as the next opportunity, it will be especially important they calibrate their response to randomized field trials, and learn what many medical product researchers have discovered through painful experience – a promising exploratory academic study doesn’t generally translate into an approvable product, and even an approved product can struggle to demonstrate real world impact.
Having said that, would I like to try a treadmill desk? You bet. But if I tried to persuade my lean company to pick up the tab, I suspect the pithy response would echo the memorable words of Clark W. Griswold: “Good talk, son.”