BLOG: Why we can't measure wellness outcomes

I recently attended a terrific Care Continuum Alliance (CCA) conference, and kudos to the new regime with Chris Coloian and Fred Goldstein.

LewisOne highlight was a packed-house panel discussion, hosted by Dee Edington, featuring myself and three other panelists Bob Stone, Sean Sullivan and Tom Parry. Panelists largely agreed that the population health industry, broadly speaking, had failed at outcomes measurement after 15 years of trying. One commenter hoped we wouldn’t be having this discussion 15 years hence. To the consternation of other panelists, I opined that we would probably be having this exact discussion in 15 years.

There are several reasons for that:

• Anytime measurement is standardized, there will be clear winners and losers. The losers will therefore never go along with any standardization agreement.

• Not only that, but they will-as they did with the CCA outcomes guidelines committee-pack the group that volunteers to develop these guidelines so that the measurement template favors them.

• Even the winners will join this committee because measurement is not a zero-sum game. Rather, it’s a positive-sum game. It’s possible to do exactly what the outlines guidelines committee did and create measurement tools that inflate savings. The only people who won’t be represented will be the buyers. Prospective buyers simply aren’t interested enough to participate in these committees, while few current purchasers of wellness programs want to learn that everything they’ve been told about the success of their programs is officially wrong. More fundamentally, we don’t even know what we are measuring and probably can’t even agree on what to measure, let along how to measure it. Risk factors? Presenteeism? Cost savings? And won’t the correct metrics, as panelist Bob Stone observed, vary by company? Also, these metrics often conflict with one another. Reducing risk factors may require a punitive program that saps morale.

• What’s measured is what’s managed, but sometimes a “high” measurement isn’t a good measurement. Employers and vendors like to brag about how many employees undergo annual biometric screens and preventive doctor visits, but the literature is quite clear that neither is a good idea. Even so, achievement of high rates is part of most measurements today.

• The final problem is the question: measurement as compared to what? For three decades, we have been comparing participants to non-participants, even though everyone knows that participants and non-participants have fundamentally different mindsets.

Why We Don’t Want to Measure Wellness Outcomes

Even if we could standardize measurement, do we want to? Standardization is the enemy of innovation. You can see the insidious effect of standardization in health risk assessments (HRAs) today. They are the most standardized tool in all of wellness and are also about three years behind U.S. Preventive Services Task Force guidelines. An HRA committee needs to read the guidelines, agree that there needs to be a change to the HRA, rewrite it, get it revalidated, resubmit it for approval, and then convince customers that it is worth changing to the new HRA, even though the new results won’t be comparable to the old ones.

On the other hand, consider the recent innovation of well-being. Well-being obsoletes many typical wellness tools and metrics. If there were standard measurements of wellness success, well-being might flunk them. Do we want to discourage well-being just because it would score lower in some preordained categories than wellness does? Would well-being have its own measurement categories?

What We Should Do Instead

Perhaps instead of attempting to standardize reporting, we should standardize integrity in reporting. Rather than standardizing metrics, standardize validation. A validator could agree to certain criteria, like disclosing biases. So that if a result is based on participants vs. non-participants, a validator would acknowledge that limitation and bias. And, like an auditor in the real world, a validator couldn’t just run away from the validation after the check clears-validators would need to put up a bond to back their attestations so that end-users could have faith in them like stockholders have faith in an auditor’s letter.

 This isn’t a wacky hypothetical idea.

The alternative is to own a validation monopoly in an industry asymptotically approaching irrelevance due to in-fighting over the quixotic quest for a measurement ideal that can and should never be achieved.

Al Lewis is an MHE Editorial Advisory Board member and the executive director of the Disease Management Purchasing Consortium