In Digital Health Space, Many Companies Sidestep Rigorous Study

Even as digital therapeutics have emerged as an increasingly important therapeutic category, a new report shows many digital health companies have avoided rigorous clinical evaluation of their products.

The study, published in the Journal of Medical Internet Research, highlights the gap that often exists between the claims made by digital health software developers and the scientific evidence available to validate their claims.

Simon C. Mathews, MD, of the Johns Hopkins University School of Medicine, was the study's senior and corresponding author. He and colleagues noted that previous research into the evidence behind digital health tools has often been narrowly focused on a single therapeutic area, or has been limited in scope. In the new study, they wanted to use a wide lens.

“While some of my prior work and other studies have highlighted deficits in digital health, this was among the largest studies to show the real gap between reality and expectations of digital health,” Mathews told Managed Healthcare Executive.

The investigators pulled data from the Rock Health Digital Health Venture Database, the Food and Drug Administration (FDA), and the National Library of Medicine. They searched for companies selling digital health products aimed at prevention, diagnosis, or treatment of medical conditions. In the end, they identified 224 digital health companies, which had been in existence for an average of 7.7 years.

Mathews and colleagues then assigned each company a “clinical robustness” score, based on the number of regulatory filings and clinical trials it had completed. On average, the companies had a score of 2.5, with an average of 1.8 clinical trials per company and 0.8 regulatory filings per company.

However, that average conceals a significant dichotomy: 20% of companies had clinical robustness scores of 5 or greater, while nearly half of companies—44%—had clinical robustness scores of 0.

Mathews said the latter figure is a significant problem for digital health companies.

“For digital health to be taken seriously and seen as legitimate clinical tools, far more companies need to be engaging in the rigorous processes that are required to show efficacy,” he said.

Though many companies had scarce clinical evidence, that did not stop them from making public claims about their product’s efficacy, economic impact, or engagement capabilities. The study found no correlation between the number of public claims a company made and their clinical robustness score, suggesting that many companies made claims they could not back up with rigorous clinical trials or regulatory filings.

In some ways, the study highlights an emerging split between digital health and wellness apps, which are marketed widely and widely available, and prescription digital therapeutics, which are more rigorously tested, require a physician’s prescription, and have been authorized or cleared by the FDA.

Mathews said clinicians should be aware of such differences when they counsel patients about the use of digital health software.

“​There are many [digital health products] that provide education and social support that can be helpful and don't require extra diligence,” he said. “However, for any tools that make any clinical claims, I would have a healthy dose of skepticism. In these scenarios, I would encourage clinicians to review the evidence first to help manage patient expectations.”

Mathews and colleagues said patients will benefit from greater testing and validation of digital health products, but he said the companies also stand to gain by enhancing their legitimacy. Only by more rigorously publishing evidence to back their claims can digital health solutions meet their potential, he said.

“If you are like me and believe they have the potential to deliver more patient-centric care, then you should also expect evidence comparable to traditional clinical treatment options,” he said.

Reference:

Day S, Shah V, Kaganoff S, Powelson S, Mathews SC. Assessing the Clinical Robustness of Digital Health Startups: Cross-sectional Observational Analysis. J Med Internet Res. 2022;24(6):e37677. Published 2022 Jun 20. doi:10.2196/37677