Three top criticisms against CMS’ overall hospital star ratings

The Hospital Compare Overall Hospital Quality Star Ratings, published by CMS in July and updated in December 2016, are meant to provide transparency on hospital quality to help patients select the best site for their healthcare needs.

In reality, leading healthcare institutions are warning consumers that the ratings leave a lot of information out and may not paint a complete picture of a hospital’s qualifications.

“Patients should beware of making decisions about hospitals using the newly released star ratings. They are based on a deeply flawed methodology that does not take into account important differences in the patient populations and the complexity of conditions that teaching hospitals treat,” Association of American Medical Colleges (AAMC) President and CEO Darrell G. Kirch, MD, wrote in a statement when the ratings were released in July. “As a result, many of the nation’s leading teaching hospitals-institutions that provide the most advanced healthcare in the world-have been assigned lower ratings than other hospitals that treat patients with less complex conditions or that treat only certain conditions. This new system could have very serious consequences for patients who are choosing where to go for treatment, potentially driving them away from some of the best hospitals for their conditions.”

Methodology

A hospital’s overall quality star rating encompasses 64 of the more than 100 measures displayed on the Hospital Compare website. The methodology for the ratings was developed with the help of technical experts and public comment, according to CMS, and will be revised as needed, with ratings updated quarterly.

While CMS states on its website that the agency has received “numerous” letters of support from patient and consumer advocacy groups, many industry leaders are much less happy with the functionality of the ratings system.

The AAMC convened a panel of experts on quality reporting back in 2014 to help develop a set of guiding principles to help CMS create a consistent and meaningful ratings system. Those principles included recommendations about specifying target audiences for the ratings, and using measures and methodologies supported by the National Quality Forum, says the AAMC. CMS, however, did not utilize the work of the AAMC and continued with its “one-sized-fits-all-model,” according to the organization.

“The flawed methodology is unsuited to provide comprehensive ratings for a wide variety of hospitals and communities, and the ratings do not accurately reflect the full picture of hospital care in the United States,” AAMC leaders wrote in a statement in October 2016. “As healthcare professionals at our teaching hospitals continue to provide world-class healthcare and lead quality improvement efforts, the AAMC and our member institutions will urge CMS to modify the Star Ratings methodology to take into account patient populations and complex care provided by teaching hospitals.  Until the ratings are revised, it is patients who will suffer the most from the lack of clarity and transparency, who may choose a hospital for its rating over a hospital that would provide the best care for their condition.”

Even Congress questioned the ratings system, asking CMS in the spring of 2016 to review its methodology. The push from Congress caused CMS to delay the launch of the system, originally planned for April 2016, to July 2016.

Still, the ratings appear to be skewed in the favor of smaller, private and non-teaching hospitals that take fewer complex cases and have patients in higher socioeconomic classes, say critics.

Just 2.2%-102 in total-of U.S. hospitals earned a five-star rating in the first release of data from the new system, with 20.3% earning four stars, 38.5% earning three stars, 15.7% earning two stars and 2.9% earning one star. Another 20% of hospitals did not have adequate reporting data to be rated. Broken down by size, hospitals with fewer beds fared better, with hospitals with less than 99 beds having the highest percentage of five-star ratings and fewer ratings of three stars or less when compared to hospitals with 100 beds or more.

Criticism #1: Ratings are vague, generalized

Janis M. Orlowski, MD, chief healthcare officer at the AAMC, says the association supports a transparent ratings system that would make healthcare choices easier for the public, but that doesn’t mean that the star ratings are that system.

“We’re completely in favor of rating and making good information available to the public. The thing is that if you take a look at a large complex academic medical center or health system and give it a rating, what does that really mean?” Orlowski says.

The current ratings system does little to inform a patient seeking a liver transplant or a healthy pregnant woman where their needs would best be met, she says. Instead, the system gleans incomplete information from hospitals and places them under a generic rating.

The AAMC has suggested that the ratings be separated into service lines to provide better information.“The fact that it comes together under a single star rating is not the best way to present information to the public,” she says.

Criticism #2: Less reporting leads to higher ratings

Another major concern is that reporting is spotty, at best, says Orlowski. Seven domains are assessed, but half of the hospitals that received five-star ratings did not report on all those domains. Large teaching hospitals report everything, but there are many specialty or smaller hospitals that don’t report on mortality or morbidity, she explains.

“What we believe and what the data shows is that the more measures you reported on, the lower your score was. I think that’s a reflection that there are some measures that are easier to report on,” Orlowski says. “It’s not that those hospitals didn’t voluntarily report those measures, it’s that they didn’t have enough numbers to report. But we should compare like hospitals to like hospitals. Everyone was thrown into one pot and there were significant differences. The system is not set up the way it should be.”

Akin Demehin, director of policy at the American Hospital Association (AHA), echoes Orlowski’s concerns and says the ratings system does not achieve its intended goals.

“Transparency is certainly a goal we have long supported. Hospitals have been reporting quality data on hospital compare for over a decade now and the star ratings were supposed to help make the data easier to understand and more accessible,” Demehin says. “But we think the star rating that’s currently designed is deeply flawed and raises far more questions than answers.”

Like the AAMC, Demehin says the AHA found gross discrepancies in how hospitals were rating based on the number of measures that were reported.

“We see clear evidence that the more measures you report on at hospital compare, and the more you have included in the star rating, the worse you do,” Demehin says. “For large hospitals, they’re going to be the ones that are more likely to report on a significant number of measures. That’s a source of bias we think needs to be looked at.”

Internal server error