Press Releases How to help fix the Centers for Medicare & Medicaid Services’ flawed hospital rating system

The current hospital star-rating system used by the U.S. government’s Centers for Medicare & Medicaid Services is plagued with numerous flaws, and University of Chicago Booth School of Business Professor Dan Adelman has come up with a new way to address one of its most controversial issues.

In his recent study, “An Efficient Frontier Approach to Scoring and Ranking Hospital Performance,” forthcoming in the INFORMS journal Operations Research, Adelman proposes an alternative methodology to scoring U.S. hospitals that addresses some of the limitations underlying the CMS’ Hospital Compare ratings and improves accuracy. His findings are illustrated in a new searchable database of hospital quality ratings. Developed in collaboration with Chicago Booth Review, the database compares his system’s ratings against the government’s system.

“My suggested approach rewards hospitals that do well and penalizes hospitals that don’t on the measures they actually report,” said Adelman, the Charles I. Clough, Jr. Professor of Operations Management at Chicago Booth. “It proposes an alternative approach that could one day help lead to a better system that allows patients to have more accurate information to make choices. The goal is to create a ratings system that allows competition between hospitals to be based on more realistic assessments of hospital quality and performance than it is today.”

According to Adelman, the central problem with CMS’ current approach—which uses what statisticians call the latent variable model—is that a hospital’s rating may go down even if it improves along every performance measure, relative to all other hospitals.

For example, until recently, Chicago’s Rush University Medical Center boasted the maximum of five stars in the CMS’ rating system. Data used to compute the July 2018 rating indicated that the hospital had improved in many areas, so it came as a shock when hospital administrators got a preview of the new ratings and Rush had dropped to three stars. Nationwide hospital uproar over the new results caused CMS to delay their publication until February 2019 and modify the ratings. Rush was then bumped from three stars to four.

Adelman argues that using an efficient frontier approach when calculating hospital ratings would result in more equitable comparisons that are not prone to such dramatic and unexpected swings. The key to this approach is that every hospital is given its own, unique set of measure weights that score it as close as possible to the best performing hospitals under the same weights. This contrasts with the CMS approach, which uses a common set of parameters across all hospitals in estimating the latent variable for a specific hospital.

Adelman warns that his proposed model is designed to combat the problems with the current latent variable model but does not address other concerns in hospital rankings—including those related to underlying measures, steps in the methodology or the ratings system itself.

Adelman’s study was recently presented with the Public Sector Operations Research Best Paper Award by INFORMS, the leading international association for operations research and analytics professionals.

For a more detailed explanation of Professor Dan Adelman’s findings, visit https://review.chicagobooth.edu/hospital-ratings to compare his efficient frontier approach with CMS Hospital Star Ratings for nearly every U.S. hospital.

Contact Booth Media Relations

Marielle Sainvilus
Director of Public Relations and Communications
Phone: 773.531.2894

Email Marielle