More on Quality and Safety

Hospital rankings fail to include crucial metrics in scoring, Health Affairs says

Absence of cost of care, waste elimination, social responsibility metrics are major flaws in U.S. News Best Hospital rankings, experts say.

Beth Jones Sanborn, Managing Editor

Credit: health.usnews.comCredit:

A new Health Affairs blog post is questioning the value and validity of the oft-celebrated U.S. News and World Report Best Hospital rankings, saying the scoring is flawed and fails to take into account several metrics that should be crucial to any hospital's true value measure, not to mention consumers.

Judith Garber, health policy and communications fellow at the Lown Institute in Brookline, Massachusetts, and Shannon Brownlee, senior vice president for Lown and a former senior editor at U.S. News & World Report, claim that there are major holes in the ranking methodology because it fails to takes into account cost of care or efforts related to cutting cost and waste as well as how they are fulfilling their social mission.

[Also: CMS delays October star ratings release as advocacy groups push for major methodology overhaul]

First, though, they take issue with the weight given to hospital performance on specialties and serious, complex procedures versus care for chronically ill patients, which they say makes up the bulk of hospitalizations. Of the 448 points that a hospital can grab for it's "Honor Roll" score, 340 of them are related to specialty scores, for which they say only outcomes from "challenging or critical" procedures count. Garber and Brownlee allege this makes the rankings potentially misleading for consumers, who they say will more likely be hospitalized for an issue stemming from chronic illness than for a specialty procedure.

Also, they said more than a quarter of each specialty score stems from the results of a physician survey that asks doctors to list up to five hospitals in their specialty that provide the best care to patients with serious conditions, essentially turning the ranking into a "popularity contest." They went on to cite a 2017 study they say showed reputation had bigger impact on scores than "more objective measures."

[Also: Hospitals get time to preview star ratings, CMS says]

The absence of cost of care and waste elimination is a major deficiency in the ranking scores, Garber and Brownlee wrote, pointing out the lack reward for a hospital's work to cut unnecessary services, boost efficiency or embrace alternative payment models.

The scoring also falls short in rewarding low-value care, they said, pointing out that only 5 percent of a hospital's specialty score is decided by their efforts to avoid preventable complications, and no points are awarded for avoiding low-value care.

"A ranking that takes money into account could incentivize hospitals to improve efficiency and rein in the prices they charge, both of which could bring down the cost of healthcare for everyone," the authors wrote.

Additionally, they called out the rankings' rewarding of hospitals for having advanced technologies without also employing a scoring metric for whether the technology is being used "judiciously" or is effective on patients.

Finally, Garber and Brownlee argue the U.S. News rankings fail to adequately look at a hospital's social mission, with no examination of how much a hospital gives back to its community through such efforts as funding local nonprofits, operation of a free clinic and the number of community members served, no statistics on the proportion of women or people of color on staff and no criteria at all for community services.
They stressed that hospital rankings can motivate change of hospital practices, and these ranking should endeavor to move towards that goal.

"We need a ranking system with a different definition of 'best' hospitals. One that doesn't rely on reputation among physicians, includes multiple risk-adjusted outcomes, and takes cost of care, social mission, and high-value care into account," authors wrote.

Twitter: @BethJSanborn
Email the writer:

Show All Comments