Evaluating the merits of hospital ranking data

Ranking outcomes can vary drastically depending on criteria

  • Photo

According to Mark Valliere, using hospital ranking data released last week by Consumer Reports to make a decision about health care would be like deciding to purchase a car by only looking inside the trunk.

Hospitals can receive drastically different outcomes from the different entities that issue such rankings. That's because each one uses different pieces of data to derive its results, said Valliere, senior vice president of medical affairs and chief medical officer at Mercy Medical Center in Cedar Rapids.

Consumer Reports is one of many entities across the nation that rank hospitals. So does CareChex, Truven, Hospital Compare, the Leapfrog Group and U.S. News and World Report.

Truven, for example, uses nine criteria to determine its rankings. Among them are core measures, patient safety indices, mortality, complications, re-admissions, length of stay, operating margin and cost per case.

"People like CareChex use a lot of (criteria)," Valliere said. "When you start going down the line like Leapfrog, they're only using a couple.

"HealthGrades uses one or the other. Consumer Reports use two of them. U.S. News & World Report use two of them.

"When you get down to the point of why do they all come to different conclusions, it's because they're looking at different pieces of the puzzle."

Valliere places a much higher value on analysis completed by Truven, which specializes in health analytics.

"If you're looking at Consumer Reports, they looked in the trunk and said, 'This is a great car, based on our review of the trunk,'" he said. "As a result of that, you can get U.S. News & World Report that says Massachusetts General Hospital's the greatest hospital in the nation and then you get Consumer Reports saying Massachusetts General Hospital is the second-worst hospital in the state of Massachusetts."

Institutions also can manipulate data in different ways, he added.

"These guys all do that a little bit differently," Valliere said. "They're all buying the same information, they're just bending that information a little bit differently, and so different conclusions for us, different conclusions for Mass General ... because nobody wants to make the same car as everybody else, and nobody wants to produce the same health care report as anybody else."

Last week, for example, Consumer Reports and Chicago-based MPA issued hospital surgery rankings for 2,500 hospitals across the nation using mortality and prolonged length of stay data.

UnityPoint Health-St. Luke's Hospital, Mercy Iowa City and University of Iowa Hospitals and Clinics all earned an average ranking. Mercy Medical Center in Cedar Rapids earned a lower score.

Doris Peter, Consumer Reports Health Ratings Center associate director, said the goal of releasing the data was to show people the variation in care at hospitals across the nation.

"That's something I think that we all need to learn, is that your care is going to differ where you go," Peter said.

Like what you're reading?

We make it easy to stay connected:

to our email newsletters
Download our free apps

Give us feedback

Have you found an error or omission in our reporting? Tell us here.
Do you have a story idea we should look into? Tell us here.