Rating Process

Evaulation Criteria

Domain Examples of Criteria Strength and Weaknesses Comparison of the Ratings Systems
Potential for Misclassification Use of known measures that are flawed
Hospitals examined (numbers and type)
Risk Adjustment
Composite Methodology
Methodological Approach
Audit Mechanism
Importance/Impact Unique features that resonate with patients, referring physicians, and hospitals .PDF
Scientific Accepability Balanced Measurement
Hospitals examined (numbers and type)
Distribution/assigment of hospital grades/stars
Use of available measures
Use of unique data
Specific methodological concerns
Stability of rankings over time
Audit mechanism
Iterative Improvement Response to stakeholder feedback and scientific advances in measurement science
Review of methods prior to release
Peer review of methods
Expert panel level of involvement
Transparency Detailed methods report available (transparency)
Clear rationale for methodological decisions
Data availability (replicability)
Financial conflicts and details regarding how ratings are monetized
Usability Ease of overall use
Ability to compare hospitals easily
Attention to varying health literacy and numeracy
Audit Type and extent of data validity audits performed .PDF
Updates Notable changes made by rating systems since our grading process .PDF

Comparison of Characteristics of Rating Systems

Composite Description Data Sources

Composite score star rating (1 to 5 stars)

CMS claims, NHSN, HCAHPS, CMS reporting system (CASPER), Joint Commission, CMS Abstraction and Reporting Tool, CMS Clinical Data Warehouse

Distinguished Hospital Award for Clinical Excellence (DHACE) and America's 50 and 100 Best Hospitals Award

CMS inpatient claims, All-payer state data for ratings in Obstetrics/Gynecology, Bariatric Surgery, and Appendectomy

Hospital Safety Grade: Composite letter safety grade

Top Hospitals: Highest performing hospitals on the Leapfrog Hospital Survey

Hospital Safety Grade: Leapfrog Hospital Survey, CMS measures publicly reported on Hospital Compare (e.g. AHRQ PSIs, NHSN HAIs, HACs, HCAHPS), AHA Annual Survey and Information Technology Supplement

Top Hospitals: Leapfrog Hospital Survey

National 'Honor Roll' and 'Best Hospitals' (including 16 specialties and 9 procedures / conditions)

CMS inpatient claims, AHA Annual Survey, Hospital Compare for HCAHPS, public websites (STS, ACC, Magnet), Proprietary hospital reputation survey