Illinois Hospital Association

Members Login Automatically
login		   User ID:   Password: Forgot your password?
Don't have a password?

January 25, 2008

HCAHPS Preview Data and Result Variations

TO: CEOs

Late on Friday, January 25, CMS announced that they did find errors in the results for Composite 3, 4, 5, and 6.  As a result, CMS is going to be correcting the error and will have new results available late next week.

Even though CMS is correcting these errors, hospitals are encouraged to participate in the HCAHPS conference calls on February 4th and 7th as noted in the memo.  Hospitals need to keep pressure on CMS to release the patient mix adjustment methodology and the weights applied using this methodology to the HCAHPS data for each hospital.  Additionally, many hospitals have expressed concern that the mode adjustment will never allow them to score near or at 100% and does not accurately reflect their performance.  The mode and patient mix adjustments may also prevent hospitals in the future to ever be at the upper tier of a value based purchasing payment system. 

Since the Centers for Medicare and Medicaid Services released the HCAHPS (hospital patient satisfaction) results for hospitals to preview on January 17, many hospitals have called and e-mailed IHA regarding discrepancies between their vendor results and the CMS results. The variation in results are due to the adjustments CMS made to the results for patient mix and survey administration mode utilized by hospitals.

It is important that hospitals review their data and compare it to their vendor results. Hospitals need to understand the variations in results as many hospitals have shared their vendor results with their hospital board, management, and hospital staff only to find out in a public report that their performance is not as they had expected.

The variations in results between hospital actual results from their vendors and CMS results have been anywhere from a few percentage points to nearly 30 percentage points. Most of the hospitals we have heard from that have been affected have seen changes in scores from actual results to being downgraded to a much lower score.

As CMS has stated in their Value Based Purchasing Plan to Congress (click here to view) that they would like to use adjusted HCAHPS results as part of the CMS performance based payment system, it is important that your concerns be expressed. CMS will be publicly reporting adjusted results in March, 2008 along the same 8 domains named in the Value Based Purchasing Plan:

  • Nurse Communication
  • Doctor Communication
  • Pain Management
  • Communication about Medications
  • Cleanliness and Quiet
  • Responsiveness of Hospital Staff
  • Discharge Information
  • Overall Rating of Hospital

Letting Your Voice Be Heard
As a result of the issues being raised among hospital providers, AHA is hosting two conference calls for hospital providers with management staff at CMS and AHRQ to respond to questions and address concerns hospitals have with the adjusted results. Many providers have commented that they would like both the actual and adjusted results provided to the public so the public can choose to use actual satisfaction scores or adjusted satisfaction scores.

Conference Call Meetings:
Monday, February 4 – 3:00 p.m. central time
Thursday, February 7 – 3:00 p.m. central time
Call in Number and Pass Code: (877)-271-2502, Pass Code: 8521

While you don’t need to register for this call, registration is encouraged so AHA has enough phone lines available. To register electronically, go to http://www.surveymonkey.com/s.aspx?sm=TlS3f1_2befmST7uRuteotfQ_3d_3d.

Contact IHA
If your hospital has been adversely effected by the adjustments CMS made to the HCAHPS composite scores from what you are seeing from your vendor results, please let me know so that I can compile a list of hospitals with significant variations. Please send information to Pat Merryweather at pmerryweather@ihastaff.org. IHA will not divulge your individual hospital identity and information as we are trying to determine how many Illinois hospitals are adversely affected by the adjustments as it will be important for the public reporting release in March as well as issues to be raised on using adjustments for Value Based Purchasing.

CMS HCAHPS ADJUSTMENTS

Composite Score 6 – Discharge Instructions
On January 17, it was identified by hospitals that the results listed under "Yes" and "No" were flipped around for Composite Score 6 --- the correct percents for "Yes" were listed under "No" and the "No" responses were listed under "Yes." CMS is aware of this error and should have a correction available shortly. The correct report will have for Illinois Average "25% Yes" and "25% No" and for the U.S. Average "74% Yes" and "26% No." The majority of hospitals will also have higher "Yes" percents than with the uncorrected results.

Adjustments to ALL Hospital Scores
The challenges hospitals are having in comparing their HCAHPS hospital results provided by their vendor as compared to the Medicare HCAHPS hospital results, is Medicare has opted to apply adjustments to the results. Please remember that IHA has a number of issues and questions we have raised on these adjustments, but want you to be aware of what is driving your adjusted results.

There are two adjustments that are applied to results – one known as the patient mix adjustment and the other known as the Mode Adjustment.

Patient Mix Adjustment. When CMS was reviewing results, they felt that some populations typically rated satisfaction lower than other populations when asked. As a result, they created a patient mix adjustment based upon self reported health status, linear education, service line, age category, ER admission source, response percentile, service by linear age interactions, and primary language other than English.

Each quarter, CMS will create new adjustments based upon data received, and then apply them to that quarter of data. Unless CMS shared the adjustments to each hospital, there is no way one can even estimate the effect of these adjustments on the hospital scores.

To give you an example, CMS and several studies have found that younger and more educated patients provide less positive evaluations of healthcare. Therefore, a hospital with a younger population with higher levels of education will have their results adjusted upwards to compensate for patient characteristics outside of the hospital control.

The debate on the patient mix adjustment methodology will continue as there are many that argue the purpose of the patient satisfaction survey is to raise satisfaction and identify gaps, not to adjust for them.

Without the adjustments shared with each hospital during each quarter, there is no way for hospitals to even guesstimate the effect of the patient mix adjustment on their results.

Mode Adjustment. Depending upon the send and response mode that a hospital utilizes for the patient satisfaction survey, will determine the adjustment applied uniformly to the patient mix adjusted composite scores. Mode refers to the method by which your hospital administered the survey – through mail, telephone, Active IVR, and mixed mode. Interactive IVR refers to a live telephone interviewer working with a patient responding to questions using their telephone key pad. Mixed mode refers to a mail questionnaire with a telephone follow up if needed.

It is important to note that the mode adjustment is applied after the patient mix adjustment is applied and therefore just about impossible to determine the level of accuracy for any of the results.

For example, hospitals that utilize telephone mode will have a percent reduction from their PMA adjusted composite scores ranging from 1.3% to as high as 6.3%. Hospitals utilizing a mixed mode will have a percent reduction to their PMA adjusted composite scores ranging from 0.3% to 3.1%. Hospitals utilizing Active IVR mode will have a percent reduction to their PMA adjusted composite scores ranging from 0.3% to 10.2%. Please note that the mail mode is the reference mode and as a result remains neutral with no adjustments to the PMA adjusted composite score.

IHA has also raised concerns about this given the wide variation in adjustments made to varying composite scores based upon the survey mode utilized by the hospital.

To better understand the background on these adjustments and to view the individual composite adjustments made to the PMA adjusted composite results, click here to view the CMS white paper on the adjustment methodology.

We will keep you posted on developments. Thank you all for your assistance in identifying problems and issues with the HCAHPS while it is still under hospital review.

Staff Contact: Pat Merryweather