|
EDITORIAL
Medicare
Quality Improvement,
Bad Apples or Bad Systems?
JAMA Editorial, January
15, 2003
By David C. Hsia, JD, MD, MPH
The Quality Improvement
Group at the Centers for Medicare and Medicaid Services leads the quality improvement
organizations (QIOs, formerly the PROs [peer review organizations], PSROs [professional
standards review organizations], EMCROs [experimental medical care review organizations],
etc),1 and according to the results of a study by Jencks and colleagues2
in this issue of THE JOURNAL, their leadership is effective. No other US organization
measures quality at the hospital level. The QIO program uses 24 quality indicators
that have strong evidence to support them. Jencks et al report that between 1999
and 2001, the proportion of Medicare patients receiving appropriate care improved
from 70% to 73% on average, although this rate varied widely across states and
by indicator.2 Their analysis is valid, robust, understandable, and
correct. For the 1999-2002 QIO contract cycle, Centers for Medicare & Medicaid
Services required all QIOs to improve quality in 5 clinical areas (acute myocardial
infarction, heart failure, pneumonia, surgical infection, and outpatient diabetes),
not just to passively review charts.3 The QIO quality indicators address
some aspects of suboptimal quality, but others remain.
As summarized
in the Institute of Medicine's recent reports on medical errors, a diverse literature
describes the imperfect state of health care quality.4 The Institute
of Medicine asserts that medical errors kill more people in the United States
each year than motor vehicle crashes.5 For complex reasons, existing
systems of quality assessment, review, and improvement function suboptimally.6
A critical issue
is whether these errors represent failures of humans or systems. Peer review,
malpractice litigation, medical licensing, medical disciplinary actions, insurer
audit, governmental investigation, and most other quality assurance systems rely
on retrospective review. Examining patient charts assumes that error derives from
failure on the part of an incompetent or careless individual. Adverse events therefore
identify bad apples for removal.7 This inspection model ("name, blame,
shame") seeks to improve quality by cutting off one tail of the bell-shaped curve
of human performance.
In contrast, Deming's
continuous quality improvement (CQI) model assumes that most adverse events represent
system failures and that design of work processes should detect and eliminate
the human error that inevitably occurs.8 Industrial quality control
statistically analyzes all outcomes for systems improvement opportunities rather
than searching for single events that purportedly demonstrate individual error.
The CQI model seeks to improve quality by moving the entire bell curve to the
left.
Unfortunately,
the CQI initiative has not yet attained full acceptance by the general public.
The name-blame-shame model produces readily understandable headlines, but it does
not methodically eliminate errors to improve statistical outcomes. Yet even if
every worker in a health care system could do his or her job perfectly, most events
that are considered to be errors would still occur. Although organizations like
the Institute for Healthcare Improvement have led the effort to extend the CQI
initiative into health care,9 the recent survey by Blendon et al10
makes it clear that neither members of the public nor physicians appreciate that
poor systems cause most errors.
According to the
classic Donabedian model,11 health care quality is organized as structure,
process, or outcome. Structure refers largely to the paper qualifications
of the practitioner or institution (eg, licensed, board certified, insured, or
inspected by the Joint Commission on Accreditation of Healthcare Organizations).
Process refers to how the practitioner delivers care (eg, drug X was indicated
and prescribed). Outcome refers to what happened subsequently to the patient
(eg, felt better, returned to work, died).
At present, all
organizations use process measures for quality review. The QIOs surpass other
organizations by using validated measures and in aggregating at the hospital level.
To secure hospitals' cooperation, the QIOs do not publish their hospital-level
results. Rather, these results guide the QIOs in targeting technical assistance
to improve quality.12
Levels of aggregation
at the regional or state level lack sufficient detail to identify opportunities
for quality re-engineering within a hospital. The upcoming Agency for Healthcare
Research and Quality (AHRQ) national quality reports will measure state and national
performance with which hospitals can compare their performance. The National Committee
for Quality Assurance report cards give Health Employer Data and Information Set
(HEDIS)13 and Consumer Assessment of Health Plans14 quality
indicators for single health plans.15 However, low-scoring plans have
sometimes terminated their public reporting of National Committee for Quality
Assurance (NCQA) results.16 The NCQA has proposed hospital-level report
cards, and the National Quality Forum has drafted standardized performance measures
for evaluating hospital quality.17 The American Hospital Association,
the Association of American Medical Colleges, and the Federation of American Hospitals
have recently proposed voluntary, public reporting of 10 quality indicators, a
subset of the National Quality Forum performance measures.18 Public
reports should stimulate CQI and inform patient choices.19
Theoretically,
outcomes best assess quality, but they are the most difficult to measure because
of varying inputs (eg, severity of illness, multiple comorbidities, patient compliance,
local conditions). Using processes linked to the outcomes of interest offers higher
efficiency but also lower sensitivity to differences in severity of illness process
(variability in outcomes may be caused by patient characteristics rather than
differences in quality of medical care).20 Outcome analyses also require
high volumes of detailed data to be representative across systems (including,
for example, transaction data from multiple payers). These analyses also require
longer periods to complete (eg, 5-year cancer survival), thereby preventing timely
improvements. Centers for Medicare & Medicaid Services led the use of output
measures from 1988 through 1992,21, 22 but it had to abandon them because
of political sensitivities.23
Although the quality
measures assessed vary, health services research has largely reached a consensus
on the superiority of explicit measures (comparison to an objective standard)
over implicit measures (unstructured review).24 Peer reviewers reading
the same patient records without guidance have low interrater agreement.25
Explicit review has also become largely condition specific (eg, use of -blockers
after acute myocardial infarction) rather than generic (eg, all-cause mortality).
The literature
discussing quality improvement suggests several opportunities to build on the
QIO's results. First, Medicare and other payers currently each impose their own
quality assurance programs, pulling the hospital in conflicting directions. In
addition to supporting the initiative from the American Hospital Association,
the Association of American Medical Colleges, and the Federation of American Hospitals,
Medicare could refrain from separate QIO audit of institutions attaining a satisfactory
grade and pool transaction data with other payers for quality analysis of hospitals
declining to participate.
Second, some medical
specialties could improve by following the lead of anesthesiology, which has perhaps
progressed the most in error-proofing its systems. Human factors analysis identified
high mortality due to operators' using unfamiliar ventilators and inadvertently
turning off patients' oxygen.26 Anesthesiology organizations and manufacturers
worked together to design new ventilators with standardized controls and on which
the oxygen level cannot be physically reduced below room air. These hardware changes
largely eliminated human error from the system.27, 28
Other medical specialties
that also depend heavily on medical equipment, such as radiology, pathology, blood
banking, nuclear medicine, cardiology, nephrology, and possibly ophthalmology,
should also have opportunity for error reduction through hardware and systems
reengineering. Public and private payers could lead a consortium to develop and
finance human factors analysis, device standardization, and operator training
in these and other hardware-dependent medical specialties.29
Work process design
within a health care system presents a more difficult challenge. Each department
and medical specialty functions independently, with workers exhibiting varying
degrees of individualism. For instance, diagnostic tests might get scheduled at
conflicting times, laboratory samples may disappear in transit, and pharmacists
might misread physicians' abbreviations. Interdisciplinary teams have not proved
to be a panacea.30 Hospital management needs to identify opportunities
for errors and then create work systems that prevent them. It must also resolve
the tension between the command, control, and centralization needed to change
an organization's culture and work processes vs the bottom-up approach characteristic
of the CQI model.
Extending CQI
to the physician office presents even greater challenges.31 A sole
practitioner generally lacks the resources, incentive, and objectivity to reengineer
office systems to eliminate human error. An incremental approach might first extend
error prevention systems to health care settings organized around institutions,
such as ambulatory surgery centers, nursing homes, and clinical laboratories,
with subsequent deployment to the office setting.
Several technologies
have demonstrated effectiveness in reducing health care errors including physician
order entry systems,32 pharmaceutical software (drug-drug interactions,
drug allergies, dosing),33 and decision support systems.34, 35
Potential extensions of this technology include reporting laboratory and test
results, medication tracking (bar coding of unit doses), patient and staff authentication
and location (radio frequency identification tags), and order entry and laboratory
results by wireless personal computer or personal digital assistant, as well as
making these modules interact usefully. Adoption of these innovations has proceeded
slowly.36 Likewise, the computer-based patient record has disseminated
more slowly than anticipated.37 At present, the military and the Department
of Veterans Affairs have the most advanced and successful computer-based patient
record systems.38 However, civilian hospitals typically do not exercise
the same degree of control over their workers as do federal institutions.
In conclusion,
the QIOs constitute the nation's main infrastructure for quality improvement.
Although applicable to Medicare beneficiaries, the QIOs must exert a spillover
effect on other patients. The article by Jencks and colleagues demonstrates their
success at improving quality of care and sustaining those improvements. The Institute
of Medicine reports underscore the urgency of building on the success of the QIO
program to improve the quality of health care for all patients.
Author/Article Information
Author Affiliation: Agency for Healthcare Research and Quality, US Department
of Health and Human Services, Rockville, Md.
Corresponding Author and Reprints: David Hsia, JD, MD, MPH, Agency for
Healthcare Research and Quality, 6010 Executive Blvd, Rockville, MD 20852-3809
(e-mail: dhsia@ahrq.gov).
Editorials represent
the opinions of the authors and THE JOURNAL and not those of the American Medical
Association.
Disclaimer:
The views presented in this editorial do not represent the policy of any US governmental
agency.
Acknowledgment:
AHRQ librarians Deby K. Blum, MLS; Renee A. W. McCullough, MHum, MLS; and Lynette
F. Lilly, BA, helped obtain references cited in this editorial.
REFERENCES
1.
Sprague L.
Contracting for Quality: Medicare's Quality Improvement Organizations.
Washington, DC: National Health Policy Forum; 2002. Issue Brief No. 774.
2.
Jencks SF, Huff ED, Cuerdon T.
Change in the quality of care delivered to Medicare beneficiaries, 1998-1999 to
2000-2001.
JAMA.
2003;289:305-312.
ABSTRACT | FULL TEXT | PDF
3.
Quality Improvement Organizations (QIOs) statement of work.
Centers for Medicare & Medicaid Services Web site.
Available at: http://www.cms.hhs.gov/qio/2.asp. Accessed December 27, 2002.
4.
Committee on Quality of Health Care in America, Institute of Medicine.
Crossing the Quality Chasm: A New Health System for the 21st Century.
Washington, DC: National Academy Press; 2001.
5.
Kohn LT, ed, Corrigan JM, ed, Donaldson MS, ed.
To Err Is Human: Building a Safer Health System.
Washington, DC: National Academy Press; 2000:1.
6.
Hsia DC.
DHHS Office of Inspector General: an overview.
R I Med J.
1989;72:47-55.
MEDLINE
7.
Berwick DM.
Continuous quality improvement as an ideal in health care.
N Engl J Med.
1989;320:53-56.
MEDLINE
8.
Deming WE.
Out of the Crisis.
Cambridge, Mass: MIT Press; 2000.
9.
Institute for Healthcare Improvement Web site.
Available at: http://www.ihi.org. Accessed December 23, 2002.
10.
Blendon RJ, DesRoches CM, Brodie M, et al.
Views of practicing physicians and the public on medical errors.
N Engl J Med.
2002;347:1933-1940.
MEDLINE
11.
Donabedian A.
The Definition of Quality and Approaches to Its Assessment.
Ann Arbor, Mich: Health Administration Press; 1980.
12.
Berwick DM.
Public performance and the will for change.
JAMA.
2002;288:1523-1524.
FULL TEXT | PDF | MEDLINE
13.
Health Plan Employer Data and Information Set.
Available at: http://www.ncqa.org/Programs/HEDIS/. Accessibility verified December
27, 2002.
14.
Consumer Assessment of Health Plans.
Agency for Healthcare Research and Quality Web site. Available at: http://www.ahcpr.gov/qual/cahpsix.htm.
Accessibility verified December 27, 2002.
15.
National Committee for Quality Assurance Web site.
Washington, DC. Available at: http://www.ncqa.org/. Accessed December 23, 2002.
16.
McCormick D, Himmelstein DU, Woolhandler S, Wolfe SM, Bor DH.
Relationship between low quality-of-care scores and HMOs' subsequent public disclosure
of quality-of-care scores.
JAMA.
2002;288:1484-1490.
ABSTRACT | FULL TEXT | PDF | MEDLINE
17.
National Quality Forum Web site.
Washington, DC. Available at: http://www.qualityforum.org/. Accessed December
23, 2002.
18.
Hospitals to share quality information [news release].
Washington, DC: American Hospital Association, December 12, 2002. Available at:
http://www.hospitalconnect.com/aha/press_room-info/
releasedisplay.jsp?dcrpath=AHA/Press_Release/data/
PR_021212_Quality&domain=AHA. Accessed December 23, 2002.
19.
Marshall MN, Shekelle PG, Leatherman S, Brook RH.
The public release of performance data: what do we expect to gain?
JAMA.
2000;283:1866-1874.
ABSTRACT | FULL TEXT | PDF | MEDLINE
20.
Mant J, Hicks N.
Detecting differences in quality of care: the sensitivity of measures of process
and outcome in treating acute myocardial infarction.
BMJ.
1995;311:793-796.
MEDLINE
21.
Hartz AJ, Krakauer H, Kuhn EM, et al.
Hospital characteristics and mortality rates.
N Engl J Med.
1989;321:1720-1725.
MEDLINE
22.
Krakauer H, Bailey RC, Skellan KJ, et al.
Evaluation of the HCFA model for the analysis of mortality following hospitalization.
Health Serv Res.
1992;27:317-335.
MEDLINE
23.
Wilensky GR.
Contracting for quality: Medicare's quality improvement organizations.
Presented at: the National Health Policy Forum; September 27, 2002; Washington,
DC.
24.
McGlynn EA, Brook RH.
Evaluating the quality of care.
In: Andersen RM, Rice TH, Kominski GF, eds. Changing the US Health Care System:
Key Issues in Health Services, Policy, and Management. 2nd ed. San Francisco,
Calif: Jossey-Bass; 2001:150-182.
25.
Clement DG, Retchin SM, Brown RS, Stegall MH.
Access and outcomes of elderly patients enrolled in managed care.
JAMA.
1994;271:1487-1492.
MEDLINE
26.
Gawande A.
When doctors make mistakes: the real problem isn't how to stop bad doctors from
harming, even killing, their patients; it's how to prevent good doctors from doing
so.
New Yorker.
1999;74:40-55.
MEDLINE
27.
Bates DW, Gawande AA.
Error in medicine: what have we learned?
Ann Intern Med.
2000;132:763-767.
MEDLINE
28.
Cooper JB, Gaba D.
No myth: anesthesia is a model for addressing patient safety.
Anesthesiology.
2002;97:1335-1337.
MEDLINE
29.
Corrigan JM, ed, Eden J, ed, Smith BM, ed.
Leadership by Example: Coordinating Government Roles in Improving Healthcare
Quality Programs.
Washington, DC: Institute of Medicine; 2002.
30.
Womer RB, Tracy E, Soo-Hoo W, Bickert B, DiTranto S, Barnsteiner JH.
Multidisciplinary systems approach to chemotherapy safety: rebuilding processes
and holding the gains.
J Clin Oncol.
2002;20:4705-4712.
MEDLINE
31.
Schyve PM.
Statement of the Joint Commission on Accreditation of Healthcare Organizations
at the National Summit on Medical Errors and Patient Safety Research.
September 11, 2000. Available at: http://www.jcaho.org/accredited+organizations/
ambulatory+care/sentinel+events/position+statements/
summit+on+medical+errors.htm. Accessed December 23, 2002.
32.
Bates DW, Leape LL, Cullen DJ, et al.
Effect of computerized physician order entry and a team intervention on prevention
of serious medication errors.
JAMA.
1998;280:1311-1316.
ABSTRACT | FULL TEXT | PDF | MEDLINE
33.
Gandhi TK, Weingart SN, Seger AC, et al.
Impact of basic computerized prescribing on outpatient medication errors and adverse
drug events.
J Am Med Inform Assoc.
2002;9(suppl 6):S48-S49.
34.
Hunt DL, Haynes RB, Hanna SE, Smith K.
Effects of computer-based clinical decision support systems on physician performance
and patient outcomes: a systemic review.
JAMA.
1998;280:1339-1346.
ABSTRACT | FULL TEXT | PDF | MEDLINE
35.
Bates DW, Spell N, Cullen DJ, et al.
The cost of adverse drug events in hospitalized patients.
JAMA.
1997;277:307-311.
MEDLINE
36.
Ortiz E, Meyer G, Burstin H.
Clinical informatics and patient safety at the Agency for Healthcare Research
and Quality.
J Am Med Inform Assoc.
2002;9(suppl 6):S2-S7.
37.
Dick DS, ed, Steen EB, ed, Detmer DE, ed.
The Computer-Based Patient Record: An Essential Technology for Health Care.
Rev ed. Washington, DC: Institute of Medicine; 1997.
38.
Lovis C, Payne TH.
Extending the VA CPRS electronic patient record order entry system using natural
language processing techniques.
Proc AMIA Symp.
2000;:517-521.
MEDLINE
|
