Knowing what to measure: by investing in a new clinical information system, one organization was able to improve patient care without adding staff while also increasing efficiency and productivity

In theory, calculating return on investment should be a simple exercise: add up the benefits realized and subtract the cost of implementation. Truly, that part of the exercise is simple enough. The difficult part comes much earlier: determining what metrics to use to quantify the benefits. Is saving costs the most relevant factor? What about measurement of quality or patient saris faction? What a re reasonable proxies for quantifying those intangibles?

Barger Clinic, part of the Peace Health integrated delivery system in Eugene, Ore., confronted these questions when it implemented an electronic medical record (EMR) as part of a larger organizationwide technology initiative at PeaceHealth. PeaceHealth's vision was to create a seamless continuum of care for its patients supported by an organizationwide clinical information system (CIS) and comprehensive, lifetime electronic patient records across its five hospitals in Oregon, Washington, and Alaska and its ambulatory facilities. The care team would benefit front access to the entire medical record, whether patients were seen in a main clinic, a satellite clinic, or a hospital emergency department. Clinicians would have access to information from both ambulatory visits and inpatient care. Likewise, all financial and demographic information would be available when patients arrived for an appointment.

When adoption of the EMR was first proposed in 1999, Barger's leadership was concerned about red ink. The CIS was relatively new to the organization and to PeaceHealth's ambulatory environment, specifically in a clinic that sees more than 3,000 patients per month in every specialty, from obstetrics to gerontology. Effective measurement of the system's performance was crucial to gaining support from PeaceHealth's senior management.

Deciding What to Measure

PeaceHealth leadership prescribed four criteria for the success of the project. The CIS must:

* Be used and valued by physicians and nurses

* Improve patient satisfaction--or at a minimum, avoid adverse impact

* Be cost-neutral or better

* Improve provider productivity--or at a minimum, avoid adverse impact

To be successful, a CIS needs to have a positive impact on patient care, improve operational efficiencies, and support clinicians in their work. If clinicians don't see the value of new technology, they won't keep using it. By beginning with the end in mind--defining the clinical benefits the organization wishes to achieve--a healthcare provider is better positioned to design its CIS and measure value once the new CIS is in place.

To this end, the clinic formed the Barber Implementation Group, a multidisciplinary team of physicians, administrative leadership, nurses, and information technology staff. Working with the CIS vendor, the team developed measurement criteria, redesigned processes, created training materials, and led the system implementation.

The group first identified key process indicators (KPIs) that could be improved by the CIS implementation. KPIs are tangible, quantifiable measures that tie back to the organization's goals for process improvement, and are measured before, during, and after a technology installation. In this case, KPIs were linked to the four high-level success criteria established by Barber clinic leadership. The group defined KPIs related to speed and efficiency of patient visits, reduced volumes of paper documents, reduction in potential adverse drug events (ADEs), and reduced costs based on better use of clinician amt staff time. (Before installation of the CIS, clinicians and health information management staff spent significant time tracking down patient information.)

Next, the group determined a baseline measurement for each KPI, and compared those measurements with industry benchmarks and best practices. Finally, the entire team agreed on a measurement timeline and hammered out the logistics of the measurement tools to be used. Most KPIs were measured before implementation to establish a baseline and at 45 and 90 days after go-live to assess improvement.

Measurement techniques do not have to be arcane or elaborate. To measure its KPIs, the Barger team used straightforward techniques. The team learned that efficiency and productivity could be evaluated fairly easily, without complex time-and-motion studies. For example, timely access to patient information was an important issue for both clinicians and administrative staff. To measure the backlog of documents waiting to be filed in patient charts, rather than count every sheet, clinic staff measured the depth of the stacks of paper. Incomplete patient information was another concern for Barger clinicians. Again, complex measurement procedures were not required. Clinic staff attached a short form to every chart, and physicians were asked to indicate whether all the information needed for the patient's appointment was included.

Analysis in Action

The clinic converted a significant amount of patient information to an electronic formal with out going entirely paperless. The conversion brought positive results in several areas, as measured by the project team.

Availability of electronic information reduces dependence on paper. Before implementing the EMR, the stacks of paper to be filed in medical records totaled almost two feet. Within a few months of implementing the EMR, the clinic reduced the paper backlog about 90 percent, to a couple inches or less. The primary change was the elimination of clinic transcription from that pile; transcribed documents are now exclusively available electronically.

Electronic availability of test results (such as lab and radiology information) was another improvement. No longer did clinic staff have to send hard copies of test results to physicians for review before adding the information to the patient's chart.

More complete patient information supports more productive patient appointments. Before EMR implementation, patient information was present 85 percent of the time. In other words, physicians would have to interrupt an appointment for one out of every seven patients and send someone to find missing data. The time spent looking for missing data was six to seven minutes per patient on average, but sometimes reached half an hour.

Within one year of the EMR go-live, the clinic reduced the incidence of missing information to one out of 50 patients. A family practitioner seeing two dozen patients a day now saves enough time to fit in one additional appointment, effectively demonstrating the value of the CIS to clinicians. In addition, because charts are more complete, practitioners have information readily available during patient appointments. More complete patient charts have led to an increase in efficiency and productivity.

Reducing administrative burdens creates more time for patient care. Achieving greater efficiencies through the use of a CIS requires changing processes, not just replicating manual processes electronically. The Barger team's measurement of patient cycle times documented the time savings achieved through process change. With the CIS, nurses had to key in information from patient intake, which required more time for the intake process. However, use of the CIS relieved nurses from having to call in prescriptions to the pharmacy, saving time during patient checkout. As a result, the time nurses spent on patient appointments declined, supporting more efficient use of clinicians' time.

New processes help prevent ADEs. The Barger team measured the number of potential ADEs to show the impact of the EMR on quality of care. Several months before go-live, clinic physicians received carbon-copy prescription pads. Clinic staff collected a copy of every prescription issued and reviewed the copies for legibility, appropriate indication for the medication, duplicative therapy, correct dosing, and the presence of an allergy or drug--drug interaction with another medication in the patient's chart. Even with physician awareness of the test, potential ADEs existed in 6 percent of the prescriptions. Six weeks after go-live, the same review was performed, only this time the study was blinded because staff could pull records directly from the system without alerting the prescribing physicians. The number of potential ADEs had dropped to less than 1 percent.

EMR use contributes to patient and provider satisfaction. Patient and provider satisfaction rose after implementation of the EMR. Both physicians and nurses reported increased satisfaction with the CIS. Contributing factors included better access to patient information, improved capabilities for identifying patients about recalled medications or new treatments for their particular conditions, and simpler prescribing.

Putting It All Together