JUDGING PERFORMANCE IMPACT OF IT: DON'T OVERLOOK ACTUAL USAGE, SAYS O.R. STUDY (April 21, 2003)

The study shows a link between higher usage levels of a Decision Support System (DSS) – a standard tool made available by operations researchers – and quality and profitability benefits such as lower patient death rates and increased hospital profitability.

"For example, our findings indicate that if managers execute and use the analysis from 10 additional DSS reports per month, the hospital can increase revenues by over $140 per patient," says Rajiv Kohli of the University of Notre Dame. "In today’s healthcare environment, where profits are razor-thin, all of this could be additional profit. This is one more way for managers to get value from information technology in which they have already invested."

Furthermore, co-author Sarv Devaraj adds, "we observe a definite reduction in mortality rate. Our research shows that lives are being saved."

Kohli cautions that "usage of the DSS analysis must be translated into action on the hospital floor to ensure financial and quality-based value."

"Performance Impacts of Information Technology: Is Actual Usage the Missing Link?" is by Prof. Kohli and Sarv Devaraj of the University of Notre Dame. It appears in the current issue of the INFORMS journal Management Science. A summary of the study can be found online at http://www2.informs.org/Press/ITUsageabstract.pdf

Breaking with Old Ways of Measuring Payoff
With increased investment in technology, managers are pressured to provide economic justification for their spending. A surge in the number of studies that examine IT’s payoff reflects academia’s attempt to provide this justification.

The authors fault earlier IT studies that focus on technology investment only: merely examining the dollars spent, they say, may not accurately reflect the effectiveness of IT. Their study offers an alternative.

In their usage study, the authors evaluate the success of technology based on three measures: patient mortality, which attests to a hospital’s quality and success treating patients; and two measures of profitability, revenue per admission and revenue per day.

The research took the form of a longitudinal study of a health care system comprising eight hospitals. They chose to study data over an extended period rather than over the short term of most studies because they were concerned that researchers were overlooking impacts that may not become manifest until some time later.

The hospitals included in the study are members of a private health system. The system is a national organization with member hospitals in various markets across the US. The member organizations are suburban, mid-sized hospitals with over 4,000 combined beds, 20,000 employees, and annual operating revenues of about $1.5 billion.

Data were collected for 36 monthly periods from the eight hospitals in the health system that had implemented and utilized a Decision Support System, or DSS.

The authors measured actual usage of the strategic IT by examining the total number of reports executed by a user; Central Processing Unit (CPU) time, which indicates intensity of usage; and the number of records accessed for each report.

The researchers controlled for other variables, including patient demographics and the size of hospital staff.

Role of Decision Support Systems
The researchers examined DSS because hospitals are increasingly relying on the operations research tool to improve health care and economic performance.

Clinicians and risk managers analyze competing patient treatment plans through outcomes such as length of stay, readmission rates, and mortality, and identify those with better-than-expected outcomes.

Administrators use DSS to analyze contracts by comparing costs of expected services and expected payments from insurers, then identifying areas of cost cutting and operations improvements necessary for financial viability.

The Institute for Operations Research and the Management Sciences (INFORMS®) is an international scientific society with 10,000 members, including Nobel Prize laureates, dedicated to applying scientific methods to help improve decision-making, management, and operations. Members of INFORMS work in business, government, and academia. They are represented in fields as diverse as airlines, health care, law enforcement, the military, the stock market, and telecommunications. The INFORMS website is at http://www.informs.org.