ROI for Using Data Standards Case Study

Return-On-Investment For Using Data Standards: A Case Study Of New York State's Data System

On September 4, 2001 the Governor of New York State signed into law a bill passed unanimously by both branches of the state legislature mandating the annual collection of all emergency department visits in hospitals regulated by the state of New York. In establishing the new law, the legislature did not appropriate funds for the development and implementation of the data collection system. Rather, the system’s development relied heavily on the use of existing resources.

Using data standards to develop the emergency department data collection system facilitated the design and development processes. Intuitively the use of data standards to develop new or enhanced data systems should provide: 1) a cost effective roadmap to implementation; 2) simplify the development process; and 3) strengthen the relationships between data suppliers and data users. For the purposes of this case study, the author asserts that achieving success in each of these three areas contributes to a positive return-on-investment (ROI) for a project. This case study documents the time and resources required to develop the New York State Emergency Department Data Collection System. It is meant to show a positive financial return on investment, an implementation process that adhered to timelines, and a strengthened relationship between the NYS DOH, the agency that manages these data, and the hospitals in New York State.

The New York State discharge data system (Statewide Planning Research and Cooperative System – SPARCS) began collecting inpatient data in the late 1970s. Developed with federal monies, the system served as the first national test of using a standard uniform bill that (1) would be used by all payers in New York State, and (2) would be reported to SPARCS. New York State has had a long history of using data standards. When the state billing form (UBF-1) was replaced by a national uniform bill (UB-92), the payer and provider communities agreed to follow the standards path. Consistent with historical precedent, the New York State Department of Health (NYS DOH) provided leadership in the migration from a state-specific billing form to a national uniform bill.

The latest standards migration that resulted from the new legislative mandate was driven by the Health Insurance Portability and Accountability Act (HIPAA) transactions and codes final rule. The two hospital associations in New York State – Healthcare Association of New York State (HANYS) and the Greater New York Hospital Association (GNYHA) – both indicated support for the legislation on the condition that the new data collection system use the transactions and codes standards mandated by HIPAA. It is not surprising that with their long history of adapting to trends in data standards, the provider community insisted that the SPARCS system should also conform to those same data standards. Much of the success of New York State’s new data can be attributed to the use of HIPAA transactions and codes data standards.

Data standards prevent information technology project failures. Using the search criteria “information technology project failures” on an Internet search results in many hits. One article cited the top 10 reasons for information technology project failures:(1)

Another article emphasized how common project failures are. For example, “The average project was 189% over budget and 222% behind schedule and contained only 61% of the originally specified features.”(2)

Exhibit 1 below outlines how each of the above mentioned reasons for information technology project failures were addressed in the New York State Emergency Department Data Collection System design. As suggested previously, the scope of New York State’s information technology project was confined and defined by the HIPAA transactions and codes data standards. In the development of New York State’s Emergency Department Data Collection System, 6 of the 10 reasons cited for information technology failure were addressed by having a definitive set of standards and operating within the boundaries of those standards. Any requests for data elements not supported by the standards were postponed to later phases of implementation, after the initial system was operational with the existing data standards.

Exhibit 1: Reasons for Information Technology Failures and How They Were Addressed
by New York State’s Emergency Department Data System Design

Reason for Failure	How addressed by New York State
Lack of user input	User input was solicited while legislation was being debated and continued throughout the entire design, development and implementation phases
Incomplete requirements and specifications	HIPAA transactions and codes standards provided necessary structure
Changing requirements and specification	HIPAA transactions and codes standards provided necessary structure
Lack of executive support	Legislative mandate
Technology incompetence	Existing staff highly competent
Lack of resources	Controlled by limiting scope to available resources
Unrealistic expectations	Controlled by limiting scope to available resources
Unclear objectives	Controlled by limiting scope to available resources
Unrealistic time frames	Controlled by limiting scope to available resources
New technology	No new technology would be used that would overextend existing resources

Part of the expenses incurred by the NYS DOH were those necessary to adapt the HIPAA transactions and codes standards for use by a state discharge system. Since a standard implementation strategy for this purpose did not exist, management at the NYS DOH committed staff and travel resources to enable staff to fully participate in the national standards development process, which resulted in the publication of a nationally balloted, HIPAA-compatible implementation guide. The Health Care Service: Data Reporting guide is a technical document detailing how to use the HIPAA claim format (837) for reporting state discharge data. Data overlapping with claim requirements are identical to those mandated by the HIPAA legislation. Additional data needs conform to the ANSI ASC X12 standards upon which all uses of the 837-claim format were built. The guide was developed with input from many states and was written for national use.

The expenses described below would not necessarily be incurred by other states attempting to implement a similar data collection system. However, these expenses are included in this analysis to provide a complete picture of the work necessary to implement an information system using national data standards in New York State.

The NYS DOH participated in the development of national standards. Over a four-year period, one staff person participated in trimester meetings held by the ANSI ASC X12. Participants in these meetings were given responsibility for developing the consensus-based transactions currently mandated in the HIPAA legislation. The task of the New York State DOH staff person at these meetings was to build consensus for the development of what would become the Health Care Service: Data Reporting guide. The result of this effort was the approval and publication of the guide to serve as a standard for collecting discharge data. This technical document served as a definitive source for defining system requirements and specifications of the New York State Emergency Department Data Collection System. The hospital associations in New York State approved use of the Health Care Service: Data Reporting guide as a conformance document with the HIPAA mandates.

This same NYS DOH staff person also participated in meetings of the National Uniform Billing Committee (NUBC) and was eventually named a voting member of that group. The NUBC was named by the Secretary of Health and Human Services as a data content advisor. In that role, the NUBC was a significant player in determining the allowable content in the HIPAA standards. That being the case, meeting the data reporting needs in conformance with HIPAA required working closely with the NUBC, which holds four two-day meetings per year.

Below in Exhibit 2 are the estimated cost figures for staff to participate in the national standards development activities.

Exhibit 2: Estimated Costs to Participate in National Standards Development
(2000 – 2003)

ANSI ASC X12 - Estimated Costs
3 meetings per year x 4 days per meeting x 4 years	48 days
Daily salary rate	$400
Total estimated salary to attend ANSI ASC X12 meetings	$19,200
Travel expenses for each meeting (approximate)	$1,500
Total travel expenses: 3 meetings a year x 4 years	$18,000
Total ANSI ASC X12 expenses	$37,200
NUBC - Estimated Costs
4 meetings per year x 2 days per meeting x 4 years	32 days
Daily salary rate	$400
Total estimated salary to attend NUBC meetings	$12,800
Travel expenses for each meeting (approximate)	$1,000
Total travel expenses for 4 meetings a year x 4 years	$16,000
Total NUBC expenses	$28,800
Total Estimated Expenses to Participate in National Standards Work	$66,000

Notes: The daily rate is based on the actual salary of the staff person assigned this work, and does not include increments to account for non-salaried benefits.

A strengthened relationship with the state’s two hospital associations reflected a positive return on the NYS DOH’s investment to participate in standards development. After the NYS DOH staff person was named as a voting member to the NUBC, the relationship between the NYS DOH and the hospital associations in New York State was greatly enhanced. Having another New York State voting voice on this committee was a huge benefit to data users and suppliers. That staff person served as a resource and meeting facilitator who, at events sponsored by the hospital associations, addressed the New York State data needs that were affected by decisions made by the National Uniform Billing Committee.

The system design phase included designing the system and educating the potential data suppliers (i.e., the hospitals, and data users). For the most part, the relationship between the users and suppliers of the data was based more on impressions than actual experiences and interactions. Timely and successful completion of the project with the collection of high quality, useful data depended on developing a foundation of trust and mutual understanding between data users and data suppliers. The challenge of limiting the scope of the system to available resources could only be addressed by an educational effort that would foster mutual understanding of each constituent’s needs and capabilities. Specifically, the users of the data needed to be educated about the existing robustness of the HIPAA standard and the supporting provider information systems. The suppliers of the data needed to be educated about the reasonability of the data users’ needs. During this consensus building process, both stakeholders discovered that needs could be met without major infrastructure changes.

The effort to educate data users and data suppliers began with the formation of an internal NYS DOH Work Group charged by the Department’s executive staff with the responsibility of developing the Emergency Department Data Collection System design documentation. A plan was developed to conduct state outreach meetings for the affected providers and potential outside users of the data, for the purpose of presenting the system design plan and soliciting comments. It is important to note that the purpose of these outreach meetings was not to present a final plan for collecting emergency department data. It was made clear that final plans would incorporate input solicited from the outreach meetings. To stimulate the necessary participation, these meetings held regionally around the state were sponsored and advertised by the two hospital associations in New York State and, in one instance, the New York State Chapter of the College of Emergency Department Physicians.

After the system design was finalized with input from both the provider and user communities, the New York State Health Information Management Association (NYHIMA) sponsored another set of regional statewide educational sessions. The members of NYHIMA hold most of the responsibility for coding and submitting the required emergency room data. This round of statewide meetings again was supported by New York State DOH’s executive staff. The system design documentation is available on the SPARCS Web site at:

Below in Exhibit 3 are the estimated cost figures for DOH staff to participate in the internal Emergency Department Work Group activities. Included in the cost estimates for the system design phase of this project are cost estimates

The NYS DOH teams that conducted the regional outreach meetings consisted of two staff persons from the SPARCS unit (always the same individuals) and two members of the internal Work Group that represented the potential users of the data. This mix of staff was able to address provider questions, comments on the proposed system design and provide justifications for how the data were intended to be used.

Internal Work Group Meeting time costs
8 meetings lasting 2 hours each with 15 attendees	240 hours
Average hourly rate	$50
Total estimated salary to attend ANSI ASC X12 meetings	$12,000
Specification development costs
3 months 1.5 FTE (37.5 hours per week)	675 hours
Average hourly rate	$50
Total estimated salary to attend ANSI ASC X12 meetings	$33,750
Outreach staff and travel costs
6 out-of-town meetings with 4 DOH staff attending	180 hours
Average hourly rate	$50
Total estimated salary for out-o- town outreach meetings	$9,000
Travel expenses for each meeting	$500
Estimated out-of-town travel expenses for 6 meetings	$3.000
2 in-town meetings (4 hours) with 6 DOH staff attending	48 hours
Average hourly rate	$50
Total estimated salary to attend in-town outreach meetings	$2,400
Total Estimated NYS DOH Work Group Costs	$60,150

Notes: The daily rate is based on an average hourly rate of staff persons assigned this work. It does not include increments to account for non-salaried benefits. A 7.5-hour work day was used in the calculations.

The next step in the process was to conduct a set of regional training sessions. These meetings were sponsored by NYHIMA. As stated earlier, this organization represents the hospital staff most likely responsible for coding and transmitting the emergency department data to the state. There were 6 meetings conducted by SPARCS staff.

Below in Exhibit 4 are the estimated cost figures for NYS DOH staff to conduct training for the Emergency Department Data Collection System.

Out-of-town training
6 out-of-town meetings conduct by 2 SPARCS Staff	90 hours
Average hourly rate	$50
Total estimated salary to attend ANSI ASC X12 meetings	$4,500
Travel expenses for each meeting	$300
Estimated out-of-town travel expenses for 6 meetings	$1,800
In-town training
2 in-town meetings (4 hours) with 3 DOH staff attending	24 hours
Average hourly rate	$50
Total estimated salary to attend in town outreach meeting	$1,200
Total Estimated ED Training Costs	$7,500

Notes: The daily rate is based on an average hourly rate of staff persons assigned this work. It does not include increments to account for non-salaried benefits. The work day used in the calculation was 7.5 hours.

Developing consensus around the system design took longer early in the design process but proved beneficial in the long-run. System design documentation developed in an environment that balanced the needs and capabilities of both users and suppliers was less subject to change once the development work began.

The on-time completion of the activities subsumed in the design phase of the project reflect a significant success.

The most significant ROI from the approach for designing the system was the improved relationship between the data suppliers and data users. Questions in both directions could be addressed directly rather than mediated by the data collection unit (SPARCS). Clearly, the lack of contention by the hospital associations and the user community over the final specifications documentation is a measure of the success of this methodology for developing the system design, which intended to balance the needs of the users with the capabilities of the providers.

Through the outreach meetings, the provider community delivered some clear messages that affected the system design and development processes. Prior to passage of the legislation authorizing the collection of the emergency department data, the SPARCS system required providers to submit data for inpatient discharges and ambulatory surgery visits. Provider input during the outreach meetings indicated that emergency department visits for patients not admitted as inpatients were tracked as part of the same outpatient hospital systems that reported ambulatory surgery visits. The coding and guidelines for submitting data on emergency department and ambulatory surgery visits were in essence the same. However, the rules for submitting inpatient discharges were different, which meant that data for patients who were admitted as inpatients from a hospital’s emergency department had to follow the inpatient coding and submissions rules. Nonetheless, the SPARCS inpatient data collection system data content had previously been converted to be compatible with HIPAA standards. Consequently, no changes were necessary for providers to report those emergency department patients to SPARCS that were admitted as inpatients.

Because data on ambulatory and emergency department visits were generated from the same hospital outpatient information systems, it was necessary to change the existing ambulatory surgery data collection system. In addition, the SPARCS system had to be developed in a way that could receive ambulatory surgery and emergency department data in one submission file or in separate submission files. The provider community advised that ambulatory surgery and emergency department visits should be distinguished from one another by use of national UB-92 revenue codes.

The system development plan was to make the necessary changes to the existing SPARCS ambulatory surgery data collection system to align it with the emergency department data collection system. Those changes were tested and implemented on the existing ambulatory surgery data collection system prior to initiating efforts to collect and edit emergency department data. That first phase of development tested changes to the input and output formats as well as collection of any new data elements necessary for collecting emergency department data. That work took two in-house programmers four months to complete.

Because the changes to the ambulatory surgery data collection system changed the output format, implementation of Phase One required one additional task prior to the scheduled implementation date of January 1, 2003. By request of the data users, prior years’ data stored in the old format had to be migrated to the new format to enable easier cross-year analysis of SPARCS outpatient data. This task took one programmer one month to complete.

The second phase of system development, which focused on programming for data collection and editing the emergency department data, took two in-house programmers three months to complete. It is important to note that during the coding, testing, and implementation phases of this project there were no additional requests for data elements to be added to the system specifications document. Several hospitals volunteered to beta test the changes to the ambulatory surgery system and the emergency department data collection system. The only significant development challenges encountered by the beta testers were related to the validity of the edits. These problems were resolved relatively easily during the beta testing phase of the project when the reality check of “live” data was applied to the beta system.

Below in Exhibit 5 are the estimated cost figures for New York State DOH staff to develop and conduct training for the emergency department data collection system.

Phase One Programming
2 programmers working for 4 months each	1200 hours
Average hourly rate	$40
Total estimated salary complete phase one	$48,000
Phase Two Programming
2 programmers working for 3 months each	900 hours
Average hourly rate	$40
Total estimated salary complete phase two	$36,000
Old File Conversion Task Programming
1 programmer working for 1 month	150 hours
Average hourly rate	$40
Total estimated salary complete migration task	$6,000
Total Estimated System Development Costs	$90,000

Notes: The daily rate is based on an average hourly rate of staff persons assigned this work. It does not include increments to account for non-salaried benefits. The work week used in the calculation was 37.5 hours.

It is important to note that all system design and development for this project was completed on time or in advance of the legislation’s mandated collection date. See Exhibit 6 for a summary of total project costs.

Total Estimated Expenses to Participate in National Standards Work	$66,000
Total Estimated NY DOH Work Group Costs	$60,150
Total Estimated ED Training Costs	$7,500
Total Estimated System Development Costs	$90,000
Total Estimated Project Costs	$223,650

The staff time included in the estimates in the exhibits above came from existing SPARCS staff, as well as existing user bureau staff (i.e., staff from bureaus who participated in the development who use data, not collect data); the state legislation mandate did not allocate additional funding for new staff. It has been estimated by the hospital industry that there are about 6.5 million emergency department visits each year in New York State, which would be added to the approximate 2.5 million inpatient discharges and the 1.2 million ambulatory surgery visits reported each year to SPARCS. The increased hardware requirements for maintaining the increased volume of data are not included in the above estimates. The SPARCS system was allocated additional available disk space to accommodate one month of data. The system was designed to operate under those constraints.

In the future, improvements in the availability of multi-year data can be planned as more resources are made available. Separate and apart from the Emergency Department Data Collection System development activities is to support the HIPAA compatible 837-format. The original system was designed to use HIPAA-compatible data content. The translation software to convert the existing HIPAA-compatible data content to the HIPAA 837-format had been purchased by the NYS DOH from a third party vendor and is being implemented as a separate project with separate funding. Consequently those estimates are not included in this analysis.

The start of this paper identified three criteria that should be factored into the calculation of a data system development project’s Return-On-Investment (ROI). Specifically, the use of data standards to develop new or enhanced data systems should: (1) provide a cost effective roadmap to implementation; (2) simplify the development process; and (3) be a positive factor in developing the relationship between data suppliers and data users.

The New York State hospital discharge system, with full support of NYS DOH management, fully committed to the use of data standards. Using data standards at the foundation of the Emergency Department Data Collection System development project contributed to the project’s success. Specifically, the use of data standards provided:

The bottom line is this project was completed on time with existing staff and with consensus between the data users and the data suppliers. The financial cost incurred by the NYS DOH was about a quarter of a million dollars, most of which came from reallocated existing resources.

It is important that other data collection organizations share their system development experiences and the ROI as a result of using data standards so the field can gain insight into how data standards can generate a positive ROI for organizations developing tomorrow’s information systems. Together, data collection organizations can promote the value in a long-term investment in the use of data standards, and provide hard number justifications for the following intuition: It is good business practice to use data standards.