Table 3 Joint display of findings

  Agency and MCA-level variation

At the agency level, average (mean) rates of missing or invalid values for the year 2015 were consistently larger than missing or invalid values for the same variables at the incident level. MCA-level results also show significant varying rates of missingness by MCA.

Agency:

While some participants reported that they were confident in the completeness and quality of their own agency’s data, others acknowledged that data entry was often a problem for their agencies.

MCA:

Participants reported this in the context of MCAs as well whereby certain MCAs perform better in data collection or have more resources to do so.

  Software and data mapping variation

Different software platforms exhibited greater or lesser levels of missingness.

Participants expressed that much of the variation in data completeness and quality was due to data mapping issues, which is primarily a result of the data reporting software used.

  Data quality: data entry

Of the 18 variables studied, only five exhibited less than 10% missingness, while only Incident Zip Code and Provider’s Primary Impression exhibited less than 5% missingness.

Participants expressed frustration that despite the time and effort that is required to collect and report high-quality data, the resulting dataset has levels of missing or invalid data that make them of limited use to QI efforts.

  Data quality: “bare-minimum” effect

At the agency level, required demographic data was found to be missing or invalid at much lower rates than Vital Sign Data. Age, Gender, and Race were missing or invalid 17.9, 17.5, and 18.8% of the time, respectively. This is notably lower than for unrequired Vital Signs Data such as Medical Allergies (53%), Current Medication Name (67.7%), Pulse Oximetry (59.4%), and Body Temperature (95.2%).

During interviews, key participants referred to the fact that data reporting software is not used to best practice. Instead, they claimed that the bare minimum amount of data is often entered into reports in order to meet reporting and compliance requirements.

Utility

There was no MCA level variable for regional oversight. The system itself is difficult to query, requires the downloading of many files and statistical expertise.

Many participants expressed discontent that there was no way to query MI-EMSIS to answer clinically relevant questions and that the lack of regional identifiers (MCA or county variables) made oversight using MI-EMSIS difficult.