Standards for private organizations

2020 Edition

Performance and Quality Improvement (PQI) 5: Analyzing and Reporting Information

The organization systematically collects, aggregates, analyzes, and maintains data.
2020 Edition

Currently viewing: PERFORMANCE AND QUALITY IMPROVEMENT (PQI)

VIEW THE STANDARDS

Purpose

An organization-wide performance and quality improvement system uses data to promote efficient, effective service delivery and achievement of the organization’s mission and strategic goals.
1
The organization's practices fully meet the standard as indicated by full implementation of the practices outlined in the PQI 5 Practice standards. Comprehensive PQI data management procedures support the organization's ability to systematically collect, aggregate, analyze and maintain data.
2
Practices are basically sound but there is room for improvement as noted in the ratings for the PQI 5 Practice standards.
3
Practice requires significant improvement as noted in the ratings for the PQI 5 Practice standards.
4
Implementation of the standard is minimal or there is no evidence of implementation at all, as noted in the ratings for the PQI 5 Practice standards.
Self-Study EvidenceOn-Site EvidenceOn-Site Activities
  • Procedures for collecting, reviewing, and aggregating PQI data
  • Data analysis/reports related to the elements in PQI 5.02
  • Most recent summary documents or reports provided to internal and external stakeholder
  • Procedures for sharing and reviewing reports and findings with staff and stakeholders
  • PQI committee/work group minutes for analyzing PQI information
  • Documentation of stakeholder review and discussion of PQI results, including meeting minutes and agendas
  • Governing body meeting minutes regarding review of PQI data
  • Documentation of improvements made from the analysis and use of PQI data, including any related corrective action/improvement plans
  • Interviews may include:
    1. PQI personnel 
    2. Relevant personnel 
  • Observe system for collecting, aggregating, analyzing, and maintaining data

PQI 5.01

Procedures for collecting, reviewing, and aggregating data include:
  1. cleaning data to ensure data integrity including accuracy, completeness, timeliness, uniqueness, and outliers;
  2. quarterly aggregation of data; and
  3. developing reports for analysis and interpretation.

Interpretation

 Data should be collected, aggregated, and reviewed at least quarterly at all three levels of performance measurement as addressed in PQI 3.03, PQI 4, and the Client-Centered Logic Model Core Concept in each assigned Service Standard.
Examples: Cleaning data, also known as data cleansing, means checking for errors and inconsistencies in order to improve the quality of your data prior to aggregating and analyzing it. Common things to check for include:
  1. accuracy - making sure the data was recorded correctly including misspellings, correct numbers, addresses, etc.;
  2. completeness - making sure all the data was recorded and none is missing;
  3. timeliness - ensuring that the data is current and/or relevant to the current time frame;
  4. uniqueness - ensuring that data was recorded only once and not multiple times; and
  5. outliers - look for data that is unexpected (Note: This could mean you have a PQI issue that warrants attention but sometimes a single extreme result, even if it is legitimate, can tip the results so they are not truly representative).
1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Procedures for ensuring data integrity and reliability are sufficient for sustaining the PQI system but need some improvement, e.g., formats for reports are not consistently useful for analysis; or
  • In a few instances, data was not aggregated and reviewed quarterly.
3
Practice needs significant improvement; e.g.,
  • Procedures are insufficient to sustain consistent data review or do not address one of the standard's elements; or
  • Only some of the collected data is reviewed and/or aggregated for review; or
  • Data is rarely aggregated into a form that permits analysis.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.

PQI 5.02

The organization analyzes PQI data to:
  1. track and monitor identified measures;
  2. identify patterns and trends; and
  3. compare performance over time.
1
The organization's practices reflect full implementation of the standard. The organization analyzes PQI data per the requirements of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Data from across the organization is analyzed, but data is not analyzed for one of the organization’s programs; or
  • Data analysis does not include one of the elements of the standard.
3
Practice needs significant improvement; e.g.,
  • Most of the organization's PQI data has not been analyzed; or
  • Data analysis is not performed for most of the organization's programs or services; or
  • Data related to management and operational performance is not analyzed.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.

PQI 5.03

Reports of PQI findings are:
  1. shared and discussed with board members, staff, and stakeholders; and
  2. distributed in timeframes and formats that facilitate review, analysis, interpretation, and timely corrective action.
Examples: Discussions with board members, staff, and stakeholders about PQI findings can include:
  1. areas of strength and quality practice;
  2. areas for improvement; and
  3. how to prioritize targeted areas, identify interventions, and monitor the effectiveness of interventions over time.
In order to engage in meaningful discussions about the data being collected, organizations should decide how results will be communicated to staff and stakeholders. Organizations can start by determining who needs what data, with what frequency, and how best to share the information.

Methods for sharing findings include:
  1. performance dashboards, report cards, or other types of summary reports;
  2. discussion at board, staff, and departmental meetings;
  3. using monthly reports of key service delivery outputs and outcomes in staff supervision activities;
  4. conducting focus groups and presentations at community meetings;
  5. soliciting feedback via interviews or surveys;
  6. providing quarterly reports to oversight entities, stakeholder advisory groups, and leaders on important data related to key operations and management functions; and
  7. quality review activities that engage community providers.
 Graphic presentation of data is very useful in communicating results of PQI activities. Data visualization techniques can facilitate understanding of complex information and reveal underlying patterns and relationships within the data that may otherwise go unnoticed.
1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Summary reports are created and distributed, but practice could be improved; or
  • Stakeholders have complained about reports that are hard to read or understand; or
  • Summary reports are not always distributed in a useful timeframe.
3
Practice requires significant improvement, e.g.,
  • There are many examples of relevant PQI data not being provided to stakeholders for review; or
  • Data is not formatted into reports; or
  • The format of reports is unclear and confusing; or
  • Confidentiality concerns have been raised or noted.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.

PQI 5.04

The organization:
  1. reviews PQI findings and stakeholder feedback and takes action, when indicated; and
  2. monitors the effectiveness of actions taken and modifies implemented improvements, as needed.
Examples: Organizations can use PQI findings and feedback to:
  1. improve services;
  2. eliminate or reduce identified problems;
  3. replicate good practice;
  4. recognize and motivate staff; and
  5. improve organizational systems, processes, policies, and procedures.
Examples: Information generated by the PQI system can be used to:
  1. monitor progress toward achieving its mission and strategic and annual goals;
  2. meet funder requirements; and
  3. promote the organization and its services throughout the community.
Examples: Corrective Action Plans or Improvement Plans can be implemented when issues have been identified that will involve ongoing effort and monitoring.

Improvement Plans formally lay out the actions that will be taken to address areas in need of improvement that are identified by staff and stakeholders as crucial to meeting the organization's goals and delivering quality services. Improvement plans should be implemented when it is necessary to monitor and address the issue over time.

Corrective Action Plans are implemented to correct problems or deficiencies, including those related to compliance with regulatory requirements (e.g., Medicaid documentation requirements). The need for a Corrective Action Plan suggests that the issue has moved beyond program improvement to the level of oversight by the organization's leadership.
 
Organizations may also wish to create an annual summary report for oversight entities, stakeholders, and staff that includes:
  1. key PQI activities that are ongoing, have been resolved, or that need further intervention;
  2. issues that require continued monitoring within the PQI system; and
  3. PQI priorities and goals for the coming year.
1
The organization's practices fully meet the standard as indicated by full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The organization uses PQI data to improve programs; however, some available findings and recommendations are not being used; or
  • Actions made in response to findings and feedback are being monitored, and modifications are made when needed, but practice could be improved, e.g., the data is not being reviewed in a timely manner.
3
Practice requires significant improvement, e.g.,
  • PQI data is not routinely used; or
  • Except for a few examples, the organization does not generate enough usable data to take meaningful action, or does not routinely use data in either of the ways listed in the standard; or
  • Important modifications are often not made despite evidence that they are needed.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.