Standards for public agencies

2020 Edition

Performance and Quality Improvement (PA-PQI) 5: Gathering Data and Communicating Information

The agency’s data management practices facilitate the collection, analysis, communication and interpretation of data.
2020 Edition

Currently viewing: PERFORMANCE AND QUALITY IMPROVEMENT (PA-PQI)

VIEW THE STANDARDS

Purpose

 An agency-wide performance and quality improvement system effectively engages staff, persons served, and other stakeholders in advancing the agency’s mission and achieving strategic goals through continuous, integrated, data-driven efforts to improve service delivery and administrative practice. 
1
Full Implementation, Outstanding Performance
A rating of (1) indicates that the agency's practices fully meet the standard and reflect a high level of capacity.  
  • All elements or requirements outlined in the standard are evident in practice, with rare or no exceptions: exceptions do not impact service quality or agency performance. 
2
Substantial Implementation, Good Performance
A rating of (2) indicates that an agency's infrastructure and practices are basically sound but there is room for improvement.
  • The majority of the standards requirements have been met and the basic framework required by the standard has been implemented. 
  • Minor inconsistencies and not yet fully developed practices are noted; however, these do not significantly impact service quality or agency performance.
3

Partial Implementation, Concerning Performance
A rating of (3) indicates that the agency's observed infrastructure and/or practices require significant improvement.  

  • The agency has not implemented the basic framework of the standard but instead has in place only part of this framework.  
  • Omissions or exceptions to the practices outlined in the standard occur regularly, or practices are implemented in a cursory or haphazard manner.  
  • Service quality or agency functioning may be compromised.  
  • Capacity is at a basic level.
4
Unsatisfactory Implementation or Performance
A rating of (4) indicates that implementation of the standard is minimal or there is no evidence of implementation at all.  
  • The agency’s observed administration and management infrastructure and practices are weak or non-existent; or show signs of neglect, stagnation, or deterioration.
Self-Study EvidenceOn-Site EvidenceOn-Site Activities
County/Municipality Administered Agency, State Administered Agency (Central Office), or other Public Entity
  • Policies and/or procedures for:
    1. Data management 
    2. Reviewing and aggregating data 
  • Most recent aggregate data reports and additional summary documents (e.g., performance dashboards, reports of gains made against goals, annual scorecards, etc.) 
  • Documentation of reporting to:
    1. staff, oversight entities, and stakeholders at least annually 
    2. the public 
  • Documentation of:
    1. Decisions made at the agency level based on PQI findings (corrective actions, initiatives, etc.) 
    2. Tracking the impact of decisions made (data reports that link to areas named in PIP, annual reports, etc.) 
  • Current agency and/or program improvement plans 
State Administered Agency (Regional Office)
  • Aggregate data reports and additional summary documents (e.g., performance dashboards, reports of gains made against goals, annual scorecards, etc.) 
  • Documentation of reporting to:
    1. staff, oversight entities, and stakeholders at least annually 
    2. the public 
  • Documentation of:
    1. Decisions made at the worker, program, and regional level based on findings (corrective actions, initiatives, etc.) 
    2. Tracking the impact of decisions made (data reports that link to areas named in PIP, annual reports, etc.) 
  • Current regional and/or program improvement plans
All Agencies
  • Documentation of stakeholder review and discussion of PQI summary reports
  • PQI meeting minutes for the past six months
  • Leadership team, management, and staff meeting schedules, agendas, and minutes from the past six months
All Agencies
  • Interviews may include:
    1. Relevant staff 
    2. External stakeholder groups
  • Observe systems for collecting, analyzing, and communicating data

PA-PQI 5.01

Procedures for collecting, reviewing, and aggregating data: 
  1. ensure data integrity and reliability;
  2. engage staff at all levels of the agency including frontline staff; and
  3. facilitate the development of useable reports for analysis and interpretation.

PA-PQI 5.02

Data is collected and maintained in a manner that allows for:
  1. tracking and monitoring identified measures;
  2. identifying patterns and trends; 
  3. comparing performance over time; and
  4. comparing data against the results of internal or external targets or benchmarks, when appropriate

PA-PQI 5.03

Summary reports of PQI information: 
  1. are distributed and discussed with staff and stakeholders in a timeframe and format that facilitates review, analysis, interpretation, and timely corrective action;
  2. reflect multiple data sources, when appropriate, including quantitative and qualitative data and formal and informal information gathered; 
  3. enable the comparison of data against the results of similar programs, internal or external benchmarks, etc.; and
  4. facilitate compliance with regulatory data reporting requirements. 

Interpretation

The content and format of PQI summary reports should take into account the needs of regional and/or local offices to ensure the data is presented in a useful way that facilitates corrective action at the worker and program level.

Interpretation

Timely corrective action should include ensuring information is distributed early enough that regional and local offices can evaluate and implement changes prior to the next round of internal or external reviews. 

In regard to element (d), in addition to the data itself, child and family services agencies participating in the Child and Family Services Reviews must be prepared to provide the federal government with:
  1. the data source; 
  2. the methodology for calculating or analyzing the data;
  3. the scope of the data (i.e. geographic, population, etc.); 
  4. the time period applicable to the data;
  5. information pertaining to the completeness, accuracy and reliability of the data; and 
  6. other known limitations of the data.
Examples: Methods for sharing findings can include:
  1. performance dashboards, report cards, or other types of summary reports;
  2. using monthly reports of key service delivery outputs and outcomes in staff supervision activities; 
  3. conducting focus groups and presentations at community meetings; 
  4. soliciting feedback via interviews or surveys;
  5. providing quarterly reports to the oversight entities, stakeholder advisory groups, and leaders on important data related to key operations and management functions; and
  6. quality review activities that engage community providers.
Examples: In regard to element (a), discussions with staff and stakeholders about PQI information can include:
  1. areas of strength and quality practice;
  2. areas for improvement; and
  3. how to prioritize targeted areas, identify interventions, and monitor the effectiveness of interventions over time.

PA-PQI 5.04

The agency has a mechanism for reporting, at least annually, to oversight entities, stakeholders, and staff on:
  1. key PQI activities that are ongoing, have been resolved, or that need further intervention;
  2. issues that require continued monitoring within the PQI system; and  
  3. PQI priorities and goals for the coming year.

PA-PQI 5.05

The agency shares PQI information with the public as part of its public outreach and education strategy.
Note: See PA-AM 4.01 for more information on developing a public outreach and education strategy.

PA-PQI 5.06

The agency:
  1. acts on PQI findings at the worker, program, region/community, agency, and system level; and
  2. monitors the effectiveness of interventions and adjusts interventions, as needed.

Interpretation

Information generated by the PQI system serves as evidence for identifying interventions in relation to:
  1. fulfilling the mission and meeting legal mandates;
  2. monitoring progress toward strategic plans and long-term goals;
  3. managing programs and operations efficiently and effectively;
  4. supporting direct service staff to meet program goals, make infomed case-level decisions, and have a positive impact on persons served; and
  5. meeting regulatory requirements. 
Examples: Agencies can use PQI findings and feedback to:
  1. develop solutions;  
  2. replicate good practice;  
  3. recognize and motivate staff; 
  4. update staff training and other professional development activities;
  5. improve organizational systems, processes, policies, and procedures; and
  6. eliminate or reduce identified problems.

PA-PQI 5.07

The agency develops improvement plans when issues have been identified that will involve coordinated and ongoing activities and monitoring.

Interpretation

Improvement plans formally lay out the actions that will be taken to address areas in need of improvement that are identified by staff and stakeholders as crucial to meeting the agency’s goals and delivering quality services. Improvement plans should be implemented when it is necessary to monitor and address the issue over time. 
 

Interpretation

State-administered agencies should manage a statewide and regional performance improvement action planning process in order to take system-wide action and also allow for targeted PQI activities based on regional context.
 

PA-PQI 5.08

Agency leaders, senior managers, program directors, and supervisors:
  1. keep PQI on the agenda of management and staff meetings;
  2. integrate data discussions and outcomes monitoring into case reviews, supervision, performance review, and contract monitoring; 
  3. regularly evaluate the need for and uses of data at the worker, program, region/community, agency, and system level; and
  4. evaluate the PQI infrastructure, processes, and procedures.