Standards for private organizations

2020 Edition

Performance and Quality Improvement (PQI) 3: Performance and Outcomes Measures

The organization identifies measures and outcomes related to:
  1. the impact of services on clients;
  2. quality of service delivery; and
  3. management and operations performance.
2020 Edition

Currently viewing: PERFORMANCE AND QUALITY IMPROVEMENT (PQI)

VIEW THE STANDARDS

Purpose

An organization-wide performance and quality improvement system uses data to promote efficient, effective service delivery and achievement of the organization’s mission and strategic goals.
Examples: Organizations providing child welfare services are encouraged to integrate the Federal Child and Family Service Review (CFSR) Outcomes measures and Systemic Factors, particularly those identified in Performance Improvement Plans, into their overall PQI system.
1
The organization's practices fully meet the standard as indicated by full implementation of the practices outlined in the PQI 3 standards.
2
Practices are basically sound but there is room for improvement as noted in the ratings for the PQI 3 Practice standards.
3
Practice requires significant improvement as noted in the ratings for the PQI 3 Practice standards.
4
Implementation of the standard is minimal or there is no evidence of implementation at all, as noted in the ratings for the PQI 3 Practice standards.
Self-Study EvidenceOn-Site EvidenceOn-Site Activities
  • See PQI plan (PQI 1) for a description of what is being measured. Response must address PQI 3.01, PQI 3.02, and PQI 3.03 and include:
    1. outcomes
    2. outputs
    3. data sources
    4. indicators
    5. targets
  • See PQI outcomes documentation provided in the Client-Centered Logic Model Core Concept in each service standard
  • Interviews may include:
    1. PQI personnel
    2. Relevant personnel

 

PQI 3.01

The organization identifies key outputs and outcomes, and related:
  1. measurement indicators;
  2. performance targets; and
  3. data sources including data collection tools or instruments for each identified output and outcome.

Interpretation

Organizations are encouraged to use standardized or recognized outcomes evaluation tools when such tools are available and appropriate.
 

Interpretation

Program outputs and client outcomes must be identified in the logic model submitted in the Client-Centered Logic Model Core Concept in each assigned Service Standard.
Examples:
Outputs are what the program delivers. Examples of program outputs include:
  1. number of educational or clinical sessions provided;
  2. total number of clients served over a specified period of time; and
  3. number of housing placements made.
 Outcomes are the observable and measurable effects of a program's activities on its service recipients. Examples include:
  1. improved functioning as measured by the Children's Functional Assessment Rating Scale (CFARS);
  2. number/percent of homeless and runaway youth that are reunited with family during past quarter;
  3. reduction in criminal justice system involvement; and
  4. improved family/community involvement.
 For some programs, outcomes, outputs, indicators, tools, etc. may be established by contractual and/or funding requirements.
1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The organization has not developed indicators or performance targets for some of its programs.
3
Practice needs significant improvement; e.g.,
  • At least one of the standard's elements are not being addressed at all; or
  • Outputs and outcomes have not yet been identified for one of its high-risk programs.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.

 

PQI 3.02

The organization surveys clients annually to assess program quality.
Examples: Types of information that the program may collect from clients can include client satisfaction or outcome information.

According to the Urban Institute, client surveys can be an indispensable source of outcome information. They provide a systematic means of gathering data on service outcomes from all or a portion of clients. Client surveys help organizations learn whether services are producing anticipated or desired results and, if not, provide clues for how to improve them.

Issues covered by a client survey should correspond to the key service outcomes an organization wishes to track. Because survey length generally affects response rates, issues not pertinent to improving outcomes should probably be limited. The goal is to develop the shortest possible list of questions consistent with the survey’s objective of assessing outcomes.
1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement.
3
Practice needs significant improvement.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.

 

PQI 3.03

The organization identifies measures for management and operational performance to:
  1. measure progress toward achieving its mission and strategic and annual goals;
  2. evaluate operational functions that influence the capacity to deliver services and meet the needs of persons served; and
  3. identify and mitigate risk.
Examples: Examples of operations and management performance measures can include:
  1. efficiency in the allocation and utilization of its human and financial resources to further the achievement of organizational objectives;
  2. effectiveness of risk prevention measures;
  3. effectiveness at retaining a competent and qualified workforce through staff retention/turnover and satisfaction;
  4. costs versus benefits of fundraising efforts;
  5. achievement of budgetary objectives ;
  6. effectiveness of community education and outreach; and
  7. efforts to diversify the governing body.
Organizations may consider if any data is currently being collected related to these elements. Then, the organization may identify an outcome or goal in some of these areas.
 
Network Examples: Network management entities may also measure important network administrative processes, such as:
  1. the average length of time between receiving a clean claim and paying the claim;
  2. the proportion of services that are evidence-based or meet nationally recognized treatment guidelines developed by consensus groups;
  3. the effectiveness of network training;
  4. the satisfaction of stakeholders, such as high volume referral agents (e.g., judges, court workers, employee assistance agents);
  5. penetration rates, or the proportion of the whole population eligible to be served by the network who actually receive services; and
  6. results of retrospective case record reviews, including the percentage of cases in which a placement decision includes an appropriate application of clinical criteria.
1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The organization has identified performance measures related to two of the three elements of the standard.
3
Practice needs significant improvement; e.g.,
  • The organization has identified performance measures related to only one of the standard's elements.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.

 

PQI 3.04

Findings and recommendations from external review processes are integrated into the organization's PQI system.
Examples: External reviews can include:
  1. licensing and other reviews related to federal, state, and local requirements;
  2. government and other funder audits;
  3. accreditation reviews; and
  4. other reviews, where appropriate.
1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The process for review of findings and recommendations can be improved, e.g., while findings are reviewed by management, they are not integrated into the PQI improvement cycle when appropriate.
3
Practice needs significant improvement; e.g.,
  • There is evidence that the organization has not adequately addressed the findings or recommendations of at least one key external review; or
  • It does not review or address findings in a timely manner and thus may be putting itself at risk of sanction.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.