Standards for child and youth development programs

2020 Edition

Child and Youth Development Program Administration (CYD-AM) 11: Continuous Quality Improvement

The program develops and implements a system for promoting continuous quality improvement (CQI).
2020 Edition

Currently viewing: CHILD AND YOUTH DEVELOPMENT PROGRAM ADMINISTRATION (CYD-AM)

VIEW THE STANDARDS

Purpose

Sound administration and management increase program quality and sustainability; promote financial accountability and viability; support transparency and openness; and reduce risk, loss, and liability exposure.
Note: Information obtained during continuous quality improvement activities should also inform long-term planning, as addressed in CYD-AM 3.
1
The program’s practices fully meet the standard, as indicated by full implementation of the practices outlined in the Practice Standards.
2
Practices are basically sound but there is room for improvement, as noted in the ratings for the Practice Standards.
3
Practice requires significant improvement, as noted in the ratings for the Practice Standards.
4
Implementation of the standard is minimal or there is no evidence of implementation at all, as noted in the ratings for the Practice Standards.
Self-Study EvidenceOn-Site EvidenceOn-Site Activities
  • A description or plan that outlines the programs CQI system/process
  • A list of what is being measured, including outcomes, outputs, indicators, targets, data sources, and methods/instruments for collecting data (may be included in the general CQI description or plan)
  • Data analyses/reports
  • Procedures for collecting and aggregating data, including procedures for cleaning data (CYD-AM 11.03, 11.04)
  • Procedures for reviewing and taking action based on findings (CYD-AM 11.05, 11.06)
  • Evidence of actions taken based on the analysis and use of data, including any related improvement/corrective action plans
  • CQI summary report
  • Instruments for collecting data
  • Minutes from meetings where CQI items are reviewed and discussed
  • Long term plan
  • Logic model (or equivalent framework)
  • Budget (for resources allocated to CQI)
  • Documentation of staff CQI training
  • CQI training curricula and materials
  • Documentation of staff involvement in CQI planning/activities
  • Final Accreditation Report (FAR), for programs seeking re-accreditation
  • Interview:
    1. Program Administrator and/or Director
    2. Program Personnel
    3. Children, youth, families, and other stakeholders
  • Review of management information system for collecting, aggregating, analyzing, and maintaining data

CYD-AM 11.01

The program’s leadership fosters a culture of excellence and continual improvement by:
  1. regularly considering how the program is using data to promote improvement and goal achievement;
  2. emphasizing the importance of continuous quality improvement in regular meetings and conversations;
  3. training staff at all levels to understand how CQI functions within the program;
  4. encouraging and supporting staff involvement in CQI activities;
  5. involving other key stakeholders, including children, youth, families, and community partners, in the program’s CQI process; and
  6. allocating sufficient resources to sustain the program’s CQI system.

Interpretation

The process of implementing COA’s Standards is an additional way to foster a culture of excellence and continual improvement.
Note: See CYD-AM 11.02, 11.03, 11.05 and 11.07 for more information regarding expectations for obtaining information from, and sharing information with, key stakeholders. 

CYD-AM 11.02

In an effort to monitor progress toward goals and desired outcomes, the program identifies:
  1. key goals, outcomes, and outputs that merit ongoing monitoring;
  2. related measurement indicators; and
  3. related performance targets. 

Interpretation

 Programs should strive to measure factors related to: the impact of services and supports on children and youth; the quality and quantity of services and supports provided; satisfaction with services; and the management/administration of the program.  For example, programs might seek to measure factors ranging from staff preparedness, to program climate, to activity quality, to attendance level, to parent and child satisfaction, to the implementation of disciplinary practices, to the accuracy and completeness of files, to the development or academic progress of children and youth, to equity in relation to staffing (e.g., considering whether the cultural characteristics of personnel are generally reflective of the children and families served).
 
Note: As noted in CYD-AM 3.02, the program’s goals and desired outcomes should be specified in a logic model or equivalent framework that establishes a clear connection between the program’s mission, community needs, required inputs/resources, planned services and supports, expected outputs, and desired goals/outcomes. 

CYD-AM 11.03

The program supports the measurement and monitoring of key goals, outcomes, and outputs by:
  1. identifying pertinent data sources, including data collection tools or instruments; and
  2. collecting relevant data on an ongoing basis.

Interpretation

Programs may collect different types of data and use a variety of different collection methods, from surveys, to observations, to focus groups.  In some cases a program may need to partner with other organizations and agencies to gain access to relevant data.  For example, an after school program striving to improve academic outcomes might establish an agreement with the school district in order to obtain data such as grades or test scores.  When it is necessary to use a tool to collect data (e.g., a survey or observation tool) programs should ideally use standardized and/or evidence-based tools when such tools are available and appropriate.  

Timeframes for data collection will vary based upon data type.  For example, while a program may administer satisfaction surveys on an annual basis, attendance data will be collected on an ongoing basis.  When a program establishes an agreement with another entity in order to access relevant information (e.g., grades or test scores), the agreement may specify that the information will be provided at specific intervals (e.g., on a quarterly basis).

 

CYD-AM 11.04

The program cleans, aggregates, and analyzes the collected data in order to: 
  1. ensure data integrity;
  2. track identified measures; and
  3. identify patterns and trends.

Interpretation

“Cleaning data” means checking for errors and inconsistencies in order to improve the quality of data prior to aggregating and analyzing it.  Common things to check for include accuracy, completeness, timeliness, uniqueness, and outliers.  

Data aggregation and analysis can occur at a frequency determined by the program based on how the data will be used, but should occur no less than annually.  Programs should ideally compare data both over time and against internal and external benchmarks.
Note: An information management system, as addressed in CYD-AM 9, can help to facilitate the maintenance and aggregation of data.

CYD-AM 11.05

The program reviews findings with staff and other stakeholders and takes action, when indicated, to:
  1. eliminate or reduce identified problems;
  2. improve services and supports for children and youth;
  3. improve program systems, processes, policies, and procedures;
  4. replicate good practice; and
  5. recognize and motivate staff.

Interpretation

 Programs should have procedures for reviewing findings with staff and other stakeholders, including for discussing: (1) areas of strength and quality practice; (2) areas in need of improvement; and (3) how to prioritize targeted areas, identify needed improvements, and monitor the effectiveness of improvements over time.  In order to facilitate meaningful engagement programs must first decide when and how findings will be communicated to different parties. A program can start by determining who needs what data, with what frequency, and how best to share the information. Methods for sharing findings include, but are not limited to: performance dashboards, report cards, or other types of summary reports; discussion at staff and departmental meetings; and presentations at family/community meetings.   
 

When the program identifies issues that will involve ongoing effort or monitoring, improvement or corrective action plans should be developed and implemented. These plans should include: (1) goals, (2) action steps, (3) needed resources, (4) timetables, and (5) expectations for monitoring/review, including expectations regarding the frequency of monitoring and whether stakeholders will be involved in assessments of progress. 

CYD-AM 11.06

The program monitors the effectiveness of actions taken and modifies implemented improvements, as needed.
Note:  As noted in CYD-AM 11.05, the program should develop improvement or corrective action plans that specify both: (1) how often monitoring should occur, and (2) and whether stakeholders should be involved in assessments of progress.
 

CYD-AM 11.07

At least annually the program creates a CQI summary report that addresses the program’s successes, challenges, improvements, and goals.
 

Interpretation

The summary report should address: key CQI activities or issues that have been resolved; key CQI activities or issues that are ongoing or require continued monitoring; and CQI priorities and goals for the coming year. It should typically be shared with stakeholders including, but not limited to: program personnel; program participants and their families; parties responsible for providing guidance and oversight; funders; and community partners.