Sound administration and management increase program quality and sustainability; promote financial accountability and viability; support transparency and openness; and reduce risk, loss, and liability exposure.
Note: See CYD-AM 11.02, 11.03, 11.05 and 11.07 for more information regarding expectations for obtaining information from, and sharing information with, key stakeholders.
In an effort to monitor progress toward goals and desired outcomes, the program identifies:
key goals, outcomes, and outputs that merit ongoing monitoring;
related measurement indicators; and
related performance targets.
Programs should strive to measure factors related to: the impact of services and supports on children and youth; the quality and quantity of services and supports provided; satisfaction with services; and the management/administration of the program. For example, programs might seek to measure factors ranging from staff preparedness, to program climate, to activity quality, to attendance level, to parent and child satisfaction, to the implementation of disciplinary practices, to the accuracy and completeness of files, to the development or academic progress of children and youth, to equity in relation to staffing (e.g., considering whether the cultural characteristics of personnel are generally reflective of the children and families served).
Note:As noted in CYD-AM 3.02, the program’s goals and desired outcomes should be specified in a logic model or equivalent framework that establishes a clear connection between the program’s mission, community needs, required inputs/resources, planned services and supports, expected outputs, and desired goals/outcomes.
The program supports the measurement and monitoring of key goals, outcomes, and outputs by:
identifying pertinent data sources, including data collection tools or instruments; and
collecting relevant data on an ongoing basis.
Programs may collect different types of data and use a variety of different collection methods, from surveys, to observations, to focus groups. In some cases a program may need to partner with other organizations and agencies to gain access to relevant data. For example, an after school program striving to improve academic outcomes might establish an agreement with the school district in order to obtain data such as grades or test scores. When it is necessary to use a tool to collect data (e.g., a survey or observation tool) programs should ideally use standardized and/or evidence-based tools when such tools are available and appropriate.
Timeframes for data collection will vary based upon data type. For example, while a program may administer satisfaction surveys on an annual basis, attendance data will be collected on an ongoing basis. When a program establishes an agreement with another entity in order to access relevant information (e.g., grades or test scores), the agreement may specify that the information will be provided at specific intervals (e.g., on a quarterly basis).
The program cleans, aggregates, and analyzes the collected data in order to:
ensure data integrity;
track identified measures; and
identify patterns and trends.
“Cleaning data” means checking for errors and inconsistencies in order to improve the quality of data prior to aggregating and analyzing it. Common things to check for include accuracy, completeness, timeliness, uniqueness, and outliers.
Data aggregation and analysis can occur at a frequency determined by the program based on how the data will be used, but should occur no less than annually. Programs should ideally compare data both over time and against internal and external benchmarks.
Note: An information management system, as addressed in CYD-AM 9, can help to facilitate the maintenance and aggregation of data.
The program reviews findings with staff and other stakeholders and takes action, when indicated, to:
eliminate or reduce identified problems;
improve services and supports for children and youth;
improve program systems, processes, policies, and procedures;
replicate good practice; and
recognize and motivate staff.
Programs should have procedures for reviewing findings with staff and other stakeholders, including for discussing: (1) areas of strength and quality practice; (2) areas in need of improvement; and (3) how to prioritize targeted areas, identify needed improvements, and monitor the effectiveness of improvements over time. In order to facilitate meaningful engagement programs must first decide when and how findings will be communicated to different parties. A program can start by determining who needs what data, with what frequency, and how best to share the information. Methods for sharing findings include, but are not limited to: performance dashboards, report cards, or other types of summary reports; discussion at staff and departmental meetings; and presentations at family/community meetings.
When the program identifies issues that will involve ongoing effort or monitoring, improvement or corrective action plans should be developed and implemented. These plans should include: (1) goals, (2) action steps, (3) needed resources, (4) timetables, and (5) expectations for monitoring/review, including expectations regarding the frequency of monitoring and whether stakeholders will be involved in assessments of progress.
The program monitors the effectiveness of actions taken and modifies implemented improvements, as needed.
Note: As noted in CYD-AM 11.05, the program should develop improvement or corrective action plans that specify both: (1) how often monitoring should occur, and (2) and whether stakeholders should be involved in assessments of progress.
At least annually the program creates a CQI summary report that addresses the program’s successes, challenges, improvements, and goals.
The summary report should address: key CQI activities or issues that have been resolved; key CQI activities or issues that are ongoing or require continued monitoring; and CQI priorities and goals for the coming year. It should typically be shared with stakeholders including, but not limited to: program personnel; program participants and their families; parties responsible for providing guidance and oversight; funders; and community partners.