Program Evaluation Approach: I will be designing an outcome (Center for Disease Control) evaluation for learning and use (Green, 2017) for the purpose of program and organizational improvement (Donaldson & Lipsey, 2011). The scope of the evaluation will be determining if or to what extent the program is achieving expected outcomes in terms of teacher practice and student behaviour or mental health (Center for Disease Control).
Considerations for this approach: To be effective for learning purposes, my evaluation must consider context (Sanders, 1994), can be on-going or cyclical, and should make allowances to include stakeholders in discussing evaluation results and next steps (Torres, 2002).
To be effective in improving the program or organization, the evaluation needs to consider utility standards – including report clarity and evaluation impact – and accuracy standards – such as data analysis and justified conclusions (Sanders, 1994). That way stakeholders or designers can continuously adjust and improve their program, program goals, or organizational learning by gaining knowledge and an understanding of evaluative thinking (Donaldson & Lipsey, 2011).
Due to the nature of the program, the evaluation needs to carefully consider the analysis of qualitative information (Sanders, 1994), such as attitude or behaviour changes in students.
Fortunately, the Egale organization has completed two wide reports based on student and educator surveys regarding the issues addressed in this program (Peter et al., 2021). The 2021 report shows data collected about inclusivity in Canadian schools. A specific data collection survey based on similar criteria given at each participating school should be effective in collecting data that directly addresses stakeholder concerns.
Evaluation Questions:
Based on feedback from other students, the instructor, and my new understanding of evaluation, I’ve made adjustments to my evaluation questions. Feedback is appreciated!
- 1. Are teachers implementing program practices in their classrooms or school environments?
- 2. How have students been affected? Do 2SLGBTQI students report changes to their mental health, feeling of belonging, or expectation of future success? Do other students demonstrate an increased understanding of the 2SLGBTQI community?
- 3. Are there fewer incidences of anti-2SLGBTQI cyberbullying?
- 4. Do students who identify as members of the 2SLGBTQI community demonstrate improved academic success or increased engagement in extra-curricular activities?
References:
Center for Disease Control. Types of Evaluation. [PDF]. Retrieved from: https://www.cdc.gov/std/Program/pupestd/Types%20of%20Evaluation.pdf
R. T. Torres. (2002). What is a learning approach to evaluation? The evaluation exchange: A periodical on emerging strategies in evaluation. Volume VIII, Number 2, Fall 2002. Harvard Family Research Project. Retrieved from: https://archive.globalfrp.org/evaluation/the-evaluation-exchange/issue-archive/evaluation-for-continuous-improvement/what-is-a-learning-approach-to-evaluation
Sanders, James R. (1994). The Joint Committee on Standards for Educational Evaluation: The Program evaluation Standards (2nd ed.). Sage. [PDF]. Retrieved May 14, 2022, from https://www.oecd.org/dev/pgd/38406354.pdf
S. I. Donaldson & M. W. Lipsey. (2011). Roles for Theory in Contemporary Evaluation Practice: Developing Practical Knowledge. The Sage Handbook of Evaluation.
Peter, T., Campbell, C.P., & Taylor, C. (2021). Still in every class in every school: Final report on the second climate survey on homophobia, biphobia, and transphobia in Canadian schools. Toronto, ON: Egale Canada Human Rights Trust.
Richard Stefano - PME Course progress 