Richard Stefano – Additional PME-802 Materials and Notes for Program Evaluation Design

Below are some additional materials (mostly for my own use) to develop my understanding of Evaluation and improve my Program Evaluation Design in the form of a program rationale, program plan, and logic model based on Chen (2005) and Wilder (2009).

Program rationale:

  1. Problem – 2SLGBTQI students often have negative mental health and well-being, internalized shame and low self-esteem and struggle with academic success, school attendance and high dropout rates.
  2. Target group – teachers in Ontario schools
  3. Goal – Improve mental health of 2SLGBTQI students in schools and improve school environment.
  4. Determinant – Teachers become more educated on 2SLGBTQI issues and how to implement inclusivity in the classroom.
  5. Intervention – Webinars provided to teachers and resources available on Egale organization website.

Program plan:

  1. Implementing organization – Egale organization and school boards or individual schools.

*2. Program implementers – Teachers provided with training and resources by Egale

  1. Associate organizations and community partners – School boards/schools and the Ontario government (providing funding)

*4. Ecological context – Reports done in 2011 and 2021 determined that Ontario schools were not an inclusive, supportive environment for 2SLGBTQI students.

*5. Intervention and service delivery protocols – Webinars and free online resources to educate teachers. Teachers better understand 2SLGBTQI issues, implement 2SLGBTQI-focused lessons in their curriculum, and use inclusive language in the classroom.

*6. Target population – Students in Ontario schools. The focus is on improving the mental health of 2SLGBTQI students in Ontario schools, however: there will also be an effect on staff in those schools, in other students at the schools, and in the community connected to each affected school.

References:

Chen, H.-T. (2005). Practical program evaluation: Assessing and improving planning, implementation, and effectiveness. Thousand Oaks, CA: Sage Publications

Wilder Research. (August 2009). Program theory and logic models. Retrieved May 19, 2022, from: http://www.evaluatod.org/assets/resources/evaluation-guides/logicmodel-8-09.pdf

What’s your approach to curriculum?

What’s your approach to curriculum?

Even if you’ve been given guidelines or a complete curriculum by your school, school board, or a previous teacher, you probably have an idea of what you feel is important when it comes to designing and implementing your course.

Click the picture that best fits your personal philosophy:

Do you feel that your relationship with your students is a top priority? Do you adjust your activities constantly to meet their needs?

Can learning be broken down into a set of clear steps? Do your students need to meet certain expectations to succeed?

Do you prioritize what your students will do when they leave your class? Do you feel your students can “change the world”?

Click here for more information about curriculum approach!

PME-810 – Module 3 – Updated visual – Kathryn, Heidi, Richard

Our visualization changed significantly from our first Coggle design in Module 2. The first reason is purely practical: the more we added, the less clear our visual became for an outside reader. Second, one of the benefits of an online program is learning about and experimenting with new collaborative technology. Coggle is one tool, but we felt it would be good to try a new, slightly more complicated method for sharing our work. Finally, rather than linking the five conceptions of curriculum to philosophies and attaching other concepts to link them, we felt it was better to see each approach as a separate piece and see the concrete, practical ways those approaches work in terms of planning, instruction and assessment. From there, the strengths and weaknesses of each approach became clearer. 

The Subject-centred approach is considered an “objective” approach (McMillan, 2014) based on standardized testing as a method of determining student success and making learning efficient (Shepard, 2000). Subject-centered curricula often come from government or school district policy and may not allow for student-specific approaches and are delivered at the age or grade level despite variations in student ability, background knowledge or interest. Planning is based on conveying individual skills or pieces of knowledge for each subject that can be measured using uniform tests. Subject matter is generally delivered in chronological order and organized by subject matter. Instruction is often uniform, through teacher centred approaches, such as lectures, and assignments, and may rely on rote memorization and “teaching to the test” (McMillan, 2014). Additionally, this curriculum may privilege Western-centric views of the word and ways of knowing. Assessment is summative and measured against externally imposed standards, and it is a priority for teachers and schools to maintain government or district standards. In-class assessment is often not classroom- or student-specific and reflects the uniform content delivered. It’s form can mimic the external standardized assessment, as McMillian (2014) observes assessments may be a constructed response like multiple choice, matching, true and false, short answer, or a diagram to be labelled. Despite the limitations of this approach, “backward mapping” (starting with assessments and outcomes and creating a course that leads to them) can lead to positive outcomes (Hayes, 2003). Having local teachers or administrators make mid-course adjustments to content and instruction can also balance standards-based learning with a more student-centred approach. 

Society- or problem-centered approaches are, by design, more local and personalized in their approach. They presume, as Sowell (2005) observes, the purpose of education is to either prepare students for the world, or enable them to reform it in the future, or ambitiously, to do both. This approach to curriculum design is not as widely utilized as subject centered approaches for as Orstein and Hunkins (2013) note, planning, instruction and even subject areas addressed must be tailored to the problem selected and the needs, interests and abilities of the learners targeted. Unlike subject-centered approaches, planning may not be completed in advance of instruction, rather it must be revisited throughout the process in response to unknown outcomes as they arise during instruction and to ensure that any predetermined learning outcomes are interwoven into the learning process through instruction or selection of resources.  

Instruction in this approach often takes a dual focus with problem solving and social relation skills being addressed in addition to the content, that is the how is equally, if not more important than the what. The role of the teacher in this approach may be understood as a facilitator, providing appropriate resources and fostering opportunities to practice social and problem-solving skills in a supported environment.  Assessment in this approach may vary widely.  Students may be measured against their own individual starting point, or against external criterion depending upon how this design is implemented. This approach may use teacher-created standards, standards co-created with the students, or external standardized measures such as reading levels or a combination of these. Samuelsson et al. observe that a child’s degree of involvement may be seen as an indicator of learning. As observed by educators in A Teachers Point of View of Starting Inquiry Based Learning in the Classroom (n.d.), assessments for the same content or experience may vary by student as different marking tool may be utilized for different projects or students within the group. The same educators note that assessment may be understood as being ‘expanded’ in this approach to include representations of learning such as portfolios, finished products or journaling and that assessment should be a tool of not only measuring and demonstrating student growth, but also for driving teacher growth through reflection, self-assessment and collaboration (A Teachers P.O.V., n.d.).  

Finally, the learner-centred approach is the most personalized of the three approaches. It conceptualizes the learner not as a receptacle for knowledge but rather as a source for curriculum design. Ornstein and Hunkins (2013) posit that in this perspective the curriculum should be developed in response to how a learner (or learners) learn, form attitudes, generate interests, a develop values (p. 154). Unlike subject-centred curriculum design, planning is undertaken throughout the course of instruction and is not the sole domain of the teacher. Rather learners are invited to help select and organize the content they will learn (Sowell, 2005). Instruction may center around constructing projects or experiential approaches for the students to fully encounter results first-hand. Samuelsson et al. (2006) summarize this type of instruction concisely with their depiction of the Plan-Do-Review model, where the teacher supports students through a routine of planning learning, undertaking the task as planned and then reviewing the activity and its outcome with a guided reflection. Instruction is learner centered and the skills needed for learners to work together are deliberately and explicitly taught and cultivated. Subject matter from a variety of disciplines or subject areas is combined to support the learners to understand and solve social problems and to meet their developmental needs (Orstein and Hunkins, 2013). Assessment extends through the entire learning process, as McMillan (2014) notes, with pre-assessments, formative and summative assessments being employed. These assessments may take many forms and may be selected by students to best reflect their preferred learning style or goals. Self-assessment of learning for both the students and the teachers is emphasized, as the teachers model growth and collaboration towards learning goals with their colleagues.  

REFERENCES 

Canadian Education Association. (2011, August 31). John Ralston Saul: Where is the Standardized Testing Trend Taking Us?. [Video file]. Retrieved from https://vimeo.com/28412154 

Canadian Education Association. (2013, March 11).The Power of Student Voice to Enhance Teacher Practice.. Retrieved from https://vimeo.com/61528845 

Canadian Education Association. (2014, January 30). A Teacher’s P.O.V. on Starting Inquiry-based Learning in the Classroom. . Retrieved from https://vimeo.com/85470752 

Hayes, D. (2003) Making learning an effect of schooling: aligning curriculum, assessment and pedagogy, Discourse: studies in the cultural politics of education, 24(2), 225-245 

McNeil, J. D. (2009).  Contemporary curriculum in thought and action (7th ed.).  Hoboken, NJ:  John Wiley.  Pages 1, 3-14, 27-39, 52-60, 71-74. 

McMillan, J. H. (2014). Classroom assessment: Principles and practice for effective standards-based instruction (6th ed., pp. 1-20, 57-64, 74-88). Boston, MA: Allyn & Bacon. 

Ornstein, A. C., & Hunkins, F. P. (2013). Curriculum: Foundations, principles, and issues (6th ed.). Boston, MA: Pearson.  Read Chapter 6, pp. 149-173. 

Samuelsson, I. P., Sheridan, S., & Williams, P. (2006). Five preschool curricula—comparative perspective. International Journal of Early Childhood, 38(1), 11. 

Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4-14. doi:10.3102/0013189X029007004 

Sowell, E. J. (2005). Curriculum: An integrative introduction (3rd ed., pp. 37-51). Upper Saddle River, NJ: Pearson. 

TEDTalks. (2013, May 10). Ken Robinson: How to escape education’s death valley. [Video file]. Retrieved from https://www.youtube.com/watch?v=wX78iKhInsc 

FINAL: Completing my Inquiry Project

So: I am prepared for the first unit of my course. I have a learning goal for my students, I have created a plan for each lesson, I can use my and my students’ reflections to improve my curriculum, and I can demonstrate all of this to the new reading teacher arriving this semester.

What worked: My goal motivated me and “forced” me to complete smaller goals over a long period of time. It helped my self-efficacy and made me aware that I can complete self-set goals without (much) external motivation. I completed goals that I had considered for my last four years of teaching that can help me in the future, and it showed me that I can do something similar for each unit. It also gave me insight into my own thoughts, and gave me some possible strategies to use moving forward.

What didn’t work: I fell back quickly on old habits. I finished most of the goals at the last minute, did not seek help or use a diverse group of resources, and my work was not specific and still relies too heavily on my ability to improvise in the classroom.

Why? The first problem was the virus affecting my work schedule. With no classes, I was unable to reflect on my lessons or see how they affected the students. But the larger problem was with my own brain. I could get away with doing things on the last possible day, and relying on my existing knowledge and skills. So I did.

What can I do to change? I hope the first step in changing was taking this course. Being aware of goal-setting, SRL and reflecting on my work has already increased by self-efficacy and I hope it will help me develop my skills more in the future. In reflecting on my goals, I believe that shorter, more numerous proximal goals would help me be more specific and to work regularly each day. I would like to experiment with setting rewards for myself, and choosing one specific day of the week to complete reflections.

5. Student Feedback and Reflection

I tried to correct some of my previous issues. I used outside resources for some reflection ideas and tried to create a specific task for the students to complete. But most of the work is still a loose discussion that I can have with my students after each unit. I prefer teaching that way, but I also feel it is more dynamic and gives me more useful information, I can also nudge my students towards different ideas and adapt as they answer the questions. I can then use my previous reflection document to assess how the feedback has helped me and the effect I think it had on my students. And then adjust as I move forward.

I tried to balance questions focused on my own improvement of the course (to benefit me and my students) with questions that would spur self-reflection and meta-cognition in my students:

4. Creating a reflection document

While writing this reflection document, I really realized the flaw in my distal and proximal goals. They aren’t specific enough. I’m doing the same thing that I always do, which is trusting that I can do a good job without organization, curriculum or detailed plans. That can be a benefit when teaching critical thinking. It’s important that I’m able to go off the map and have strange, unplanned discussions with my students. But “plan for the worst” means having each lesson, each reflection, and each interaction planned out as much as possible. That’s the best way to ensure that – at the very least – my students are able to demonstrate the knowledge and skills I was intending to cover in the unit. And it also means that any new teacher can take my curriculum, understand it, and either reproduce it or adjust it based on their own teaching style. However, these reflections might be better saved for my final submission. For now, I found the reflection document quite straightforward. It’s very similar to some critical thinking exercises I’ve done with my students before. The worry is – as always – that I won’t continue to use it for each week’s lessons. I hope if I can continue doing this for each unit, I can develop my SRL skills to the point where I know how to motivate myself to complete the reflections every week instead of “Work’s done, time to sleep!” every Friday.

3. Creating an outline for Unit 1: Creative Writing

As I discussed before, my major obstacle was a lack of motivation due to external circumstances. I had to start looking at my own monitoring document as a real assessment; the same way I looked at my coursework. I decided to combine my next two goals into one single, achievable document. It was a flexible lesson plan (or at least an outline) and a goal-setting document for the unit. However, in combining these two goals I made them less specific. My lesson plan is really an outline of what I did last year, rather than a new plan that takes advantage of some of the ideas in this course. For now, I’m okay with that. My next step is to create a reflection outline for each lesson. Using this, I can go back and edit my lesson plans based on my own reflections and student feedback. If I look at my own thinking, I made my goal slightly more attainable in an effort to improve my self-efficacy. I still feel I accomplished something in creating this outline, and I feel it will help me moving forward. My original goal was not helping me motivate myself. Below I have attached my outline:

2. Course outline for new teachers

One of my first proximal goals was to create an outline for new Reading Teachers. I’ve had to do this before, but I’ve never put it in writing beyond an email following by a personal discussion. I reflected on two things: 1, the questions and concerns I had and the things I struggled with when I first arrived, and 2, what problems arose when other teachers first arrived and what were they confused by?

I remember when I arrived, I followed the curriculum of the other reading teacher, and it took several weeks to realize how each part of the course was helping students prepare for study overseas. I was lucky that I’d worked with Chinese students in Canada before, so I understood how necessary a critical thinking course is for many of them. I feel it’s a good idea to make this explicit for new teachers, especially since I might not be aware of their familiarity with international students.

Since becoming the senior reading teacher here, I’ve made many changes to the course, and I feel it does a good job of preparing students of many different English levels. I’ve created a possible message to send to a new reading teacher, with the assumption that they will have many follow-up logistical questions, and I will have to go over individual units and lessons with them in person.