Moving at the Speed of Creativity

Moving at the Speed of Creativity by Wesley Fryer

Marzano Causal Teacher Evaluation Model by Robert La Grassa

These are my notes from Robert La Grasssa’s presentation, “Marzano Causal Teacher Evaluation Model: Based on the Art and Science of Teaching” in Yukon Public Schools on February 21, 2012. Robert is “Director of District Partnerships” for “Learning Sciences International, provider of iObservation.” MY THOUGHTS AND COMMENTS ARE IN ALL CAPS. BACKGROUND: ALL OKLAHOMA SCHOOL DISTRICTS HAVE TO SELECT A NEW MODEL FOR TEACHER EVALUATION BY APRIL 16, 2012. AVAILABLE MODELS ARE THE TULSA MODEL, THE MARZANO MODEL, AND THE DANIELSON MODEL. SEE OEA’S RESOURCES COMPARING MODELS. THE OKLAHOMA STATE DEPARTMENT OF EDUCATION ALSO HAS A PAGE OF RESOURCE LINKS FOR TLE.

State of Florida, which is a Race to the Top state, adopted the same high stakes teacher evaluation process which Oklahoma
– half of Florida districts have adopted the Marzano model, including Broward County
– today we are providing the facts about the Marzano model

Who is Dr Robert Marzano and Learning Sciences International
– our relationship with him goes back to 2008
– Dr Marzano is among most respected national researchers and authors
– LSI provides expertise in field of teacher and principal growth, development and evaluation
– statewide provider of teacher evaluation technical assistance for Departments of Education
– have implementations nationally and in Oklahoma

MY COMMENT: I’M WONDERING WHICH OKLAHOMA DISTRICTS HAVE ALREADY SELECTED LSI?

Research tells us the role of the teacher is the single greatest factor in student learning (Sanders, et al)

Research also tells that one of the greatest factors central office staff can contribute is to maintain a singular focus on improving instruction (Marzano and Waters, 2009)

fundamental question for selecting a TLE model: Is the purpose of teacher evaluation to measure teachers’ effectiveness (solely) or to raise student achievement by helping teachers get better over time?

Marzano Causal Teacher Evaluation model is approved and meets or exceeds all criteria of the Oklahoma Commission on Teacher Leadership

How we got here
– familiar with ‘the widget effect?’
– teacher evaluation systems in the past had binary ratings (unsatisfactory or satisfactory) or a range of ratings
– 98 – 100% of teachers were rated at the top level of those systems
– that may not be a result of the model so much as it can be inflation among raters
– because of those results, following Race to the Top, states are moving to new measures of teacher quality

Components of the Oklahoma Evaluation system (phased in)
– 50% qualitative instructional practice measured by Marzano Framework (or other TLE model adopted)
– 35% is student academic growth, 15% is other academic measures

MY COMMENT: THIS 35% + 15% REFLECTS THE “50% OF TEACHER EVALUATION WILL BE BASED ON STUDENT TEST SCORES” STATEMENT MADE BY MICHAEL BARLOW ON 20 JANUARY 2012.

Reality says: this is now “high stakes evaluation”
– lots of the consequences in this system

We need to have good tools to help teachers grow

Goal: An expectation that all teachers can increase their expertise from year to year which produces gains in student achievement from year to year with a powerful cumulative effect”

41 elements in the student achievement domain 1: Classroom Strategies and Behaviors
– these have the most impact on student achievement
– teachers planning and preparing for units and lessons
– 5 elements: routine segments
– 18 elements: content segments
– 18 elements: on the spot segments

Domain 2: Planning and Preparing
– 3 elements: lesson and units
– 2 elements: use of materials and technology
– 3 elements: special needs of students

Domain 3: Reflecting on Teaching
– 3 elements: evaluating personal performance
– 2 elements: professional growth plan

Domain 4: Collegiality and Professionalism
– 2 elements: promoting a positive environment
– 2 elements: promoting exchange of ideas
– 2 elements: promoting district and school development

60 elements total in 4 domains

“I’m going to show you a way this becomes very easy and efficient to implement”

Principals never go into a classroom looking for all 41 elements
– teachers are never trying to show all of them
– it’s recommended that the model is phased in over 2-3 years

“When implemented with fidelity, these are the kind of results you can get…”

Classroom Strategies Showing Signficant Student Learning Gains

The days of checklist are over: “present / not present”

This is a developmental continuum for teachers to implement research-based strategies
– specific guidance for teachers to improve instruction
– evidences of sufficient implementation to raise student learning
– guidance on the appropriate instructional context (when) to use each strategy to have the highest probability to raise student learning
— helps teachers understand what to do in order to sufficiently improve their teaching to show gains in student achievement

MY THOUGHT AND RESPONSE: THE BIGGEST PROBLEM WITH THIS IS IT PLACES (AS WE’VE SEEN OVER THE PAST DECADE WITH NCLB) A LASER-LIKE FOCUS ON JUST THE RESULTS REFLECTED IN THE STANDARDIZED TESTS. EVERYTHING THAT MATTERS IS NOT ON THE TEST. (EVERYTHING THAT MATTERS CAN’T BE ON THE TEST.) AS A STATE WE ARE PLACING OUR POLITICAL FAITH IN TESTS FOR CCSS WHICH HAVEN’T EVEN BEEN DEVELOPED YET. YES, THERE ARE GOOD STRATEGIES AND IDEAS IN THIS MARZANO TEACHER EVALUATION MODEL. I DON’T THINK THERE IS ANYTHING IN HERE THAT IS LINKED TO WHAT IS CALLED THE ‘VALUE ADD,’ WHICH MEANS GIVING TEACHERS 50% OF THEIR COMPENTENCY SCORE BASED ON THEIR STUDENT TEST SCORES.

I ASKED ROBERT ABOUT THIS SPECIFICALLY, AND HE CONFIRMED THERE IS NOT ANY RESEARCH FROM THE MARZANO GROUP WHICH ADDRESSES OR IDENTIFIES THE VALUE OF MEASURING TEACHERS BY STUDENT TEST SCORES. THIS IS A HUGE ISSUE WHICH WE DO NOT HEAR STATE LEADERS IDENTIFYING. TO MY KNOWLEDGE, THERE IS NOT A SINGLE PIECE OF RESEARCH FOR A DISTRICT OR STATE LEVEL IMPLEMENTATION WHICH SUPPORTS THE BENEFIT OR VALUE OF EVALUATING TEACHERS DIRECTLY BASED ON THEIR STUDENT TEST SCORES. OUR POLITICAL ADMINISTRATION IN OKLAHOMA (LIKE THE ELECTED OFFICIALS IN FLORIDA) HAVE FORCED THIS UPON SCHOOLS, TEACHERS, STUDENTS, PARENTS, ETC. WITHOUT A RESEARCH BASE FOR SUPPORT.

YES, MARAZNO STRATEGIES HAVE A STRONG RESEARCH BASE OF SUPPORT. NO, EVALUATING TEACHERS BASED ON STUDENT STANDARDIZED TEST SCORES (50% AS OUR STATE LAW IN OKLAHOMA MANDATES TO START IN 2012-2013) DOES NOT HAVE ANY SUPPORT IN ACADEMIC RESEARCH LITERATURE. IF I AM INCORRECT IN MAKING THIS STATEMENT, PLEASE ADD SPECIFIC RESEARCH CITATIONS AND LINKS (IF AVAILABLE) FOR THOSE RESEARCH ARTICLES AS COMMENTS TO THIS POST.

We hosting the first Marzano Model Conference in Orlando this year for districts who have adopted the model and are interested in the model June 20-22, 2012

iObservation is our “companion data system” which can be used in calculating the evaluation
– model helps principals and teachers identify where teachers are using specific strategies
– growth continuum model

Information available on www.marzanoevaluation.com/evaluation/products_services/

Innovating – Applying – Developing – Beginning – Not Using

Innovating: adapts and creates new strategies for unique student needs and situations
Applying: signals to students which content is critical versus non-critical and monitors the extent to which students are attending to critical information
Developing: signals to student which content is critical versus non-critical
Beginning: uses strategy incorrectly or with parts missing
Not using: strategy was called for but not exhibited

Goal of our model is to get teachers to the applying and innovating levels of the model

MY OBSERVATION: THIS MODEL IS CLOSELY ALIGNED TO THE “PROFICIENCY SCALES” WHICH MARZANO DISCIPLIES RECOMMEND TEACHERS DEVELOP AND USE FOR INDIVIDUAL STUDENT LESSONS. SEE MY POSTS FOR “TRANSITION TO COMMON CORE STANDARDS BY JAN HOEGH” PART 1 AND PART 2 FOR MORE ON PROFICIENCY SCALES PER MARZANO. I LIKE THIS CONSISTENCY IN HAVING A SIMILAR 0 – 1 – 2 – 3 – 4 RATING MODEL FOR BOTH STUDENT LEARNING AND TEACHER PERFORMANCE.

This tool sends immediate feedback to teacher emails after administrator submits it
– each element has a video with an actual example of the strategy

Tool has place for self-assessment feedback, feedback from student interviews (which the administrator submits directly)

MY QUESTION: SO IF THIS MARZANO TEACHER SYSTEM WAS DEVELOPED TO ADDRESS/CHANGE THE FACT THAT 98 – 100% OF TEACHERS ARE RATED AT THE TOP OF THEIR PROFICIENCY SCALES, WHERE HAS THIS BEEN IMPLEMENTED OVER TIME WITH RESULTS WHICH SHOW HOW THE TEACHER EVALUATION RESULTS ARE LESS SKEWED IN THEIR DISTRIBUTION? I’M ALSO WONDERING WHAT THE ASSUMPTIONS ARE ABOUT WHAT THE DISTRIBUTION OF STATISTICALLY REPRESENTED TEACHER EVALUATIONS “SHOULD” LOOK LIKE. CERTAINLY I WOULDN’T EXPECT IT TO LOOK LIKE A ‘NORMAL’ CURVE. BUT IT UNDOUBTEDLY SHOULD BE A WIDE DISTRIBUTION IF IT’S ACCURATE, BECAUSE WE DO HAVE SOME TEACHERS WHO ARE TEACHING IN WAYS WHICH ARE BELOW EXPECTATIONS. IT’S VERY INTERESTING TO ME THAT THIS IS PRESENTED/MARKETED AS BEING BASED ON ‘THE ART AND SCIENCE OF TEACHING.’ THIS APPEARS TO MAINLY BE AN ATTEMPT TO MAKE TEACHING INTO A STATISTICALLY MEASURABLE SCIENCE. REMINDS ME OF POSTMAN’S TECHNOPOLY.

I’M NOT SAYING EVERYTHING HERE IS BAD, BUT I’M WONDERING ABOUT ITS ASSUMPTIONS… ESPECIALLY AS THEY APPLY TO TEACHING QUALITY AND TEACHER QUALITY. FOR EXAMPLE, WHERE IS THERE AN EMPHASIS ON TEACHER RELATIONSHIPS WITH STUDENTS? WE KNOW THE RELATIONSHIP CONNECTIONS BETWEEN LEARNERS AND TEACHERS CAN BE VITAL. ARE THOSE ELEMENTS IN THIS MODEL? IS THERE ANYTHING IN HERE ABOUT THE TEACHER’S LANGUAGE AND BEHAVIOR COMMUNICATING CONCERN AND LOVE FOR THE STUDENT? THAT MAY SEEM TO SOME INAPPROPRIATE TO INCLUDE IN THIS CONVERSATION, BUT LET’S REMEMBER THAT THIS MODEL IS OSTENSIBLY REPRESENTING EVERYTHING WHICH WE VALUE AS ADMINISTRATORS, EDUCATORS, SCHOOL SYSTEMS, AND COMMUNITIES WHEN IT COMES TO TEACHING AND LEARNING.

I ASKED ABOUT THIS IN THE MODEL: CARE FOR STUDENTS IS RELFECTED IN THIS MODEL UNDER DOMAIN 4: COLLEGIALITY AND PROFESSIONALISM ITEM #56: PROMOTING POSITIVE INTERACTIONS ABOUT STUDENTS AND PARENTS, AND UNDER DOMAIN #1 ITEMS 36, 37 AND 38:
36 = UNDERSTANDING STUDENTS INTERESTS AND BACKGROUND
37 = USING VERBAL AND NONVERBAL BEHAVIORS THAT INDICATE AFFECTION FOR STUDENTS
38 = DISPLAYING OBJECTIVITY AND CONTROL

Dr Marzano’s Video On Demand Library
– not just a collection of random videos
– these are targeted directly toward the domain 1 strategies
– these can be used for PD by teachers

Student growth is not calculated in our system

About value add: “Our experiences with Florida is other systems calculate the value added numbers are computed in other systems, and then those numbers are aligned

So all these scores in these different domains come down to one final, statistical score from 1 to 5 which the teacher has
– superior
– highly effective
– effective
– developing
– unsatisfactory

“This is more than just a system which collects data”
– is designed to facilitate growth, help with professional

I ASKED ABOUT WHEN THE FIRST FLORIDA SCHOOL DISTRICTS HAVE IMPLEMENTED THIS
– Robert’s answer: Florida adopted this in spring 2011, trained educators in summer 2011, this is the first academic year (2011-2012) this has been adapted and is used

How long does this process take (question from a participant)
– time it takes is the same regardless of model

Books recommended to support this model by Robert Marzano:
– Effective Supervision: Supporting the Art and Science of Teaching by Robert J. Marzano (Author), David Livingston (Author), Tony Frontier (Author)
– The Art and Science of Teaching: A Comprehensive Framework for Effective Instruction by Robert J. Marzano

“We have to go back to thinking about high stakes and student achievement, because that is coming”

Administrators are encouraged to select different elements for walkthroughs on just SOME of the elements

MY COMMENT: COPYRIGHT AT THE BOTTOM OF THE “MARZANO ART AND SCIENCE OF TEACHING FRAMEWORK: LEARNING MAP” IS THE FOLLOWING STATEMENT:

Copyright 2011 Robert J Marzano. Can only be digitized in iObservation. iObservation is a registered trademark of Learning Sciences International.

THIS IS THE FIRST COPYRIGHT STATEMENT I’VE SEEN ON ANY PRINTED MATERIAL WHICH SPECIFICALLY LIMITS DIGITIZATION TO A SPECIFIC VENDOR PLATFORM. INTERESTING.

Videos in the iObservation tool now are Flash-based and most are not HTML5/iPad compatible currently

I THINK IT’S VERY IMPORTANT TO NOTE THAT THE iOBSERVATION TOOL IS FULLY FUNCTIONAL TODAY TO SUPPORT THE MARZANO TLE MODEL. IT’S NOT “IN DEVELOPMENT,” IT’S NOT “UNDERWAY,” IT’S FULLY DEVELOPED AND READY TO IMPLEMENT. I THINK IT WOULD BE FOOLISH FOR ANY SCHOOL DISTRICT TO ADOPT A TEACHER EVALUATION TOOL TODAY WHICH DOES NOT HAVE A FULLY DEVELOPED / READY TO IMPLEMENT DIGITAL/ONLINE TOOL. IF IT WAS 1980 MY OPINION ABOUT THAT WOULD BE DIFFERENT, BUT SINCE IT’S 2012 THE ONLINE/DIGITAL TOOL ASPECT OF THIS TEACHER EVALUATION SYSTEM IS CRITICAL.

Summary of Marzano Teacher Evaluation

From:
– compliance focused, annual reviews that are inflated and LACK specific guidance for instructional improvement
– misaligned system without specificity in the common language of instruction
– ambiguity and subjectivity due to lack of specificity
– lacks connections to student achievement gains

To:
– formative and summative process that is timely, specific, and honors growth over time
– coherent research-based common language of instruction with clear and objective measures and teacher and student evidences
– clarity and consistency, from the newest teacher to the most veteran practitioners and supports accuracy for observers
– causal links to raising student achievement

2 conditions having the biggest affect on fidelity of the implementation of this model: appropriate time to use a strategy and the appropriate level of the strategy to implement
– these are tools in the toolbox, but they need to be utilized appropriately and effectively

35 years of research by educational practitioners have developed this model
– has had success in rural, suburban, and urban schools/districts
– over 1000 studies validate the Marzano framework as an instructional model for raising student achievement (not correlational data from other states, there is data from Oklahoma schools

METs (Measures of Effective Teaching from Gates Foundation) studies recommend evaluators have 35-40 hours of training before evaluating teachers in a high stakes situation
– SO THAT IS A HUGE FINDING WHEN IT COMES TO PROFESSIONAL DEVELOPMENT FUNDING AVAILABLE FOR OUR ADMINISTRATORS TO OBTAIN PRACTICAL TRAINING FOR THIS EVALUATION PROCESS

Implementation, Efficacy, and Fidelity are the three aspects we really emphasize in implementing

Currently iObservation does not support active directory integration to support single sign on in districts
– districts have to export files and have them imported, teachers have a separately maintained login userid/password for the iObservation system

MY COMMENT ON THIS: IOBSERVATION SHOULD ABSOLUTELY IMPLEMENT ACTIVE DIRECTORY SUPPORT FOR DISTRICTS SO TEACHERS CAN HAVE A SINGLE SIGN-ON

Other approved resources:
– Book studies are a very effective way to communicate the shared language of this model
– online courses are available for teachers in all elements of domain 1
– onsite coaching for principals
– data coaching
– additional workshops on all areas of the model
– professional, experienced consultants

Responses to some misconceptions about the model:
– elements can be phased in over time and select focus areas in domain 1
– observations can take the same amount of time regardless of the framework, the important question is how powerful the feedback is can can be
– there are low cost comprehensive training solutions

MY CLOSING THOUGHT: I AGREE THERE ARE LOTS OF POSITIVES ABOUT THIS, BUT THE COMPLETE LACK OF RESEARCH BASE SUPPORTING THE CONNECTION OF TEACHER EVALUATION TO STUDENT STANDARDIZED TEST SCORES IS A HUGE PROBLEM. THIS IS A POLITICAL PROBLEM WE HAVE, IT’S NOT A PROBLEM WITH THE MARZANO MODEL OR LSI AS A VENDOR. THEY ARE SIMPLY RESPONDING TO THE POLITICAL AND BUDGETARY ENVIRONMENT WITH SOME OF THE THINGS THEY ARE SAYING HERE. WHAT’S NOT BEING SAID IS WE HAVE NO RESEARCH BASE THAT AFFIRMS THE VALUE AND OUTCOMES-BASED BENEFITS OF DIRECTLY TYING TEACHER EVALUATIONS TO STUDENT TEST SCORES. THAT’S THE ELEPHANT IN THE ROOM, AND I HOPE OUR EDUCATIONAL LEADERS IN OKLAHOMA WILL ADDRESS IT HEAD ON.

'Elephants' photo (c) 2008, Elroy Serrao - license: http://creativecommons.org/licenses/by-sa/2.0/

Good resources are available from the Oklahoma Education Association comparing TLE models

If you enjoyed this post and found it useful, subscribe to Wes’ free newsletter. Check out Wes’ video tutorial library, “Playing with Media.” Information about more ways to learn with Dr. Wesley Fryer are available on wesfryer.com/after.

On this day..

Posted

in

,

by

Wesley Fryer

Tags:

My Podcasts!