These are my afternoon notes from Jan K. Hoegh’s presentation on 24 January 2012 in Oklahoma City for the Oklahoma Curriculum Improvement Commission. (This is the link to my morning notes.) Jan is a consultant for the Marzano Research Lab. (@marzanoresearch on Twitter – Jan is not on Twitter yet.) MY THOUGHTS AND COMMENTS ARE IN ALL CAPS. The official description of this full-day workshop was:

Currently, a transition to Common Core Standards is prevalent in the educational environment. This session will overview key steps in successful Common Core implementation. A focus of the presentation will be resources the Marzano Research Laboratory has developed to support teachers and administrators in this transition from state standards to Common Core Standards.

ONE OF THE PARTICPANTS NEXT TO US ASKED JAN ABOUT COMING TO THEIR DISTRICT FOR PD, SHE QUOTED $6K PER DAY AS HER PRICE

Handouts from today are linked on www.ossba.org/ocic

Effective pedagogy in “Classroom Strategies that Work” by Robert Marzano
- curriculum articulation
- instructional methodology
- encapsulating both within classroom management

The instructional methodology piece is the one we hold closest as teachers

Now lets think about the definition of college and career ready
- this should make us think about instructional practices in our classrooms

think about the term “deep understanding” carefully
- if we are to be effective in that orchestration of complex skills, deep understanding is key

Now we’re going to talk about young people growing up today

Reference April 2010 FastCompany article: “A Is for App: How Smartphones, Handheld Computers Sparked an Educational Revolution
- name things kids today have no concept of because of their generation

We need to think about skills kids need now

Take a look at the research (Wells & Croxton, pg 1)

Schools may once have done an adequate job of equipping and sifting young people to take their various places in the prevailing society, but that world is gone.

Now showing a YouTube video listing short change facts “Did You Know” style

PISA assessment
- “US students scored lowest on the problem-solving items.”

Jobs for the Future (2005) “Fore every 10 students who enter eighth grade, only seven graduate high school on time, and only three complete a postsecondary degree by age 26″

MY THOUGHTS: IT IS SO IMPORTANT TO CONSIDER THESE QUOTATIONS AND STATISTICS IN CONTEXT, AND THAT CONTEXT IS NOT BEING PROVIDED HERE. THIS IS A TYPICAL PRESENTATION OF “NATION AT RISK” STYLE STATS: OUR SCHOOLS SUCK, WE’RE IN A CRISIS, SO WE BETTER CHANGE. NO DISCUSSION OF POVERTY, CHANGING DEMOGRAPHICS, WHAT STUDENTS TAKE THE PISA IN OTHER COUNTRIES VERSUS US SCHOOLS, ETC. THE CITATION ON THE PROBLEM SOLVING PISA RESULTS IS DARLING-HAMMOND, 2010, P. 35, BUT THE MAIN FOCUS OF HER RESEACH (FROM WHAT I’VE READ) ABOUT TEACHER QUALITY IS NOT MENTIONED AT ALL. MORE DE-CONTEXTUALIZED CITATION OF “RESEARCH SAYS.”

Two resources to recommend to focus on cognitive skills and cognitive skills
- “The Highly Engaged Classroom”
- “Teaching and Assessming 21st Century Skills”
- surveyed 10,000 teachers, asked 3 questions: top 3 challenges you face as a classroom teacher
– #1 students reading below grade level
– #2 unsupportive home environment
– #3 student not being engaged in the learning process

Cognitive Skills
- mental abilities like logic, reasoning, thinking and attention to detail
- teaching of cognitive skills is not unique to the 21st century (VERY GOOD POINT, GLAD SHE POINTED THIS OUT)
- what is new to the 21st century is the idea that cognitive skills should take a dominant role in the curriculum
- analyzing and using information, addressing complex problems and issues, creating patterns and mental models

Conative skills
- the process of combining what one knows (cognitive) with how one feels (affective) and deciding what action to take in light of both
- understanding and controlling yourself, interacting with others

identifying “faulty logic” is a key element in CCSS

What of all this would we call “securely held content”
- what pieces of content are going to remain with my students forever
- the reality is nonoe of it will (so process skills are most important, not content)

Consider the students you are preparing for the 21st century
- discuss implications for educators and critical steps for successful implications of CCSS

OCIC part 2

How do we use proficiency scales to build higher quality assessments?

We don’t know what those assessments are going to look like?
- what assessment practices to support Common Core implementation?

Need specific items that support level 2, 3 and 4 scale outcomes
- most of your assessment items shouldn’t be level 4, ht need to be distributed

Research by Jorissen 2006: “Most Tachers say they develop their assessment knowledge and practices on the job”

These are about how we prepare teachers to craft assessments

MY THOUGHT: THIS IS DEFINITELY THE KEY TO WHAT WE NEED TO BE DOING IN PD

Rick Stiggins, 2008: “We have trusted those we believe… (trusted our textbooks)

Educators must gain deep understanding of how to develop high-quality classroom assessments that inform instructional assessments

He do we build quality classroom assessments

Recommend: “Classroom Assessment and Grading that Works” by Marzano 2006
- “Formative Assessment and Standards Based Grading” by Marzano 2010
- Teacher-Made Assessments: Gareis and Grant 2008

I believe in dual coding theory, we need both words/text and non-linguistic representations

Forms of assessment
- obtrusive
- non-obtrusive
- student generated

Obtrusive assessments interrupt the normal flow of activity in the classroom

Student-generated assessments are the most underutilized form, students generate ideas about how they can show they have mastered the content, students will create something and share it
- we can’t do this kind of assessment all the time

Classroom assessment = anything a teacher does to gather information about a student’s knowledge or skill regarding a specific topic

“Purpose of assessment is to enhance the experience”

MY THOUGHT: THIS MAY BE THE MOST THOUGHT PROVOKING AND CHALLENGING THING SHE HAS SHARED ALL DAY. THIS VIEW OF ASSESSMENT IS SUCH A FAR CRY FROM HIGH STAKES TESTING…

Why should we do it?
- research reported here shows conclusively that formative assessment does improve learning

A quality test is:
- valid: accurately and appropriately measuring what is relevant
- reliable: consistent and precise
- fair: does not put any group at a disadvantage

Tests/assessments must have these three elements
- we’ll spend the most time talking about validity today (“if you build your lessons on our scales, validity will be there”)

reliability
- refers to the degree to which a test is consistent and stable in measuring what it is intended to measure, mist simply put a test is reliable if it is consistent within itself and across time

Remember the PARCC website has some sample items for assessments, but we don’t really know if those will be representative of the eventual assessments which are developed

We know the following things WILL be in the CCSS assessments:
- assessments may be perforance-based and multiple choice
- assessments may require technology skills to respond to questions
- technology skills will be required to respond to questions (this means kids will take the tests online)
- assessments may require critical thinking, for example: claim-evidence-counter-claim approach, compare and contrast tasks, cause and effect relationships
- assessments may require writing in response to text
- assessments may be timed
- a different approach to instruction may be required

HIgh quality item types:
- true/false use those sparingly (strong statements include a single idea, not multiple ideas for T/F)
- true/false need to be black/white, not shades of grey

Matching assessments
- homogenous in content
- matching set no more than 7
- uneven number of items to be matched or items
- uneven number of items to be matched OR items may be used more than once
- ordered logically (such as alphabetically)
- longer reading on the left, matching items on the right

Multiple choice
- being with question-starter (who, what, where, how, why)
- a single task is articulated in the question
- emphasize qualifiers
- all response items are plausible (no-throw away)
- written in present tense if possible
- avoid negatives (e.g., which is NOT)
- may include context: more complex questions

emphasizing qualifiers
- kids will be required to come up with the BEST answer, so give students experience with that as you craft items

there will be “none of the above” and “all of the above”
- in Nebraska we didn’t allow this on our state test, we thought it was poor test item construction
- a good friend of mine is now a high level test item developer for PARCC and she says these items will be included

Quality Constructed Response Items
- require students to justify their reasoning
- fill in the blank, short answer, essay

tips for fill-in-the-blank
- position the blank at the end of the sentence as often as possible
- limit the number of blanks in an item
- blanks should be the same length
- be sure information prior to/surrounding the blank is adequate
- may use a word bank

Use logs of analogies: those are high level thinking required

Short answer and essay items
- make the nature of the response desired clear to the reader
- develop and communicate a scoring criteria for the question
- provide adequate space for responses

Practice makes perfect
- absolutely include these types of items on your assessments
- Common Core Will!

You do NOT need to build assessments for all of the scales you will use, but you (as a classroom teacher) definitely DO need to craft and use a variety of assessments

Video shown as an exemplar: “Assessment at the Utmost”

My Favorite No
Grade 8, Math, Warm Up Routine
Common Core Standards: Math.MP.6

My Favorite No < Teaching Channel

When I work with district I encourage them to look at the scales and determine which ones could be combined
- you can do that, your design goes into the classroom assessment process

How many items do I need to be reliable?

MY THOUGHT: I THINK WE MAY BE PLAYING FAST AND LOOSE WITH THE CONCEPT OF ‘TEST RELIABILITY’ HERE, ESP AS IT IS CONSIDERED IN ACADEMIC SETTINGS (HIGHER ED)

If an assessment is valid and reliable, inferences about learning are more appropriate

Fairness: assessment level is appropriate
- readability
- avoid types of bias: offensive content, stereotyping, unfair representation, more…

format: directions, enough space, visually appealing

MY THOUGHT: IF WE’RE ASKING ALL TEACHERS TO BECOME ASSESSMENT DESIGNERS AND EXPERTS (WHICH IS NOT AN ENTIRELY BAD IDEA) THEN WHERE ARE THE ONLINE CCSS WEBINARS (FREE BECAUSE THEY’VE BEEN PAID FOR WITH TAXPAYER DOLLARS) FOR TEACHERS AND SCHOOLS TO UTILIZE ON SPECIFIC TOPICS LIKE ASSESSMENT CREATION?!

assessment format issues: the format of the test can district students taking a test

around the country I’m seeing a little bit of everything with CCSS implementation
1- year to year model
2- key leader model
3- shared resources model
4- scheduling options: early dismissal, late start, PLC meetings, PD days, summer meeting time

Consider the development of an ‘explicit development model

First and foremost: everyone must be on the same page with respect to goals
- what are our desired outcomes
- identify your implementation process

MY CLOSING THOUGHTS: I REALLY THINK WE NEED (IN OKLAHOMA) TO STOP PUTTING OUR EDUCATIONAL FAITH IN STANDARDS AND HIGH STAKES / COMMERCIAL ASSESSMENTS, AND INSTEAD INVEST OUR RESOURCES AS WELL AS FAITH IN DEVELOPING HIGH QUALITY TEACHERS. TEACHERS ARE THE KEY. I AM THINKING WE SHOULD START AN ADVOCACY GROUP WHICH PUSHES FOR 2 MAIN THINGS:
1- STOPPING THE DRIVE TO LINK STUDENT TEST SCORES TO TEACHER EVALUATION
2- WITHDRAWING FROM OUR FORMAL PLEDGES TO SPEND MILLIONS ON CCSS ASSESSMENTS, STOP STATE-LEVEL MANDATED HIGH STAKES ASSESSMENTS AND INVEST ALL THOSE DOLLARS INSTEAD INTO TEACHER PROFESSIONAL DEVELOPMENT, TEACHER MERIT PAY AS NATIONAL BOARD CERTIFICATION PAY, AND IN TEACHER PAY. GOOD TEACHERS ARE THE KEY. STANDARDS AND ASSESSMENTS ARE NOT SILVER BULLETS. THIS ISN’T 1983, WE DON’T NEED A ‘NATION AT RISK’ RE-RUN. WE NEED TO TURN THE SHIP OF POLITICALLY MANDATED EDUCATIONAL MALPRACTICE IN OUR STATE. OUR TEACHERS, PARENTS, AND OTHER STAKEHOLDERS HAVE BEEN BRAINWASHED OVER THE PAST DECADE TO BELIEVE THAT EDUCATIONAL QUALITY AND WORTHY EDUCATIONAL ASSESSMENT CAN ONLY BE ACCOMPLISHED THROUGH EXPENSIVE, HIGH STAKES ASSESSMENTS WHICH WE PAY COMMERCIAL COMPANIES TO DEVELOP. THAT IS A FARCE AND A LIE. WE MUST CHANGE THIS MINDSET AND THIS REALITY. THE PURPOSE OF SCHOOL IS NOT TO TAKE TESTS WELL. THERE ARE FAR MORE THINGS THAN JUST ‘ACADEMIC ACHIEVEMENT’ WHICH MATTER IN SCHOOLS. WE MUST ALWAYS PRESENT TEST SCORES AND RESEARCH RESULTS IN CONTEXT. WE MUST TALK ABOUT POVERTY AND HELPING STUDENTS AS WELL AS FAMILIES MOVE UP AND OUT OF POVERTY. NEW ACADEMIC STANDARDS AND NEW, FANCY, ONLINE ASSESSMENTS ARE NOT THE ANSWER TO THOSE CHALLENGES. PASSIONATE, DEDICATED, HIGHLY QUALIFIED TEACHERS DEFINITELY ARE A KEY PART OF THAT SOLUTION. THERE ARE SOME GREAT ELEMENTS OF CCSS WHICH WE SHOULD EMBRACE AND IMPLEMENT. THERE ARE OTHER KEY PARTS OF OUR EDUCATIONAL POLITICAL LANDSCAPE, HOWEVER, WHICH WE MUST STAND UP TO OPPOSE AND CHANGE.

WE’VE GOT TO TURN THE SHIP. AFTER ALL, WE LIVE IN THE USA. IT’S TIME TO SHOW HOW WE CAN ADVOCATE FOR DEMOCRATIC CHANGE IN A RESPONSIVE REPUBLIC.

Costa Concordia sinks

 

Technorati Tags: , , , , , , , , , , ,

 


Did you know Wes has published 3 eBooks, and 1 of them is available free? Check them out!

If you're trying to listen to a podcast episode and it's not working, check this status page. (Wes is migrating his podcasts to Amazon S3 for hosting.) Remember to follow Wesley Fryer on Twitter (@wfryer), Facebook and Google+. Also "like" Wesley's Facebook pages for "Speed of Creativity Learning" and his eBook, "Playing with Media." Don't miss Wesley's latest technology integration project, "Mapping Media to the Common Core / Curriculum."

On this day..

Share →
  • Guest

    So if you don’t have assessments, how do you measure what students are learning in a valid and reliable way?  What’s the alternative?  And again, how can it be done in a way that is consistently reliable from teacher to teacher, school to school.  

  • http://wfryer.wpengine.com Wesley Fryer

    I never said we do away with assessments. Every good teacher is always assessing. I said we do away with high stakes, standardized assessments that we pay big bucks to commercial curriculum companies to develop. That is part of the new regime of educational policy ushered in by the recent Texas governor who became our President, and unfortunately remained the educational policy of our current chief executive.

    Of course students will still take tests and be constantly assessed in various ways. They’ll also take tests like the ACT/SAT and NEAP. What we should NOT do is pay millions for high stakes examinations which cause teachers in our schools, in many cases, to focus an inordinate amount of attention on test prep. This is only going to get worse with proposed CCSS assessments. This is a toxic educational learning culture and we need to change it at a political level.

  • Guess

    That’s not what I asked. I asked how you do assessment in a valid and reliable way. You’re against standardized tests that measure reading proficiency. What should be the replacement? And how does your alternative meet validity and reliability principles?

    It feels like you’re ranting more than offering am alternative,

  • http://wfryer.wpengine.com Wesley Fryer

    What is your real name?

  • Pingback: Week 2: Moving at the Speed of Creativity | Katie's Agricultural Class

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.

Made with Love in Oklahoma City