Moving at the Speed of Creativity by Wesley Fryer

Schools must be data informed: NOT data driven

This weekend I finished reading Diane Ravitch’s excellent new book, “The Death and Life of the Great American School System.” I highly commend this book to anyone interested in the topic of education reform. I agree with much of what Ravitch writes, and made extensive notes as well as annotations in my paper copy of her book as I read it. I’ll be sharing quotations and excerpts in the weeks ahead. This evening I’d like to share the following paragraph from page 228 of the book, in the chapter, “Lessons Learned.” We hear a great deal in our schools and in educational technology circles about “data driven decision making.” On this subject Ravitch writes:

Our schools cannot be improved by blind worship of data. Data are only as good as the measures used to create the numbers and as good as the underlying activities. If the measures are shoddy, then the data will be shoddy. If the data reflect mainly the amount of time invested in test-preparation activities, then the data are worthless. If the data are based on dumbed-down state tests, then the data are meaningless. A good accountability system, whether for schools, teachers, or students, must include a variety of measures, not only test scores. To use a phrase I first heard from educator Deborah Meier, our schools should be “data informed,” not “data-driven.”

In many of our public schools and in state departments of education today, “blind worship of data” IS the order of the day. This is wrong, and is a reality which must change. In many ways, Ravitch paints a clear picture in her book of the immoral political and educational culture in which we live today. At the behest of politicians, educators nationwide have been told to view students as statistics, not as individuals, and to view the purpose of the educational enterprise as raising test scores rather than developing capable minds. We know as parents and educators that we need our schools to do FAR more than simply teach students the basics of literacy and numeracy. Yet astonishingly, we have tolerated a political culture which places exclusive emphasis on those two content areas to the exclusion of all others. This is a policy which is clearly and dramatically mistaken, and it is up to us to change it.

The next time someone says to you, in a meeting or in conversation, “We need to do this to support data-driven decision making,” I challenge you to challenge them. Respond by saying, “We need to be data informed, but NOT data driven.” Then discuss what this means.

Tests can provide a window into learning, understanding, and retention, but that window is ALWAYS incomplete. We need to stop pretending like we work, as educators, to simply serve data and the masters of data. We don’t and must not act as if we do. We work with children and for children for far more than to simply raise test scores in reading and math.

– Posted using BlogPress from my iPad



, ,




13 responses to “Schools must be data informed: NOT data driven”

  1. Tim Holt Avatar

    Good luck. NCLB specifically mentions “Data Driven Decision Making.”
    Entire industries have sprouted up to support this.

    That argument should have been made about 10 years ago. As it is, we are waaaay to invested in DDDM.

  2. Techyturner Avatar

    Thanks for the recommendation of the book. I will find a copy of it and read it for myself. I too have heard “Data Driven Decision Making” in my career and it never felt right. Now I understand a little more of why. To be “Data Driven” one must make sure ALL the factors that can effect the outcomes are eliminated or the same for all participants. This can be done in a laboratory, but not with children.
    Thanks for today’s summary and the future one’s as well.

  3. Brian Crosby Avatar

    I like it Wes. Maybe an argument that should have been made 10 years ago, but how many would really have been informed enough about what was coming to know to make it? I think it is a worthwhile point to make whenever we can … Data Informed Decision Making!

  4. Wesley Fryer Avatar

    I agree this perspective would have been good to articulate 10 years ago when NCLB was conceived, but it is never too late to share truth.

    The investments being made in DDDM, like many educational fads, are misdirected. We should be investing in teachers, professional development, and high quality curriculum materials which are openly accessible and licensed. Further investments in DDDM support the mindset Ravitch is criticizing and I also reject, which myopically focuses on reading and math scores.

    When people say “let’s invest in DDRM” and data warehouses, I think we should counter with the suggestion, “Let’s invest in digital portfolios which permit learners to demonstrate their knowledge, skills, and understanding with diverse types of media and archive those artifacts electronically online.” Being “data informed” means having multiple data points for assessment, and many of those will be / should be messier / less standardized than many of the “state tests” in use today.

  5. rob a Avatar

    we still need some tracking of data if for no other reason than to make sure all students have a basic proficiency in reading and math. Sadly, we have too many educators who ignore data such as what instructional strategies make the biggest difference on student learning. The data needs to look at inputs and outputs.

  6. Worldgate Avatar

    Agreed. Information should drive the decision-making process. Data warehouses can provide a global perspective, but educational administrators need to “look at inputs and outputs” as rob a puts it.

  7. Adam Timothy Avatar

    Thanks Wes – Ravitch is a great writer on this topic.

    Part of the difficulty is that we have a cultural sense of numbers as being infallible truths as opposed to a tool for holding an informed discussion and dialog without relying completely on gut instincts or anecdotal evidence.

    Data can be a wonderful platform for giving all sides a common ground to discuss the issues surrounding organizational development and policies. It does not make decisions though.

    When we say that data drives decisions, what we really need are indeed open minded discussions surrounding what the data is telling us based upon current inputs and what we should be doing differently to achieve desired outputs. It’s an iterative, never-done, process and no one gets all the metrics right let alone the decisions right the first time round. That’s not to say we can’t our shouldn’t try to measure things like student interest in a subject following instruction, or the creativity of students to adapt curriculum to their own abilities, passions and line of Just employing student-centric metrics in the equation does a lot to change the direction of the discussion. And don’t worry about NCLB and industry talk of DDDM, it’s just a matter of catching the next organizational wave of how decisions will be made in the future.

  8. Goofy Avatar

    What do you think of Ravitch’s opposition to the Partnership for 21st Century Skills (P21)?

  9. Wesley Fryer Avatar

    I haven’t read or heard about Ravitch’s opposition to P21, but it doesn’t surprise me based on what I heard in Florida this spring about the partnership. Apparently P21 is advancing the same agenda of privatization, deregulation, charter schools, high stakes testing, and linking teacher performance/evaluation to student test scores as NCLB and the Gates, Walton and Broad foundations.

    Do you have any links to Ravitch writing or speaking about this? I’d be very interested to hear her views directly on this.

  10. Jennifer Avatar

    Regardless of if we call it data-driven or data-informed, I think the greater challenge is clarifying what we mean by “data”. Data are not only numbers. Once we get beyond that reality, it’s easy to accept that teachers have been using data since time immemorial. To me, the energy in our push-back should be around advocating for multiple measures. I.e. “It seems like we’re spending a lot of time looking at qualitative data from one measure. Are there any other sources of data, such as student work samples or teachers’ running records, that we could use to inform our decisions?”

    I think we’re on the same page – I disagree if only because to me, there’s no real difference between “data-driven” and “data-informed”. It’s semantics and seems like making sure your bed is made when your house is burning down. A question I often cycle back to in discussions about student learning data is “If you’re not using data [provided its from multiple, quality measures] to make your decisions, what are you using?”.