Who Stole Effective Education?
by Jay C. Powell
We all walk in a constrained envelope of knowledge.
There are things we know we know.
There are things we think we know but actually,
we don’t know the truth about this topic.
There are things we know we don’t know
There are things we don’t know we don’t know.
Should the function of education be to help us to push bask the frontiers of this envelope of ignorance? If so, can we identify these zones of ignorance? They are the places where we make the most mistakes. By paying close attention to our mistakes, we can learn to expand our zones of understanding more effectively than we can in any other way. Such is the history of human discovery.1
Why then do we ignore the mistakes our students make on tests or in classroom interactions with us? Where tests are concerned, we assume that there is nothing meaningful there. Wrong answers are either meaningless or inaccessible. We assume meaninglessness by expecting them to be “blind guesses” and therefore random events. We assume them inaccessible because of the “linear dependency” problem that make including them into our linear assessment models mathematically impossible. Both these assumptions are false! Students give us answers based upon the ways they interpreted the questions, not randomly. Actually, learning is not linear. It is complexly non-linear with gains in some areas and losses in others and no change at all in still other places. Without this wrong answer information, learning dynamics are indeterminate.
Powell and Shklov (1992)2 presented an adaptation of the multinomial procedure that bypasses the linear dependency problem and brings out all these learning characteristics just claimed. We showed that, by considering every answer, between two and three times as much information are available from the same test as can be provided by the right answers alone.3 Furthermore, by giving the same test twice, the dynamics of the changes in status, and therefore the procedure reveals the learning processes because we can now look at performance on an answer-by-answer basis. This procedure makes such tests powerfully diagnostic, combining both summative and formative assessment in the same data set.
We have rendered this mistake invisible to us by removing the wrong answers from consideration in the scoring process before we begin to make sense of these data. We are culpable only because we have paid insufficient attention to relatively obscure research by relatively unknown individuals.
Who stole effective education? Everyone who continues to rely upon total-correct scores, or even partial credit systems that remain linear in their analytic system is stealing effective education from our schools every day. See my book; Powell, Jay C. (2010) Making peasants into Kings, Bloomington, IN: Author House. (ISBN: 978-1-4490-0634-1; Available with Membership.)