Wednesday, October 16, 2013

Knowledge and Judgment Scoring - Operational to Instructional

This post (and the next three) introduce why we need a KJS Group. The software, Power Up Plus (PUP), that contains both Knowledge and Judgment Scoring (KJS) and traditional multiple-choice (TMC) is now free to registered KJS Group members.  Version 5.22, is free to teachers and administrators. Please see instructions below. 

This reflects a change in use of the software as an operational program for scoring individual classroom tests, to use as an instructional program to promote student and teacher development in preparation for the CCSS movement assessments. Students and teachers can readily see the difference between lower and higher levels of thinking when students are offered the opportunity to report, in a non-threatening environment, what they actually trust they know and can do, that serves as the basis for further learning and instruction. Practice riding the tricycle is poor preparation for a riding test on a bicycle.

Last week I finished a series of 22 posts on this Multiple-Choice Reborn blog. The series makes clear, that no amount of “statistical work” can extract from TMC marked answer sheets, some of the claims now being marketed about them. These tests can, at best, only do a good job of ranking students.

They so imperfectly and incompletely tell us what students know and can do that North Carolina is now spending six months figuring out how and where to place the cut scores on their new CCSS traditionally scored end-of-grade, multiple-choice math test results. 

[They must guess where to put the cut score on the results from uncommitted, low scoring, improperly prepared students, who were guessing at the right answers to questions the test maker guessed, would produce a satisfactory score distribution, with high statistical reliability and precision. The more nonsensical the student mark data are, the more subjective the process.]

Accurate, honest, and fair testing can be done with Knowledge and Judgment Scoring and the partial credit Rasch model analysis. These methods allow students to report what they actually know and can do that is meaningful, useful, and empowering. Student development (the judgment to appropriately use all levels of thinking) is as important as knowledge and skills for successful students and employees (Knowledge Factor). 

The NCLB decade has laid the foundation for real change by making schools designed for failure (that promote students beyond their abilities, rather than developing the necessary abilities for their success) so bad and so visible, that something had to be done. The CCSS movement has rekindled the old alternative (to TMC) testing and authentic testing methods; with the addition of CAT and elaborate assessment methods.

My concern now is that, after expending a large amount of time and money on promoting the CCSS movement ideals, a major part of the assessments will once again be reduced back again to traditional guess testing at the lowest levels of thinking. 

Both KJS and TMC scoring can use the same test questions. In fact both methods are used on the same test to accommodate students working at all levels of thinking and with all degrees of preparation (PUP).

IMHO, KJS is a practical method of achieving the CCSS movement goals. It prepares students for  standardized tests presented at all levels of thinking.  [I still cannot predict when KJS or the partial credit Rash model will be used on standardized tests as current standardized tests are not designed to assess what students know or can do. They are designed, using the fewest questions, to produce an acceptable spread of student scores.]

Rather than a rank of 60 on a test, a student may get a quality score of 90% on questions used to report what the student actually knows and can do, as well as, a rank of right marks on the test using KJS. We now know what a “just passing” student knows with the same accuracy as a student earning a 90% score on a traditional test. This can be valuable formative assessment information. 

Letting students tell us what they know or can do makes more sense than the guessing game now in use during preparation and assessment. And over 90% of my students preferred Knowledge and Judgment Scoring after just two experiences with it. Even students like an honest and fair test over gambling for a grade.

Past performance in my classroom is no guarantee of performance in your classroom unless you are a likeminded teacher, administrator, or test maker.

[The Educational Software Cooperative, Inc. (non-profit) closed this year (2013) after 20 years of operation during which I was the volunteer treasurer. It was founded to maximize the benefits of an individual computer: infinite patience, non-judgmental, and best of all, instant formative feedback. That level of instruction and record keeping has now been surpassed by the necessity for district wide record keeping systems operating online assessments keyed to CCSS learning objectives.]

Welcome to the KJS Group: Please register at Include something about yourself and your interest in student empowerment (your name, school, classroom environment, LinkedIn, Facebook, email, phone, and etc.).

Free anonymous download, Power Up Plus (PUP), version 5.22 containing both TMC and KJS:, 606 KB or, 1,099 KB.

- - - - - - - - - - - - - - - - - - - - - 

Free software to help you and your students experience and understand how to break out of traditional-multiple choice (TMC) and into Knowledge and Judgment Scoring (KJS) (tricycle to bicycle):

No comments:

Post a Comment