Two alternative forms of multiple-choice (AMC) to the
traditional multiple-choice (TMC) developed from independent sources. Geoff Masters from Melbourne,
Australia, is credited as the developer of the parcel credit Rasch model (PMC),
a form of Information Response Theory (IRT) analysis in 1982 (Bond and Fox). It allows students to report what they
know (2 points), what they do not know (1 point), and wrong answer (0 points).
It never became popular on classroom or standardized tests.
The second form of AMC was developed at NWMSU. It started as
net yield scoring (NYS) on both essay and multiple-choice. I needed a way to
reduce the amount of reading required in scoring “blue book” essays. A 20-point
essay started with 10 points. A point was added for acceptable, related,
information bits. A point was subtracted for unacceptable, incorrect, unrelated
information bits. An information bit was basically a short sentence with
correct grammar and spelling. It could also be a relationship expressed as a
diagram, sketch, or drawing.
This reduced the amount of reading by more than a 1/3 and
improved student performance. Snow, filler, and fluff had no value but
distracted a student from doing good work. Students needed to exercise good
judgment in selecting what they wrote. This was no longer the case of their
writing, and the teacher searching, for something that could earn them
sufficient credit to pass the course; a lower level of thinking operation that
is very common in high schools and colleges. NYS required students to use good
judgment as well as be knowledgeable and be skilled.
This same idea was applied to computer scored
multiple-choice tests with interesting results. When both TMC and NYS were
offered on the same test, most students selected TMC on their first test. This
is what they were familiar with. Over 90% of students elected NYS on their
third test. Students also agreed that knowledge and judgment should have equal
value.
By 1981 NYS was renamed knowledge and judgment scoring (KJS)
to reflect what was being assessed: good judgment and a right answer (2
points), good judgment to report what has yet to be learned with no mark (1
point), and poor judgment, a wrong mark (0 points).
KJS requires and rewards students for using higher levels of
thinking. The quality score is independent from the right count score. A
struggling student with a test score of 60% may have also earned a quality
score of 90%.
With TMC there is no way of knowing what a student with a
score of 60% actually knows (when a right mark is a right answer or just luck
on test day). With KJS we can know what this student knows with the same degree
of accuracy as a student earning a 90% score on a TMC test.
More importantly, this reinforces the student’s sense of
self-judgment and encourages effort to do better. It is the equivalent to the
note a teacher marks on a special paragraph in an essay, “Good work!”
KJS provides the information needed to tell student and
teacher what has been learned and what has yet to be learned in an easy to use
report. Often a trail of bi-weekly test scores would follow a backward J.
Reducing guessing by itself did not increase the test score but moved the score
to a higher quality. Low quality students needed to change study habits. Low
scoring high quality students needed to study more.
Learning by questioning and establishing relationships
provided students the basis for answering question correctly that they had
never seen before. They then stumbled onto what I meant by, “Make things
meaningful (full of relationships) if your learning is to be really useful,
empowering and easy to remember”. They did not have to review everything for each cumulative test.
The most interesting finding was that when students mastered
meaning-making, they found themselves doing better in all of their courses.
This is what inspired me to continue to promote Knowledge and Judgment Scoring.
Students learn best when they are in charge. The quality score was the “feel good” score for struggling students until their improving development produced the
high scores earned by successful self-correcting students.
Welcome to the KJS Group: Please register
at mailto:KJSgroup@nine-patch.com.
Include something about yourself and your interest in student empowerment (your
name, school, classroom environment, LinkedIn, Facebook, email, phone, and
etc.).
Free anonymous download, Power Up
Plus (PUP), version 5.22 containing both TMC and KJS: PUP522xlsm.zip, 606 KB or PUP522xls.zip, 1,099 KB.
- - - - - - - - - - - - - - - - - - - - -
Other free software to help you and your students experience and
understand how to break out of traditional-multiple choice (TMC) and into Knowledge
and Judgment Scoring (KJS) (tricycle
to bicycle):
No comments:
Post a Comment