Wednesday, January 20, 2010

Classroom and Standardized Test Grades


Does a grade, or cut point, tell us what happened or just the appearance, that a politician or an administrator wants to give, of what happened?

Even a simple question, “Why did I get the same grade on my math test as another student got on a government test when our test scores differed by more than ten percentage points?” has no simple answer.

A Universal Cut Point Raw Score Grade Equalizer helps put things into perspective:



Most teachers, who just count right marks, use a 10-point range scale, as it is easy to remember the cut points of 90, 80, 70, and 60%. Every student can earn any letter grade (all can be A’s if all have mastered the assignment).

Other teachers use the average test score to select a scale for assigning grades. Average tests scores ranging from 70% up to 92.5% produce a range of grades for a raw score of 88%. It is an A on a 12-point range scale, a B on a 6-point scale, a C on a 4-point scale, and a D on a 3-point scale.

There are many ways to assign grades. In general, a test score below 80% means the student is not keeping up with the course and will not be prepared for the next course, whatever grade is assigned.

Right mark scoring (RMS) grades are easily manipulated by the selection of questions, question difficulty, and cut points in the classroom and on standardized tests. Lowering the cut point to 40% (a range scale of 15 points and a quality score of 40%) insures that a portion of students will pass by luck alone. There is no way to know what the student actually trusted as a basis for further learning and instruction.

Knowledge and Judgment Scoring (KJS) and Confidence Based Learning (CBL) value judgment (quality) independently from knowledge. The student is in charge of reporting what he can trust and what he has yet to master. KJS and CBL reward students for taking the responsibility to learn beyond the concrete level. They are rewarded for learning, anywhere and anytime, not just in class.  They ask questions, get help, and put in the time needed to master the assignment. It feels good to have mastered a clearly stated and understandable assignment.

KJS and CBL grades are not easily manipulated since there is a score for what is known and the degree to which it can be trusted. The grades reflect what self-motivated achievers are doing rather than how lucky passive pupils were on test day.

In my opinion, one of the main reasons schools that show a marked increase in RMS standardized test scores one year and no further increase in the following years is that passive pupils can only be pushed so far in traditional classrooms. Student development to produce self-motivated achievers, functioning at all levels of thinking, is needed to go further. These are the graduates that are successful in what they do next in school and beyond.

There are many ways for schools to promote mastery, and not just the appearance of mastery. KJS is a bridge to mastery. CBL guarantees mastery.

Tuesday, October 20, 2009

Multiple-Choice Bubbling

Students first meet multiple-choice by receiving a #2 pencil and a sheet printed with a variety of boxes, circles or ovals to be darkened (unless they can mark directly on the test). The No Child Left Behind (NCLB) game requires one properly darkened mark on one answer option for each question in the allotted time, at lower levels of thinking (LLOT); a characteristic of NCLB testing. Secondly, for marginal students, the mark should hopefully be a right answer.

Large bubble sizes were used to improve the accuracy of older answer sheet readers. [teachervision] More time was needed to fully darken the outlined space [YouTube]. “… because of such onerous requirements, test takers may become preoccupied with filling the bubbles correctly and less able to focus on the substantive tasks at hand.” [patentstorm]

Many solutions to this problem now exist: [Vertical ovals, Scantron Form 229627] [Small circles and with intermediate student spacing, Scantron Form 220610] (Both sets of circles have the same space to darken in each circle.)

The most novel solution is a large letter replacing the large bubble with a smaller letter inside. The reduced bubble now only covers the target area the optical mark reader actually sees. [Normal, Apperson Form 205090; Large, Apperson Form 23020]  The original relationships between bubble and letter have been inverted for young students, older adults, and employees.

The loss of meaningful information available from these tests, by only counting right marks, has also been solved by an inversion: inverting who is responsible for determining if a mark is right or wrong from teacher or examiner to student or examinee. [Nine-Patch Multiple-Choice and Knowledge Factor] A multiple-choice test now takes on the same characteristics of other assessments, involving judgment, by requiring students to function at all levels of thinking, plus, rapid scoring and detailed student or examinee counseling reports.