Wednesday, March 20, 2013

Visual Education Statistics - Average

Statistic Two: The average or mean collects counts into one descriptive statistic. The percent test score (79.9%) is also the average percent student score and the average percent item difficulty (Table 3). It is not reversible. The average student score count is 16.77 and the average difficulty count is 17.57. You cannot redisplay the distribution of individual marks knowing only the average or mean score.

The prior post reviewed the simple row and column table used to record student marks on multiple-choice tests. It also pointed out that the test score is derived from marks that can have a wide range of values or meanings. The two that are easily extracted are knowledge (the number of right marks) and judgment (the percent of marks that are right using Knowledge and Judgment Scoring).   

Table 3 shows that the test score is also the average of the three sub-tests, defined by item performance, found in most classroom tests: Mastery, Unfinished, and Discriminating (MUD).

Chart 1 shows the average scores of 93%, 73%, and 71% for the three sub-tests and 80% for the total test score.

The MUD distribution makes evident that a mark of 1 or 0 does not have the same value or meaning for every item in the test. Both student score marks and item difficulty marks (item scores) can report different things, all of which are ignored when just counting 1’s and 0’s with TMC.

A 1 on a Mastery item is a simple check list of what the class is expected to know or do. A 0 is not expected. [Mastery items are used to adjust the average test score.]

A 1 on a Discriminating item places that student in a group that knows or can do something that the group receiving zeros does not know or cannot do. There is a group of students in this class that needs to catch up with the rest of the class. Grouping helps identify instructional problems. [Discriminating items produce the score distribution needed to assign grades.]

Unfinished items indicate a failure in instruction, learning and/or testing. Having almost all 1’s in both Mastery and Discriminating items identifies a student functioning at higher levels of thinking.

[A fourth group, Misconceptions, identifies items that mystify students. They believe they can use the item to report something they know or can do, but are in error. Misconceptions are only identified when students elect to use KJS. If a minority of the class, less than average, marks an answer and most of the answers are wrong, it is guessing. When a majority of the class, more than average, marks an answer and most of the answers are wrong, it is a misconception.]

Students are rightly interested in their score location in the score distribution, above or below average (I am safe or I need to study more). Classroom teachers are interested in an expected average test score from which they can assign grades.

The process of data reduction from mark distribution to sub-test to total test is not reversible. From this point on we are dealing with averages (means) of groups, not with individual marks. Individual test scores and individual item difficulties are traditionally treated as single entities, but are really the average number of right marks for each case. All of these statistics work the same for TMC, KJS, and PCM scoring.

Neither Table 3 nor Chart 1 (the average or mean) does a good job of capturing the mark distribution in a number. That requires the next statistic: the standard deviation of the mean; usually shortened to standard deviation.

- - - - - - - - - - - - - - - - - - - - -

Help for you and your students to experience and to understand the change from TMC to KJS (tricycle to bicycle):

No comments:

Post a Comment