11
The first ten posts in this series developed a visual
education statistics (VES) engine that relates six statistics on one Excel spreadsheet.
This post explores their relationships by switching right and wrong marks (1
and 0) in matched pairs and in unmatched single switches at increasing distances from the diagonal equator.
A Guttman table is an extreme distribution with each student
receiving a different score. Each item also has a different difficulty. Item
discrimination is set at the maximum. There is only one possible distribution
for this 21 student by 20 item test (Table 17). (The Excel .xlsm or .xls
version is available from Table17@nine-patch.com.)
The squared student score deviations are at zero at the test
score mean and at a maximum (100) at the extremes. The opposite is the case for
item sums of squares (SS) with a maximum of 5.24 at the mean of 10.5 and a
minimum of 0.95 at the extremes. This makes sense as there is greater variation
between student score extremes and less within item difficulty extremes (Table
17).
The standard deviation (SD) of student scores decreased (6.205
to 6.050) as matched pair switching progressed from the mean to the extreme in
a linear manner (Chart 28). This makes sense as the student score deviations normally
increase at the extremes. Switching marks reduced these extremes. 
Test reliability also fell as matched pair switching
progressed from the mean to the extreme in a linear manner (Chart 29). This
makes sense as the student score N MEAN SS decreased as the switching progressed
from the mean to the extreme (36.381 to 34.857 or 1.524) and as the item N MEAN
SS only decreased (-3.492 to -3.574 or 0.082). 
The standard error of measurement (SEM) increased linearly (1.354
to 1.423) as the switching progressed from the mean to the extreme (Chart 30).
This too makes sense as a decrease in test reliability is related to an
increase in the SEM.
Item discrimination (KR20 and Pearson r) decreased in a
non-linear manner (Chart 31) as the switching progressed from the mean to the
extreme (from 0.676 to 0.637). This also makes sense as the greater the change
from a perfect Guttman table, the lower the item discrimination. Switched marks
that are the farthest from the diagonal equator are the most unexpected marks.
A second scan of the Guttman table with an unbalanced single
switch of right and wrong produced the same relationships as the balanced
switch scan. The spreadsheet (Table 16) needed to be set to three decimal
spaces to capture the detail with a minimum of rounding errors (Table 17).
The VES engine is showing three linear relationships (SD,
test reliability, and SEM) and one nonlinear relationship (item discrimination).
Just one switch of 1 to 0 or 0 to 1 can be detected in all four statistics. I
find it interesting that such detail can be captured from a 21 x 20 table.
- - - - - - - - - - - - - - - - - - - -
-
Free software to help you and your students
experience and understand how to break out of traditional-multiple choice (TMC)
and into Knowledge and Judgment Scoring (KJS) (tricycle to bicycle):
No comments:
Post a Comment