#4

The first thing I noticed when inspecting the top of the test
scoring math model (Table 25) was that the variation within the

**central cell field**has a different reference point (external to the data) than the variation between scores in the**marginal cell column**(internal to the data). Also the variation within the central cell field (the variance) is harvested in two ways: within rows (scores) and within columns (items).
The mean sum of squared deviations (MSS) or variance

**within**a column or a row has a fixed range (Chart 64 and Chart 65). The maximum occurs when the marks are 1/2 right and 1/2 wrong (1/2 x 1/2 = 1/4 or 25%). [Variance also equals p * q or (Right * Wrong)/(Right + Wrong)] The contribution each mark makes to the variance is distributed along this gentle curve. The variable**data are fit to a rigid model**.
I obtained the overall shape of these two variances by
folding Chart 64 and Chart 65 into Photo 64-65. The result is a dome or a depression above or below the upper
floor of the model.

The peak of the dome (maximum variance) is reached when a
student functioning at 50% marks an item with 50% difficulty.

**Standardized test makers**try to maximize this feature of the model. The larger the mismatch between item difficulty and student ability, the lower down the position of the variance on the dome.**CAT**attempts to adjust item difficulty to match student preparedness.
Chart 66 is a direct overhead view of the dome. Elevation
lines have been added at 5% intervals from zero to 25%. I then

**fitted the data**from Nursing124**to the roof of the model**. The data only spread over one quadrant of the model. The data could completely cover the dome in an ideal situation in which every combination of score and difficulty occurred.
The total test variance within items is then the sum of the
variance within all items (0.04 to 0.25 = 2.96). The total test variance within
scores is the sum of the variance of all scores (0.05 to 0.24 = 3.33). See
Table 8.

The math

**model adjusts to fit the data**in the marginal cell student score column (variance between scores). The reference point is not a static feature of the model but the average test score (16.77 or 80%). The plot of the variance between scores can be attached to the right side of the math model (Chart 67).
The variance

**within**columns and rows spreads across the static frame of the model. The model then adjusts to fit the variance**between**scores (rows) to match the spread of the active**within**rows.
I can see another interpretation of the model variance if
the dome is inverted as a depression. As a flight instrument on a blimp: pitch,
roll, and yaw (within item, 2.96; within score, 3.31; and between scores, 4.10)
the blimp would have the nose up, rolled to the side, and with the rudder hard
over.

- - - -
- - - - - - - - - - - - - - - - -

Free
software to help you and your students experience and understand how to break
out of traditional-multiple choice (TMC) and into Knowledge and Judgment
Scoring (KJS) (tricycle to bicycle):

## No comments:

## Post a Comment