Biased selection methods

The Prince George’s county Maryland school officials had a difficult decision to make. It seems that they managed to lose 500 tests that students took that are used as one of measures to determine who makes it into the county’s talented and gifted program. Unfortunately they decided on a biased method of dealing with the problem.

The tests that were lost were the Otis Lennon School Ability Test. They only lost the test for 500 students. They did not loose the test for all of the students. The administrators made the obvious decision to administer a retake of the test. However in the mean time they managed to figure out where they misplaced the first round of test.

They then decided that for those who took the test twice they would use the higher of the two grades to make the decision on who would make it into the talented and gifted program. And that created the biased selection method. The probability of making it into the program was greater for those who took the test twice. If they missed the cutoff the first time they had the opportunity to make it in the second round of testing.

Now an unbiased method would have been to toss the second round tests. But likely they would have a problem with parents who were already upset that their kids had to take the test a second time. Now the administrators did not have to deal with parents saying “but Johny passed the second time he took the test. And they were unlikely to have to answer to parents who would say “but Jane did not get a second chance at the test.”

How bad was the problem. There is no data at this point to know. I have not seen a press release saying that because of the error an extra x students made it into the program. Selecting students for a gifted and talented program is inexact in many ways. Plus there are those who would argue that the programs should not even exists.

It would be interesting to see the two sets of scores for the 500 students involved and how those scores affected the eventual decision on which students made it into the talented and gifted programs. It would give us one measure on the quality of the selection process.