How are the PSSA tests scored?

This is an important question for all of us. After schools have administered the PSSA test and the tests are returned to DRC (The company that is contracted to handle the PSSA test for PA.) for scoring.

After many checks and cross checks to assure that the proper number of boxes received equals the number sent to each school and the proper number of booklets are in each box, the tests are cut, divided and scanned for scoring. (DRC Tech Report 5, 8, 11)

Multiple choice items are scored using the computer (Remember when you were young and took a standardized test and your teacher insisted you use a number 2 pencil? Remember the teacher made you practice filling in the bubble neatly? The computer needs these bubbles filled in nicely in order to read the selected answer).

Open ended scoring is more complicated. Hundreds of expert readers are hired. The pool consists of educators, writers, editors, and other professionals who read and score the students open ended responses. Training and education on the 3 or 4 point scoring system (recall the Math open ended questions are worth 0-4 points and the Reading open ended questions are worth 0-3 points) and examples of writing at each level 0-4 is given to all of the readers. (Examples of each level of writing 0-4 are decided upon by Rangefinding committees consisting of Pennsylvania educators committees, PDE staff, and DRC employees.) (DRC Tech Report 5, 8, 11)

All open ended questions are scored at least once with 10% randomly picked to be scored twice. Reports and other quality controls are used to ensure scoring accuracy. However, even with all the checks and balances, there are still discrepancies in scoring when two different scorers have actually read the material. For example, in the case of the 2006 PSSA Reading open ended questions for grades 5, 8, and 11, same scores on the same question could be duplicated only 71 to 83 percent of the time. In cases where the score given by the second reader, was different from the score given by the first reader "adjacent" scoring often occurred. (Adjacent scores are scores “next” to each other: i.e. receiving 2 points from one scorer and 3 points from another scorer for a given response.) (DRC Tech Report 5,8,11 pg 45) An amazingly low agreement rate considering all the student and school district implications of the PSSA test score! It is also a strong indicator that subjectivity and bias do exist despite the fact that “readers were required to set aside their own biases about student performance and accept the scoring standards.” (DRC Tech Report 5,8,11 pg 42)