A computer adaptive test (CAT) functions like an experienced content-area teacher. A teacher may ask students a few questions to learn about their background knowledge and their instructional needs. They use their content area expertise to select a question of appropriate difficulty. The teacher then evaluates the student’s answer and asks another question to refine what is known about the student. This process continues until the teacher concludes understand the student’s current knowledge and needs. CATs use an algorithm to individualize, or adapt, the assessment for each student as illustrated in the following graph. A CAT generates the same information as a conventional assessment in just one-third of the time and, thus, is very efficient.
How Does a Computer Adaptive Test Work?
The CAT software uses research-based estimates of item difficulties to select the most optimal items based on the student’s response profile during the assessment. In graph above, there were 33 assessment items available (blue dots) that span the ability range, but the CAT needed only 10 items to estimate the student’s ability. Item difficulty and student ability are depicted on the bottom axis. In the case of the FastBridge CAT aReading, items and abilities are on a scale of 350 (basic) to 750 (advanced skills and abilities).
The order that items are selected is defined on the left axis. The assessment begins with an item of middle difficulty with a score of 500. The estimate of student ability increases with correct responses and, so, the estimated ability and item difficulty increases. The probability of a correct response is estimated for all of the other items.
As shown in the upper right of the above graph, the estimate of student ability is used to predict likely responses to all items and to generate instructional recommendations. For example, aReading provides specific estimates of each student’s reading level, profile (skills that are mastered, developing, or pending), and instructional targets.
Why use aReading or aMath?
FatBridge publishes two CATs: aReading and aMath. These take an average of 15 to 30 minutes to administer, which is half the time of other CATs with equivalent or improved reliability and validity. Assessment items are sampled from more than 3,000 active standards-aligned items, which are continuously refreshed. Research and development of aReading and aMath were funded by the US Department of Education.
Efficiency, Reliability and Validity
A CAT adapts to sample student performance in a broad domain. Scores are reliable (stable) and valid (measure what is intended), and the assessment is highly efficient (takes less time) for screening. There is no more efficient way to automate the assessment of performance with similar levels of reliability and validity.