Language
Country/Area
No result found
The transformation of psychological measurement: the theory and practice behind CCT, what you definitely want to know!
With the development of science and technology, traditional psychological measurement methods are being replaced by new technologies. Computerized classification tests (CCT), as a new evaluation system, are gaining increasing attention. It not only simplifies the testing process, but also improves the accuracy and efficiency of testing. So, how does CCT work? What theories and practices are hidden behind it?
A computerized categorized test is a computer-based assessment system designed to categorize candidates.
How it works
CCT operates in a similar way to computerized adaptive testing (CAT), where questions are presented to candidates one by one, and after each question is answered, the computer immediately scores it and assesses whether the candidate has been classified. If so, the test is terminated and the candidate is classified; if not, the next question is offered. This process repeats until the candidate is sorted or other end criteria are met (such as all questions have been answered or the test length limit has been reached).
Psychometric Model
There are two main types of psychometric models for CCT: classical test theory (CTT) and item response theory (IRT). The former classifies candidates in a specific sample and identifies the difficulty and discrimination of each question based on different groups of candidates, but this places high demands on candidate selection. In contrast, IRT assumes that ability is continuous and the classification criteria may be vague but more precise. There are different considerations in choosing between these two approaches, with CTT offering conceptual simplicity and IRT offering greater specificity when resources are sufficient.
Although CTT is relatively simple, it can be more efficient in calibrating test parameters for small test plans.
Starting point
CCT must be set to a specific starting point in order to run a specific algorithm. If you use the sequential probability ratio test as the stopping criterion, the default starting ratio is 1.0; if you use the confidence interval method, you need to specify the starting point. Typically, such a starting point is 0.0, indicating the center of the distribution; however, it is also possible to set the starting point based on the candidate's historical data.
Topic Selection
In CCT, the selection of questions is flexible. Compared with the traditional method of using fixed question sets for all candidates, the questions can be continuously adjusted according to the performance of the candidates. Question selection methods can be mainly divided into two categories: cut-off score-based selection and estimation-based selection. The former maximizes the information provided around the cut-off score, while the latter makes choices based on current estimates of the examinee's ability, and the efficiency of these two choices will vary depending on the stopping criteria used.
Depending on the termination criteria used, timely question selection will directly affect the success of the test.
Termination criteria
There are three termination criteria that are often used in CCT: Bayesian decision theory, confidence interval method, and sequence probability ratio test. These methods each have their own advantages and disadvantages, providing varying degrees of flexibility and accuracy, but may also introduce some unnecessary subjectivity. Under the confidence interval method, the examinee's current estimate of ability has a direct impact on the classification outcome, whereas the sequential probability ratio test performs the classification in the form of a hypothesis test.
Conclusion
As the times evolve, CCT not only establishes an efficient examination standard, but also has a profound impact on the future of the field of psychometrics. As its practical applications continue to expand, how will it affect our testing methods and people's understanding of ability assessment in the future? This is worth pondering for each of us.
