Modeling of Adaptive Knowledge Testing: Efficiency Threshold, Task Complexity and Completion Time

Abstract

A comprehensive informational and analytical analysis has been conducted to evaluate the feasibility of implementing adaptive computer-based knowledge testing for specific academic disciplines in educational institutions. The shortcomings of the traditional approach, which imposes fixed time constraints for test completion without considering the individual characteristics of learners and potentially causing negative reactions among test participants, have been identified. Alternatively, an integral assessment approach is proposed, accounting for both task complexity and task completion time. An adaptive algorithm has been developed based on the efficiency threshold q, which determines the adjustment of the difficulty level for subsequent tasks depending on the integral evaluation result of the previous task. Simulation modeling was carried out using Python to verify the effectiveness of the proposed approach. A test dataset comprising tasks of three complexity levels was created, with completion times modeled according to the normal distribution. The analysis revealed that significant differences in task difficulty levels necessitate establishing separate efficiency thresholds for each category of questions, while minor differences allow for a single threshold for all test tasks. Parameter tuning for the integral assessment was performed within the test dataset, and the effectiveness of the proposed method was examined. It was noted that the obtained coefficients of the integral evaluation could serve as baseline values during the initial implementation phase of the system, with further optimization based on model training results during pilot testing. The described methodology demonstrates flexibility and ease of implementation, enabling parameter customization and effective adaptation to both the individual characteristics of learners and the specific requirements of individual disciplines. Furthermore, recording task completion times can serve as an additional tool for assessing the quality of test items.