An Analysis of Item Response Theory

Abstract

This paper examines a field of psychological and educational measurement testing theory called Item Response Theory. The paper delves into the basics of the theory, its theoretical and statistical background, and discusses the usefulness. The primary method of parameter estimation, Birnbaum’s Paradigm using Newton-Raphson method and maximum likelihood estimation, is discussed with a brief overview of the mathematics involved. The bulk of the paper focuses on a data set of Medical School Admission Test in Biology (MSATB) data administered to hopeful medical students in the Czech Republic. This data of 1,407 individuals and a subset of 20 questions selected from the overall test make up the set analyzed here. The main tools of Item Response Theory are used upon the data to test different model fits and produce graphics and charts to visualize the results. After the best-fitting model is selected for the data set, it is treated as a item bank to create a five-question subtest designed for pass/fail results; the analysis on this subtest shows it working fairly well in discriminating between ability levels. Finally, statistical machine learning methods are utilized to classify students based on ability level and to identify possible clusterings of students present in the data. Success was found both in classifying students based on performance and in identifying clusters in the data.