Time: 10 October 2018, 1 - 2 pm Place: B705

Abstract

The importance of large scale achievement tests, like national tests in school, eligibility tests for university, or international assessments for evaluation of students, is increasing. Pretesting of questions for the above mentioned tests is done to determine characteristic properties of the questions. Usually, the pretest is part of a test that should be small and have a negligible burden on examinees. For the Swedish driving license test, for example, the pretest part consists of only five out of seventy questions. If a large number of questions have to be pretested, they can be randomly assigned to the population of examinees. However, in computerized tests we can improve pretesting considerably if we, instead, choose examinees which are specifically suitable for each question based on their estimated ability. Statistically, we use item-response-models for the probability to answer a question correctly. We want to improve precision of parameter-estimators in the models. In this seminar, we determine how to allocate examinees to pretest questions when we have only the possibility to assign one out of several questions to each examinee. Optimal design theory helps us to achieve this task. We discuss a newly developed algorithm which allows us efficient computation of the allocation rule for pretesting.