New statistical methods to ascertain high quality over time in standardized achievement tests

Project description

The overall aim is to develop new efficient statistical methods within the area of psychometrics, in order to ascertain and guarantee high quality over time in large standardized achievement tests.

1. To develop a new method to model items, especially the items characteristics (e.g. difficulty and discrimination) more efficiently which take care of the limits and challenges of the current classic test theory and item response theory.

2. To extend the observed-score equating framework, i.e. methods used in order to adjust test scores from different test versions so they can be used interchangeably.

3. To propose a statistical assessment of the equating transformation instead of using equating specific criteria. There is still no best way to assess the equating transformation. This is extremely important because which equating method is chosen (based on different selection criteria) has direct impact on the test scores the test takers achieve and thus can have impact if the test is used for admissions to universities.

4. To develop and adjust quality control (QC) methods so they are feasible to use with standardized achievement tests which are typically administrated several times and where the test takers results are valid many years. The focus is on standardized tests which are administered several times and where traditional methods, compiled for relatively few administrations or small amounts of data, are not suitable.

By solving the outlined issues we want to make a strong theoretical and practical contribution to the psychometric field. The research is of great relevance in order to obtain unbiased and fair tests for all test takers. The results are extremely valuable for test constructors, policymakers, teachers, researchers and every test taker who use standardized achievement tests.

The research project is planned to be conducted over five years in three steps. First, analytical statistical models and methods will developed with strong connections to psychometric theory. Second, the obtained analytical models and methods will be evaluated using simulated data and adjusted accordingly. The third step aims at using real standardized test data and obtains a conceptual framework within which test quality can be assured.