NEWS The Swedish Scholastic Aptitude Test (Högskoleprovet) works but should be examined more closely, although the test as a whole works as intended. That is shown in Jonathan Wedman’s new dissertation at Umeå university.
“Högskoleprovet can have very significant consequences for the participants' entire future as it can determine whether one gets into an attractive education program. Therefore, it is important to ensure that the test maintains high quality,” says Jonathan Wedman.
In his doctoral dissertation, he has examined the validity of the Högskoleprovet, that is, how well the test actually measures what it is intended to measure: scholastic aptitude. The Högskoleprovet underwent significant changes in the fall of 2011 by increasing the number of tasks and subtests and by dividing the test into two separate parts, a quantitative part and a verbal part. This change also necessitated new research to evaluate the validity of the new test, or more precisely: to evaluate the validity of the interpretations and uses of the test scores.
In four studies, Jonathan Wedman has developed a proposal for a theoretical model for Högskoleprovet and conducted statistical analyses of test data from 100,000, 140,000, and 250,000 test participants. The results show that although the theoretical model is robust in its current form, further validity studies would benefit from a more clearly defined content for the concept of scholastic aptitude.
The results also show that the scores on the quantitative and verbal parts provide an accurate picture of the participants' strengths and weaknesses, but the subtest scores do not do so to the same extent. Additionally, it appears that female participants are favored on about one-fifth of the vocabulary tasks, while male participants are instead favored on an equal portion of the tasks in English reading comprehension.
“If the gender patterns persist, the subtests in vocabulary and English reading comprehension should be studied more closely to see if any measures are needed so that the subtests, through their design, do not disadvantage any gender,” says Jonathan Wedman.
The results of the Högskoleprovet are valid for eight years, which means it is very important that test results from different occasions are comparable so that competition for education programs is fair. In his dissertation, Jonathan Wedman has also examined whether the statistical method used to make test scores from different test occasions comparable is appropriate and whether it is the best possible method for the purpose. The result suggests that the current method is appropriate, but the result was somewhat unclear as to whether it is the best method or not.
Download the dissertation: Theory and validity evidence for a large-scale test for selection to higher education
Editor: Ola Nilsson
Photo: Izabella Balcerzak