Effects on the demand of reading ability when translating mathematical tasks between different languages

Project description

The purpose of this project is to deepen the understanding of the relation between the language in mathematical test tasks and the tasks’ demand of reading ability, both when it comes to linguistic properties of the tasks (e.g. wording and grammatical structure) and the natural language used to formulate the task (e.g. English or Swedish).

Research questions:
1. How does the choice of natural language that a mathematical task is written in affect the task’s demand of reading ability?
2. What are the reasons for large differences regarding demand of reading ability between different language versions of the same tasks?

In this project we will use and examine PISA mathematical tasks in English, Swedish, and German. Within PISA, the tasks are intended to be identical and measure the same mathematical ability across languages, making this data suitable for this project. Furthermore, PISA utilizes elaborate procedures for the translation and adaptation of tasks in different languages, which is seen as an advantage for the purpose of our project since these procedures will hopefully have excluded all potential direct translation mistakes, making it easier to focus on other causes of differences between language versions of tasks.

Even though communication and mathematics cannot be separated, it is possible to statistically focus on one of these aspects at a time, by seeing the existence of two overlapping latent variables corresponding to mathematical ability and reading ability. We have therefore developed a method using principal component analysis (PCA) on data sets with students’ results on both reading and mathematical tasks to examine the tasks’ demand of reading ability. In comparison with other methods, our method has been shown to have good validity and reliability. If a PCA is used on a data set with students’ results on reading tasks as well as on mathematical tasks, it is possible to identify mathematical tasks with “unnecessary” demands of reading ability. A comparison of the demand of reading ability for a specific task in different language versions can identify differences in the reading component of the solution process, thus focusing on a specific potential source of Differential item functioning (DIF) for tasks written in different languages.

The project will statistically examine the demand of reading ability for mathematical tasks written in different languages. This methodology will be complemented with more traditional DIF analyses, in order to relate the different types of conclusions that can be drawn from different methods. The project is significant both internationally and nationally since large-scale international comparative studies of mathematical ability (as well as science and reading ability) are common and have impact on educational policy in many countries.