Overall, 49 literacy items, 49 numeracy items and 14 problem solving items were administered in the assessment.
This concerned several numeracy items (Module 1: 28.57%, Module 2: 24.49%), a few literacy items (Module 1: 4.17%, Module 2: 2.04%) and no problem solving items.
In contrast to the domain-specific items in math and science, problem solving items did not require the use of prior knowledge in math and science (OECD [2004]).
Thus, random equivalent groups completed the literacy items and numeracy items in the first or the second part of the cognitive assessment, respectively, while there was only one order for the two sets of problem solving items.
Moreover, one may argue that math and problem solving items allow students to employ different solution strategies, whereas science items offer fewer degrees of freedom for test takers (Nentwig et al. [2009]).
On the one hand, this finding could be due to the design of the PISA tests (Adams [2005]), since math and problem solving items are designed in such a way that modelling real-life problems is required, whereas science items are mostly domain-specific and linked to science knowledge (Nentwig et al. [2009]; OECD [2004]).
The game is an investigation adventure, sophisticated for its time, in which the player must resolve a murder mystery by searching for clues, exploring different areas, interacting with characters, and solving item-based puzzles.
The ARQ-Rev1 problem solving scale was not supported by the factor analysis, with problem solving items loading on various factors in the solution.
The Rasch model's basic equation (see Eq. 1) describes the difference between the ability of a specific person n, B n, and the difficulty of a specific item i, D i, via a logarithmic function that depends upon the probability P ni of person n to correctly solve item i: {B}_n-{D}_i=ln left({P}_{ni}/left 1-{P}_{ni}right)right) (1).
