The ex-post test took place three weeks after completion of each experimental session.
They lead to different decision processes and subjective evaluations, but the ex-post test showed that differences disappear over time.
For more complex problems, both measures indicate significantly lower correspondence between the original solution and the ex-post test.
Problem complexity has a significant effect in the ex-post test on the rank of the best alternative as well as on the total difference of rankings.
Our research questions relate the factors problem representation, problem complexity, and user characteristics to process characteristics, subjective evaluations, problem understanding, and consistency in the ex-post test.
To analyze the research questions formulated in Section 3, we performed several regression analyses of the relevant outcome dimensions (process, subjective evaluation, problem understanding, and the ex-post test) on experimental factors, user characteristics, and their interactions.
In the ex-post test, rankings of five selected class schedules elicited three weeks after the experiment were compared to the ranking of the same class schedules during the experiment.
Problem complexity had a strong effect on performance in the ex-post test, where subjects had to rank five (efficient) class schedules according to their preferences three weeks after completion of the experimental session.
First, although different problem representations induce differences in the decision making process, these differences do not seem to have long term effects on either problem understanding or performance in an ex-post test.
Heatmap users spent significantly more time than users of PCP on the decision task, nevertheless, they performed worse in the ex-post test, although this effect was statistically not significant.
Although in reality a decision would be made immediately after using the system, consistency between the original decision and the ex-post test can be considered as an additional indicator that the original evaluation has reflected the subject's preferences.
