The results indicate that LEM performance is strongly dependent on multi-process related choices, and that performance indicators can be used for ex-post testing of the influence of these choices.
The ex-post test took place three weeks after completion of each experimental session.
They lead to different decision processes and subjective evaluations, but the ex-post test showed that differences disappear over time.
For more complex problems, both measures indicate significantly lower correspondence between the original solution and the ex-post test.
Our research questions relate the factors problem representation, problem complexity, and user characteristics to process characteristics, subjective evaluations, problem understanding, and consistency in the ex-post test.
Problem complexity has a significant effect in the ex-post test on the rank of the best alternative as well as on the total difference of rankings.
Heatmap users spent significantly more time than users of PCP on the decision task, nevertheless, they performed worse in the ex-post test, although this effect was statistically not significant.
Although in reality a decision would be made immediately after using the system, consistency between the original decision and the ex-post test can be considered as an additional indicator that the original evaluation has reflected the subject's preferences.
To analyze the research questions formulated in Section 3, we performed several regression analyses of the relevant outcome dimensions (process, subjective evaluation, problem understanding, and the ex-post test) on experimental factors, user characteristics, and their interactions.
