Although the test-retest kappa statistic concerning choice of plans was 0.61 (substantial), the confidence interval for kappa (0.52, 0.69) overlapped that of the items concerning the ease/difficulty of understanding their plan (95% CI: 0.41, 0.56) and how protected the beneficiaries feel from out-of-pocket costs (95% CI: 0.43, 0.57).
Three of the COMRADE items focus on patients' ease of understanding information, patients' satisfaction with the information given, and adequacy of information about issues important for the decision, respectively.
Where resources exist to do this, we recommend that researchers employ expert review of DIF items as part of the process of understanding and interpreting DIF effects.
Six items specifically address: satisfaction with current treatment (item 1), treatment convenience (item 4), flexibility of treatment (item 5), understanding of diabetes (item 6), recommending treatment (item 7) and continuing treatment (item 8).
We also designed an outcome assessment made up of items testing students' understanding of biology concepts and their ability to apply mathematical skills in biological contexts and administered it as a pre/postcourse test to students in the experimental section and other sections of the same course.
The CTM-3 comprises three of the CTM-15 items: (i) patients' understanding of their medications; (ii) the extent to which patient preferences are taken into account in deciding what the patient's health-care needs will be on discharge; and (iii) patients' understanding of their responsibilities in managing their health [ 28].
Statements from all thematic categories were represented in expert assessments of the most important items for understanding dementia.
Six (items 1, 4-8) of them (item 1: satisfaction with current treatment; item 4: treatment convenience; item 5: flexibility of treatment; item 6: understanding of diabetes; item 7: continuity of treatment, and item 8: recommending treatment to others with diabetes), are summed to get treatment satisfaction score with a possible range of 0 (very dissatisfied) to 36 (very satisfied).
It is possible to distinguish and compare these sensible items by means of understanding, but the data themselves are available only through experience i.e., the sensation of things and reflection upon thought and mental activities, feelings, and desires.
The participants were asked to discuss the meaning of each item, their understanding of the item content, and what they thought about when choosing a response.
Fig. 2 Assessment item for testing students' basic knowledge (remembering) Fig. 3 Assessment item for testing students' understanding of skills in using 3D software (understanding) Fig. 4 Assessment items for testing students' higher-order thinking (analysing) Fig. 5 Number and types of questions in each online quiz.
