This resulted in a 5-item scoring rubric, which we complemented with an item about searching questions to represent the construct of reflection fully [ 10, 41].
Quantitative and qualitative data collected during four design cycles resulted in the Mobile App Selection for Science (MASS) Rubric, comprising six items on a four-point response scale.
Discipline-specific content requiring modification consisted of 3 clinical scenarios, 4 question stems, and the scoring rubric for 5 items.
The authors created a 35-item rubric with ≥0.90 interrater reliability, based on three raters who met to clarify the rubric and gain consensus after beginning to grade and then grading independently.
We applied the "mutation" part of our function rubric (Table 2, items 4, 4a, and 4b) to code students' short answers for the "origin of variation" concept on a 0 3 scale (0 = no mention of mutation; 1 = mutation is mentioned; 2 = mutation is mentioned and connected to genetic structures like DNA, nucleotide sequence, etc.; 3 = mutation is articulated as the causal event leading to new alleles).
The ACORNS is designed with a sixteen item scoring rubric that standardizes student responses that differ across contextual variables for evolution (e.g., gain vs. loss of traits, plants vs. animals, within vs. between species differences).
Table 2 Scoring rubric of the constructive-response item on differential height Score Level Description Students' responses 0 No answer Off-task [blank] Students write some text that either makes no meaning or no attempt to respond to the question stem.
For each assessment item, a conceptual rubric was developed to categorize and score the presence or absence of both hypothesized and emergent themes in the data.
There are up to five levels in the coding rubric for the constructed-response items (see Table 2) (Shen et al. 2014).
The student must discuss two aspects of engineering practice that favour re-designing the dam, even if this will incur greater cost (for the full item and scoring rubrics, See Tremblay et al. 2012, 261 263).
