A double coding of 20% of the corpus was conducted, randomly extracted from the videotapes (all the groups, in both activities).
The two sets of independently identified major themes were documented and discussed with coauthors (JD, JH, KM, AB), and an initial coding framework of themes was developed and then pilot tested, by independent double coding of 50 of the verbatim responses by RM and BN.
Agreement >85 % (kappa = 0.63) was found for pilot double coding on 1 % (n = 80) of titles and abstracts against the criteria, so the remaining were reviewed, and additional double coding of 5%% of all titles and abstracts (n = 400) produced over 98%% agreement (kappa = 0.80).
Following independent double coding of 100 responses, in 82 cases there was agreement on the theme/s a response was coded to, including 5 cases where one or another coder assigned the response to an additional theme as well.
Moreover, in CT examinations 7.8% of the intravenous contrast (IV) codes were deleted to eliminate double coding (of examinations both with and without IV).
The main strengths of our work includes the systematic approach, the use of a literature based data extraction method and the rigor of our analysis by employing double coding of CPG selection and data extraction.
Also, we did not undertake double coding of all transcripts.
Coding was a time-intensive procedure and double coding of all transcripts was not feasible with our available resources.
19 Finally, issues regarding interrater reliability arise during coding that require consistent coder training, double coding of a portion of recordings, and post hoc analyses of reliability.
Coding cross-checks were performed by double coding of one interview from each region and by secondary reviews of all interviews by another coauthor (JZ for the Trent Region, BB for Ile-de-France, MC for Lazio).
