I am a radiologist designing a study where 230 CT scans of cancer patients will be evaluated by 5 radiologists. There will be two sets of evaluations: one where radiologist is aided by an AI Computer-aided detection (CAD) tool, and one where they are unaided. Each radiologist will score lesions on a scale from 1-5, and time spent per case-review will be recorded. I will allow 4 weeks gap before the same radiologist interprets their cases again to allow them to forget their previous readings.
The final measurements across the two diagnostic methods (radiologist alone versus radiologist plus CAD) will be compared in terms of diagnostic accuracy and time-spent in reading scans.
I am reading this paper https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3522484/ and its authors report a split-plot study design in a similar CAD study. In their study, however, once a radiologist finished reading a case without CAD, they immediately turn CAD on, and report any additional findings. So there is a slight difference. I have unlinked the paired interpretations.
Other papers in radiology have used fully-crossed study designs https://repository.ubn.ru.nl/handle/2066/237631. A split-plot design seems very efficient but I am wondering if there are any gotcha's unknown to me in following this route. I would love to hear practising statisticians opinions on both designs.
