Evaluation of teaching in a student-led clinic environment: Assessing the reliability of a questionnaire

Abstract

Abstract Introduction The Osteopathy Clinical Teaching Questionnaire (OCTQ) has been designed to evaluate the quality of clinical teaching in osteopathy student-led clinics. Previous research has provided evidence for the scoring, generalisation and implications components of the validity argument for the OCTQ. The aim of the present study was to assess the reliability of the OCTQ providing further evidence for the validity argument. Method Senior students in the final years of two Australian osteopathy programs completed the OCTQ for each clinical educator with whom they had worked with during a semester. Generalisability analysis, test-retest reliability and internal structure estimation were used to investigate the reliability. Results Each of the forty-one clinical educators received an average of 5.97 evaluations resulting in a G-coefficient of 0.63. D-studies demonstrated eight evaluations are required for a coefficient of 0.7 and fourteen for a coefficient of 0.8. Test-retest reliability demonstrated substantial to almost perfect agreement for all but one OCTQ item. Internal structure estimations using Cronbach s alpha and McDonald s omega were both 0.93. Conclusion The results suggest that the OCTQ is a reliable tool to provide feedback to clinical educators, and potentially, used to inform decisions to reward clinical educators for their performance.