Hedy Fairbairn

Consistency and Inconsistency in PhD Thesis Examination.

This is a mixed methods investigation of consistency in PhD examination. At its core is the quantification of the content and conceptual analysis of examiner reports for 804 Australian theses. First, the level of consistency between what examiners say in their reports and the recommendation they provide for a thesis is explored, followed by an examination of the degree of discrepancy between examiner recommendations and university committee decisions on the theses. Two groups of discrepant recommendations are identified and analysed in depth. Finally the main sources of inconsistency are identified. It was found that the comments of a small minority of examiners were inconsistent with each other or with the committee decision in a significant way. Much more commonly the texts of examiner reports were highly consistent and were closely reflected in the final committee decision.

Examiner comment on the literature review in Ph.D. theses

The review of literature, so central to scholarly work and disciplined inquiry, is expected of the Ph.D. student, but how far along the road are they expected to travel? This article investigates the expectations of ‘the literature’ in research and scholarship at Ph.D. level from the examiner and assessment perspective. The analysis draws on the examiner report data for 501 candidates (1310 reports) across five Australian universities. On average about one‐tenth of an examiner report is devoted to the literature and examiners provide detail about coverage, types of errors and the nature of use of the literature. It was the latter type of comment about coherent and substantive use of the literature that provided the most information about ‘expectation’. Examiners identified ‘working understanding’, ‘critical appraisal’ of the body of literature, ‘connection of the literature to findings’, and ‘disciplinary perspective’ as key indicators of performance in the candidate’s use of the literature. While examiners appeared to anticipate that all these elements should be present in scholarly work (and identified them in the best theses), they were prepared to accept less for a barely passable thesis, but pressed for at least some demonstration of critical appraisal.

The focus and substance of formative comment provided by PhD examiners

In practice and process PhD examination is distinctive, reflecting the high expectations of students whose learning has been directed to their becoming researchers. This article builds on previous research on the examination of Australian theses that revealed that examiners in Science (n = 542) and Education (n = 241) provide a substantial proportion of formative comment in their reports, much of which is constructed in a way that anticipates reflective engagement by the student. Detailed examination of the formative text identified nine categories of comment directed at three collective groupings of weaknesses or flaws related to less favourable recommendation. The flaws are related to ‘fundamentals’, ‘project’ and ‘argument’. There were discipline differences, including significantly more comment in Science, indicating that the candidate should attend further to the data and analysis in their project and the fundamentals of presentation. In Education there was more emphasis on improving argument.

Examiner reference to theory in PhD theses

As we were aware of the confusing and wide-ranging disciplinary and individual positions on the importance of theory in research, this study sought to determine how thesis examiners emphasised theory in their reports in order to inform candidate learning. While references to theory were not prominent in reports, examiner comment coalesced into six categories indicative of ‘accuracy and completeness’, ‘grasp’, ‘alignment’, ‘coherence and consistency’, ‘treatment of findings and discussion’, and ‘dimensions of contribution’. It was evident in summative comments that most examiners expected to find accuracy and grasp in a thesis. Grasp universally inspired confidence. The findings indicate positive recognition of a spectrum of sophistication in treatment of theory irrespective of discipline. Specific comment capturing a fusion of most or all the categories in one thesis was rare and always positive. Based on these categories, the paper provides clarity in what candidates need to attend to with respect to theory.

Exploring Doctoral Examiner Judgements Through the Lenses of Habermas and Epistemic Cognition

Abstract Evaluation of research is a core function of academic work, yet there has been very little theoretical development about what it means to ‘know’ in relation to judgements made in examination of doctoral research. This chapter addresses the issue by reflecting on findings from three projects aimed at enhancing understanding of doctoral examination. In order to progress understanding about knowledge judgements in the doctoral research context, the chapter draws on two key contributions in the field of knowledge and knowing, namely, Habermas’ cognitive interests and Chinn, Buckland and Samarapungavan’s notion of epistemic cognition. It examines the common ground between the two bodies of theory, drawing illustratively on empirical work in the field of doctoral examination. The comparison of the Habermasian theory of cognitive interests with Chinn et al.’s notion of epistemic cognition led to the conclusion that there were areas of overlap between the two conceptual schemas that could be utilised to advance research into doctoral examination in higher education. Habermas’ cognitive interests (which underpin his ways of knowing theory) offer a conceptual lens that facilitates analysis of the interaction of ontological and epistemic components of knowledge production. Chinn et al.’s notion of epistemic cognition allows for finer grained analysis of aspects of the cognitive work involved in knowledge rendition. This work is particularly pertinent in an era that sees the boundaries of the disciplines being challenged by the need for new perspectives and cross-disciplinary approaches to solving complex problems.