Tomoya Okubo

Assessing Items on the SF-8 Japanese Version for Health-Related Quality of Life: A Psychometric Analysis Based on the Nominal Categories Model of Item Response Theory

OBJECTIVES The Short Form-8 (SF-8) questionnaire is a commonly used 8-item instrument of health-related quality of life (QOL) and provides a health profile of eight subdimensions. Our aim was to examine the psychometric properties of the Japanese version of the SF-8 instrument using methodology based on nominal categories model. METHODS Using data from an adjusted random sample from a nationally representative panel, the nominal categories modeling was applied to SF-8 items to characterize coverage of the latent trait (theta). Probabilities for response choices were described as functions on the latent trait. Information functions were generated based on the estimated item parameters. RESULTS A total of 3344 participants (53%, women; median age, 35 years) provided responses. One factor was retained (eigenvalue, 4.65; variance proportion of 0.58) and used as theta. All item response category characteristic curves satisfied the monotonicity assumption in accurate order with corresponding ordinal responses. Four items (general health, bodily pain, vitality, and mental health) cover most of the spectrum of theta, while the other four items (physical function, role physical [role limitations because of physical health], social functioning, and role emotional [role limitations because of emotional problems] ) cover most of the negative range of theta. Information function for all items combined peaked at -0.7 of theta (information = 18.5) and decreased with increasing theta. CONCLUSION The SF-8 instrument performs well among those with poor QOL across the continuum of the latent trait and thus can recognize more effectively persons with relatively poorer QOL than those with relatively better QOL.

Professionalism Mini-Evaluation Exercise for medical residents in Japan: a pilot study.

Context  Assessing medical professionalism among medical residents is of great importance. The Professionalism Mini‐Evaluation Exercise (P‐MEX) is a tool for assessing professionalism that was developed, tested for reliability and validated in Canada. Prior to the present study, no Japanese version of the P‐MEX had been tested.

Introducing the professionalism mini-evaluation exercise (p-MEX) in Japan: Results from a multicenter, cross-sectional study

Purpose Despite the growing importance of and interest in medical professionalism, there is no standardized tool for its measurement. The authors sought to verify the validity, reliability, and generalizability of the Professionalism Mini-Evaluation Exercise (P-MEX), a previously developed and tested tool, in the context of Japanese hospitals. Method A multicenter, cross-sectional evaluation study was performed to investigate the validity, reliability, and generalizability of the P-MEX in seven Japanese hospitals. In 2009–2010, 378 evaluators (attending physicians, nurses, peers, and junior residents) completed 360-degree assessments of 165 residents and fellows using the P-MEX. The content validity and criterion-related validity were examined, and the construct validity of the P-MEX was investigated by performing confirmatory factor analysis through a structural equation model. The reliability was tested using generalizability analysis. Results The contents of the P-MEX achieved good acceptance in a preliminary working group, and the poststudy survey revealed that 302 (79.9%) evaluators rated the P-MEX items as appropriate, indicating good content validity. The correlation coefficient between P-MEX scores and external criteria was 0.78 (P < .001), demonstrating good criterion-related validity. Confirmatory factor analysis verified high path coefficient (0.60–0.99) and adequate goodness of fit of the model. The generalizability analysis yielded a high dependability coefficient, suggesting good reliability, except when evaluators were peers or junior residents. Conclusions Findings show evidence of adequate validity, reliability, and generalizability of the P-MEX in Japanese hospital settings. The P-MEX is the only evaluation tool for medical professionalism verified in both a Western and East Asian cultural context.

Development and validation of a 15-item Japanese Health Knowledge Test.

Background Health literacy affects the acquisition of health knowledge and is thus linked to health outcomes. However, few scales have been developed to assess the level of health knowledge among the general public. Methods The 15-item Japanese Health Knowledge Test (J-HKT) was developed by using item response theory to score an item pool. We examined the construct validity of the J-HKT in relation to health literacy items, and analyzed the sociodemographic and behavioral factors associated with poor health knowledge. Results We enrolled 1040 adult participants (mean age, 57 years; women, 52%). The 15 items that best identified people with poor health knowledge were selected. For all items on the J-HKT, the information function curves had a peak in the negative spectrum of the latent trait. As compared with participants reporting high levels of income, educational attainment, and literacy, those with low levels of income, education, and literacy had a lower total score on the J-HKT. As compared with non/light drinkers, moderate and heavy drinkers had lower total scores on the J-HKT. Conclusions The J-HKT may prove useful in measuring health knowledge among the general public, and in identifying and characterizing those with poor health knowledge.

Reliability and acceptability of six station multiple mini-interviews: past-behavioural versus situational questions in postgraduate medical admission

BackgroundThe multiple mini-interview (MMI) is increasingly used for postgraduate medical admissions and in undergraduate settings. MMIs use mostly Situational Questions (SQs) rather than Past-Behavioural Questions (PBQs). A previous study of MMIs in this setting, where PBQs and SQs were asked in the same order, reported that the reliability of PBQs was non-inferior to SQs and that SQs were more acceptable to candidates. The order in which the questions are asked may affect reliability and acceptability of an MMI. This study investigated the reliability of an MMI using both PBQs and SQs, minimising question order bias. Acceptability of PBQs and SQs was also assessed.MethodsForty candidates applying for a postgraduate medical admission for 2016–2017 were included; 24 examiners were used. The MMI consisted of six stations with one examiner per station; a PBQ and a SQ were asked at every station, and the order of questions was alternated between stations. Reliability was analysed for scores obtained for PBQs or SQs separately, and for both questions. A post-MMI survey was used to assess the acceptability of PBQs and SQs.ResultsThe generalisability (G) coefficients for PBQs only, SQs only, and both questions were 0.87, 0.96, and 0.80, respectively. Decision studies suggested that a four-station MMI would also be sufficiently reliable (G-coefficients 0.82 and 0.94 for PBQs and SQs, respectively). In total, 83% of participants were satisfied with the MMI. In terms of face validity, PBQs were more acceptable than SQs for candidates (p = 0.01), but equally acceptable for examiners (88% vs. 83% positive responses for PBQs vs. SQs; p = 0.377). Candidates preferred PBQs to SQs when asked to choose one, though this difference was not significant (p = 0.081); examiners showed a clear preference for PBQs (p = 0.007).ConclusionsReliability and acceptability of six-station MMI were good among 40 postgraduate candidates; modelling suggested that four stations would also be reliable. SQs were more reliable than PBQs. Candidates found PBQs more acceptable than SQs and examiners preferred PBQs when they had to choose between the two. Our findings suggest that it is better to ask both PBQs and SQs during an MMI to maximise acceptability.

Postgraduate medical education in Japan: Missed opportunity for learning clinical reasoning

Most Japanese physicians who graduated before 2004 received training in only single specialty during their residency and thus lacked attitudes, skills, and knowledge for basic primary and emergency care.1 Some of them revealed inadequate quality of primary and emergency care and public pressure throughout Japan had promoted the introduction of mandatory medical education for all newly graduating physicians.2 Consequently, the current postgraduate medical education (PGME) was implemented in 2004 by the Japanese government as new 2year superrotation program, including internal medicine, emergency medicine, and community medicine as mandatory rotations and other specialties as selective rotations.3 Objectives of this PGME are to educate residents to acquire attitudes, skills, and knowledge for providing basic clinical care. A study showed that physicians who received the current PGME program were more likely to adhere to an appropriate standard of care in emergency medicine and were more confident in caring for patients with acute illnesses, compared to those who graduated before 2004 and did not receive the current PGME.4 Although the current PGME is better than the norotation training prior to 2004, there may be still some weaknesses of the current PGME. First, overtesting in diagnostic process is highly prevalent in Japanese hospitals,5 and thus, some residents, if not all, may tend to order a long list of diagnostic tests, which may mask lack of their diagnostic reasoning skills. Because overtesting can cause harm to patients through downstream tests including invasive tests, the overtesting practice might potentially lead to greater harmful events in the future of Japan. Second, some residents in Japan have interests in acquiring procedural skills, which can obviously show their competency to other members of their care team, including teaching faculty. However, competent residents need not only procedural skills, but also basic skills of clinical reasoning based on good history taking and physical examination. Therefore, we conducted a nationwide study comparing postgraduate year (PGY)1 and PGY2 residents for knowledge of basic clinical care, including clinical reasoning, based on results of general medicine intraining examination (GMITE). The previous results of this examination have been published elsewhere and elucidated important educational factors related to the GMITE scores.6–9 We analyzed the most recent (February 2017) GMITE scores of resident physicians between postgraduate year (PGY)1 and PGY2 resident physicians in 459 Japanese teaching hospitals which participated in the GMITE. A total of 5593 resident physicians (2678 PGY1 and 2915 PGY2) participated in the examination. A total of 491 residents were excluded from analyses, because we could not obtain their consent to the data use. Overall mean score among PGY1 and PGY2 were 32.3 ± 5.1 and 32.5 ± 5.5, respectively, and these were not significantly different. However, additional analysis was conducted using scores of residents only in top 200 hospitals in terms of the higher mean scores among participating residents. This subgroup analysis showed mean score 34.9 ± 5.0 of PGY2 was significantly higher than that 34.4 ± 4.7 of PGY1 (Table 1). Based on our results, there are few improvements of the basic clinical knowledge over the PGY1 to PGY2 years in lowscore hospitals, but residents in highscore hospitals have the significant improvement. In other words, good teaching hospitals may effectively teach knowledge for clinical reasoning to residents, but other hospitals may not. Our result should be an alarm for teachers and residents in Japan, and further studies for assessing clinical reasoning skills in each PGY year of Japanese residents should be considered. Meanwhile, clinician educators in teaching hospitals should focus more on teaching clinical reasoning.

Skills acquisition for novice learners after a point-of-care ultrasound course: does clinical rank matter?

BackgroundFew studies have compared the effectiveness of brief training courses on point-of-care ultrasound (POCUS) skill acquisition of novice attending physicians vs. trainees. The purpose of this study was to evaluate the change in POCUS image interpretation skills and confidence of novice attending physicians vs. trainees after a 1-day POCUS training course.MethodsA 1-day POCUS training course was held in March 2017 in Japan. A standardized training curriculum was developed that included online education, live lectures, and hands-on training. The pre-course assessment tools included a written examination to evaluate baseline knowledge and image interpretation skills, and a physician survey to assess confidence in performing specific ultrasound applications. The same assessment tools were administered post-course, along with a course evaluation. All learners were novices and were categorized as trainees or attending physicians. Data were analyzed using two-way analysis of variance.ResultsIn total, 60 learners attended the course, and 51 learners (85%) completed all tests and surveys. The 51 novice learners included 29 trainees (4 medical students, 9 PGY 1–2 residents, 16 PGY 3–5 residents) and 22 attending physicians (6 PGY 6–10 physicians, and 16 physicians PGY 11 and higher). The mean pre- and post-course test scores of novice trainees improved from 65.5 to 83.9% while novice attending physicians improved from 66.7 to 81.5% (p < 0.001). The post-course physician confidence scores in using ultrasound significantly increased in all skill categories for both groups. Both trainees and attending physicians demonstrated similar improvement in their post-course test scores and confidence with no statistically significant differences between the groups. The course evaluation scores for overall satisfaction and satisfaction with faculty members’ teaching skills were 4.5 and 4.6 on a 5-point scale, respectively.ConclusionsBoth novice trainees and attending physicians showed similar improvement in point-of-care ultrasound image interpretation skills and confidence after a brief training course. Although separate training courses have traditionally been developed for attending physicians and trainees, novice learners of point-of-care ultrasound may acquire skills at similar rates, regardless of their ranking as an attending physician or trainee. Future studies are needed to compare the effectiveness of short training courses on image acquisition skills and determine the ideal course design.

Development of culture-sensitive clinical teacher evaluation sheet in the Japanese context

Abstract Aim: Many instruments for evaluating clinical teaching have been developed, albeit most in Western countries. This study aims to develop a validated cultural and local context sensitive instrument for clinical teachers in an East Asian setting (Japan), Japanese Clinical Teacher Evaluation Sheet (JaCTES). Methods: A multicenter, cross-sectional evaluation study was conducted. We collected a total of 1368 questionnaires on 304 clinical teachers, completed by residents in 16 teaching hospitals. The construct validity was examined by conducting a factor analysis and using structural equation modeling (SEM). We also assessed the reliability using generalizability analysis and decision study. Results: Exploratory factor analysis resulted in three-factor (role model, teaching activities, and accessibility) model including 18 items. Confirmatory factor analysis was performed, using SEM. The comparative fit index was 0.931 and the root mean square error of approximation was 0.087, meaning an acceptable goodness of fit for this model. To obtain a reliable dependability-coefficient of at least 0.70 or higher, 5–8 resident responses are necessary. Discussion and conclusion: JaCTES is the first reported instrument with validity evidence of content and internal structure and high feasibility in Japan, an East Asian setting. Medical educators should be aware of the local context and cultural aspects in evaluating clinical teachers.

Educational environment and the improvement in the General Medicine In‐training Examination score

Dear Editor: We investigated the association between the educational environment and performance in the General Medicine Intraining Examination (GMITE) among postgraduate year (PGY)1 and 2 resident physicians in Japanese teaching hospitals since 2012. GMITE includes 100 questions designed by a committee comprised of experts organized by the Japan Organization of Advancing Medical Education Program.1 According to our previous report1, hospitals with a general medicine (GM) department or those located in a provincial community had significantly higher GMITE score compared with the other hospitals. Data from the 2013 GMITE showed that appropriate emergency department (ED) workload, inpatient caseload, and online learning resource were associated with greater clinical knowledge.2 In addition, in 2014, we found that factors affecting hospital volume, including the number of hospitalizations, had a positive impact on GMITE scores.3 While our team have been extensively studied factors related to the performance of the GMITE, because previous studies were analyzing crosssectional data, we could not evaluate the improvement in GMITE score over time. In this context, we investigate how educational environment of hospitals is related to the improvement in GMITE score using the longitudinal data obtained during two consecutive years. We analyzed GMITE scores of 241 resident physicians affiliated to 46 Japanese teaching hospitals (42 community and four university hospitals, age 26.8±3.0 years, 176 males and 65 females) underwent GMITE in 2013 and 2014. university hospitalswere collected using a selfreporting questionnaire sheet, the Residency Electronic Information System website, and the Foundation for the Promotion of Medical Training website. We examined the association between the educational environment and changes in the GMITE score (PGY2 score minus PGY1 score) using hierarchical linear regression models. The covariates with P<.10 on the univariable analysis were included in our multivariable model (Table 1). Our results showed that age, the number of ED duty (35 or ≥6 per month), and the number of inpatients the resident was responsible for (monthly average 59) were associated with the improvement in GMITE scores (Table 1). A previous study showed that a heavy call rotation (every fourth or fifth night) was related to postcall performance impairment.4 Overnight calls were associated with higher burnout and fatigue scores.5 Recently, there has been a trend for limiting workloads for residents because there might have been relationship between an excessive workload and higher risk at burnout.

Modeling Viewpoint Shifts in Probabilistic Choice

A number of mathematical models for overcoming intransitive choice have been proposed and tested in the literature of decision theory. This article presents the development of a new stochastic choice model based on multidimensional scaling. This allows decision-makers to have multiple viewpoints, whereas current multidimensional scaling models are based on the assumption that a subject or group of subjects has only one viewpoint. The implication of our model is that subjects make an intransitive choice because they are able to shift their viewpoint. This paper also presents the maximum likelihood estimation of the proposed model, and reanalyzes Tversky’s gamble experiment data.