Hiroyuki Noguchi

Development of a Japanese version of the Epworth Sleepiness Scale (JESS) based on Item Response Theory

BACKGROUND Various Japanese versions of the Epworth Sleepiness Scale (ESS) have been used, but none was developed via standard procedures. Here we report on the construction and testing of the developer-authorized Japanese version of the ESS (JESS). METHODS Developing the JESS involved translations, back translations, a pilot study, and psychometric testing. We identified questions in the ESS that were difficult to answer or were inappropriate in Japan, proposed possible replacements for those questions, and tested them with analyses based on item response theory (IRT) and classical test theory. The subjects were healthy people and patients with narcolepsy, idiopathic hypersomnia, or obstructive sleep apnea syndrome. RESULTS We identified two of our proposed questions as appropriate replacements for two problematic questions in the ESS. The JESS had very few missing data. Internal consistency reliability and test-retest reliability were high. The patients had significantly higher JESS scores than did the healthy people, and higher JESS scores were associated with worse daytime function, as measured with the Pittsburgh Sleep Quality Index. CONCLUSIONS In Japan, the JESS provides reliable and valid information on daytime sleepiness. Researchers who use the ESS with other populations should combine their knowledge of local conditions with the results of psychometric tests.

Psychological Distance Between Categories in the Likert Scale Comparing Different Numbers of Options

This study examined whether the number of options in the Likert scale influences the psychological distance between categories. The most important assumption when using the Likert scale is that the psychological distance between options is equal. The authors proposed a new algorithm for calculating the scale values of options by applying item response theory and the ideas of Wakita to reveal the influence of the number of categories. Three types of questionnaires that were composed of the same items, but used different numbers of options to assess these items (specifically, 4-, 5-, and 7-point scales), were completed by 722 undergraduate students. The results indicated that the number of options influenced the psychological distance between options, particularly for the 7-point scale. This influence was revealed only by the authors’ algorithm; descriptive statistics and coefficients of reliability did not show that the number of options had a prominent influence. The importance of the number of options and the new algorithm are discussed.

Improving the measurement accuracy of the effort-reward imbalance scales

Background: The effort-reward imbalance (ERI) scale items are answered in a two-step process, but the justification is questioned for the formulation of summary measure by combining information rated in two steps. Purpose: To examine whether the basic prerequisites of the ERI scales are empirically satisfied and to seek ways to improve the rating procedure. Methods: A polytomous item response theory (IRT) model was applied to the responses of 20,256 workers who completed the ERI scales. To determine the most appropriate statistical justification, three alternative scoring algorithms were compared with regard to the test properties revealed by the IRT analyses and efficiencies of screening performance and criterion validity against depressive symptomatology. Results: The rated raw-score units did not reflect the hypothesized order of lowest stress levels to highest stress levels. Exchanging or collapsing the lowest two categories of a Likert scaled item, where data of different quality are combined, solved this problem, thereby making the test content more appropriate. The modified rating improved the efficiencies of screening performance and the correlation of the stress summary measures against health criterion, i.e., depression. Conclusion: An avoidable measurement error exists in the current ERI scales. Modifying the rating procedure can improve the measurement accuracy.

Development and validation of a new instrument for testing functional health literacy in Japanese adults

This study aimed to develop a reliable and valid measure of functional health literacy in a Japanese clinical setting. Test development consisted of three phases: generation of an item pool, consultation with experts to assess content validity, and comparison with external criteria (the Japanese Health Knowledge Test) to assess criterion validity. A trial version of the test was administered to 535 Japanese outpatients. Internal consistency reliability, calculated by Cronbachs alpha, was 0.81, and concurrent validity was moderate. Receiver Operating Characteristics and Item Response Theory were used to classify patients as having adequate, marginal, or inadequate functional health literacy. Both inadequate and marginal functional health literacy were associated with older age, lower income, lower educational attainment, and poor health knowledge. The time required to complete the test was 10-15 min. This test should enable health workers to better identify patients with inadequate health literacy.

The Effect of Alternative Scoring Procedures on the Measurement Properties of a Self-Administered Depression Scale

To investigate the effect of response alternatives/scoring procedures on the measurement properties of the Center for Epidemiologic Studies Depression Scale (CES-D) which has the four response alternatives, a polytomous item response theory (IRT) model was applied to the responses of 2,061 workers and university students (1,640 males, 421 females). Test information functions derived from the polytomous IRT analyses on the CES-D data with various scoring procedures indicated that: (1) the CES-D with its standard (0-1-2-3) scoring procedure should be useful for screening to detect subjects with “at high-risk” of depression if the &thgr; point showing the highest information corresponds to the cut-off point, because of its extremely higher information; (2) the CES-D with the 0-1-1-2 scoring procedure could cover wider range of depressive severity, suggesting that this scoring procedure might be useful in cases where more exhaustive discrimination in symptomatology is of interest; and (3) the revised version of CES-D with replacing original positive items into negatively revised items outperformed the original version. These findings have never been demonstrated by the classical test theory analyses, and thus the utility of this kind of psychometric testing should be warranted to further investigation for the standard measures of psychological assessment.