José Muñiz

Effect of the Number of Response Categories on the Reliability and Validity of Rating Scales

The Likert-type format is one of the most widely used in all types of scales in the field of social sciences. Nevertheless, there is no definitive agreement on the number of response categories that optimizes the psychometric properties of the scales. The aim of the present work is to determine in a systematic fashion the number of response alternatives that maximizes the fundamental psychometric properties of a scale: reliability and validity. The study is carried out with data simulated using the Monte Carlo method. We simulate responses to 30 items with correlations between them ranging from 0.2 to 0.9. We also manipulate sample size, analyzing four different sizes: 50, 100, 200, and 500 cases. The number of response options employed ranges from two to nine. The results show that as the number of response alternatives increases, both reliability and validity improve. The optimum number of alternatives is between four and seven. With fewer than four alternatives the reliability and validity decrease, an...

Directrices para la traducción y adaptación de los tests : segunda edición

BACKGROUND Adapting tests across cultures is a common practice that has increased in all evaluation areas in recent years. We live in an increasingly multicultural and multilingual world in which the tests are used to support decision-making in the educational, clinical, organizational and other areas, so the adaptation of tests becomes a necessity. The main goal of this paper is to present the second edition of the guidelines of the International Test Commission (ITC) for adapting tests across cultures. METHOD A task force of six international experts reviewed the original guidelines proposed by the International Test Commission, taking into account the advances and developments of the field. RESULTS As a result of the revision this new edition consists of twenty guidelines grouped into six sections: Precondition, test development, confirmation, administration, score scales and interpretation, and document. The different sections are reviewed, and the possible sources of error influencing the tests translation and adaptation analyzed. CONCLUSIONS Twenty guidelines are proposed for translating and adapting tests across cultures. Finally we discuss the future perspectives of the guidelines in relation to the new developments in the field of psychological and educational assessment.

Validation of the Schizotypal Personality Questionnaire—Brief Form in adolescents

The main objective of the study was to validate the Schizotypal Personality Questionnaire-Brief (SPQ-B) in a sample of non-clinical adolescents. In addition, the schizotypal personality structure and differences in the dimensions of schizotypy according to gender and age are analyzed. The sample comprises 1683 students, 818 males (48.6%), with a mean age of 15.9 years (SD=1.2). The results showed that the SPQ-B had adequate psychometric properties. Internal consistency of the subscales and total score ranged from 0.61 to 0.81. Confirmatory factor analyses indicated that the three-factor model (positive, negative, and disorganized) and the four-factor model (positive, paranoid, negative, and disorganized) fit reasonably well in comparison to the remaining models. With regard to gender and age, statistically significant differences were found due to age but not to gender. In line with previous literature, the results confirmed the multi-factor structure of the schizotypal personality in non-clinical adolescent populations. Future studies could use the SPQ-B as a screening self-report of rapid and efficient application for the detection of adolescents vulnerable to the development of schizophrenia-spectrum disorders in the general population, in genetically high-risk samples and in clinical studies.

Small Sample Studies to Detect Flaws in Item Translations

The number of tests being translated and adapted from 1 language and culture to others is increasing substantially. One shortcoming in current methodology for identifying flawed items due to the test translation-adaptation process is the failure to carry out empirical analyses. One important reason for not conducting empirical studies is the view that large examinee samples are required that are often not available in translation-adaptation studies. The purpose of this article was to investigate 2 simple procedures for detecting potentially flawed items with small samples: (a) conditional item p value comparisons, and (b) delta plots. Several factors were varied in this computer simulation study: sample sizes and ability distributions of the reference and focal groups, amount of differential item functioning (DIF), and the statistical characteristics of the items where DIF was found. The findings showed that the 2 simple graphical-descriptive procedures can be valuable in identifying flawed test items, especially when the size of the flaws is substantial. An application of both procedures to actual test data also supported their utility. Although this study was stimulated by questions that have arisen in the context of language translations of tests, the procedures for identifying potentially flawed items are equally applicable for identifying other potential sources of bias in the test items such as gender and race.1

Test Use in Spain, Portugal and Latin American Countries

Summary: The two main sources of errors in educational and psychological evaluation are the lack of adequate technical and psychometric characteristics of the tests, and especially the failure to properly implement the testing process. The main goal of the present research is to study the situation of test construction and test use in the Spanish-speaking (Spain and Latin American countries) and Portuguese-speaking (Portugal and Brazil) countries. The data were collected using a questionnaire constructed by the European Federation of Professional Psychologists Association (EFPPA) Task Force on Tests and Testing, under the direction of D. Bartram. In addition to the questionnaire, other ad hoc data were also gathered. Four main areas of psychological testing were investigated: Educational, Clinical, Forensic and Work. Key persons were identified in each country in order to provide reliable information. The main results are presented, and some measures that could be taken in order to improve the current tes...

New Guidelines for Developing Multiple-Choice Items

The rigorous construction of items constitutes a field of great current interest for psychometric researchers and practitioners. In previous studies we have reviewed and analyzed the existing guidelines for the construction of multiple-choice items. From this review emerged a new proposal for guidelines that is now, in the present work, subjected to empirical assessment. This assessment was carried out by users of the guidelines and by experts in item construction. The results endorse the proposal for the new guidelines presented, confirming the advantages in relation to their simplicity and efficiency, as well as permitting identification of the difficulties involved in drawing up and organizing some of the guidelines. Taking into account these results, we propose a new, refined set of guidelines that constitutes a useful, simple, and structured instrument for the construction of multiple-choice items.

Schizotypy in Adolescence : The Role of Gender and Age

Schizotypy is a multidimensional personality construct that appears to indicate psychosis proneness. Supposedly, schizotypal traits behave differently depending on a persons age and gender, but few studies have examined this relationship. In our study we used the Thinking and Perceptual Style Questionnaire and the Junior Schizotypy Scales. The sample was made up of 321 students (169 males) with an age range of 12 to 17 years. The results show significant differences in gender and age groups. Males score higher than females on Physical Anhedonia, Social Anhedonia, and Impulsive Non-Conformity scales, while females score higher or Positive Symptoms, Negative Evaluation, and Social Paranoia scales. Significant differences were also found among age groups: Unusual experiences, self-referent ideation, social paranoia, thought disorder, and negative evaluation were more frequent in later stages of adolescence. However, the meaning of this difference could be interpreted in terms of emotional turbulence rather than as a direct indicator of vulnerability to psychosis.

Utility of the Mantel-Haenszel Procedure for Detecting Differential Item Functioning in Small Samples.

Sample-size restrictions limit the contingency table approaches based on asymptotic distributions, such as the Mantel-Haenszel (MH) procedure, for detecting differential item functioning (DIF) in many practical applications. Within this framework, the present study investigated the power and Type I error performance of empirical and inferential criteria for DIF detection in small samples. Sample sizes (50/50, 100/50, 200/50, and 100/100 for the reference and focal groups, respectively), ability distributions (equal and unequal), and amount of DIF (moderate and high) were manipulated. The results show the advantages of employing theMHchi-square statistic using high levels of significance (• = .20) as opposed to the empirical criteria (cutoffs for categorizing DIF based on the magnitude of the MH common odds ratio estimator and the standardized p-difference statistic). Some considerations concerning Type I and Type II errors are made.

Measurement invariance of the Schizotypal Personality Questionnaire-Brief across gender and age.

The purpose of this study was to examine the dimensional structure and measurement invariance of the Schizotypal Personality Questionnaire-Brief (SPQ-B) (Raine and Benishay, 1995) across sex and age in a representative sample of nonclinical adolescents and young adults. The sample consisted of 1789 adolescents and young adults (42.1% males), with a mean age of 17.1years (S.D.=2.9). The results indicated that the Likert version of the SPQ-B showed adequate psychometric properties (α total score 0.89). The schizotypal personality models that presented the best fit indices were Raine et al.s (1994) three-factor model and Stefanis et al.s (2004) four-factor model. In addition, the results support the measurement invariance of the SPQ-B across sex and age. When the latent means of the schizotypal dimensions were compared across sex and age, statistically significant differences were found. Consistent with previous literature, schizotypal personality is a multidimensional construct whose structure appears invariant across sex and age. Future studies should examine the invariance of schizotypal personality across cultures, as well as using the SPQ-B as a screening method in the general population to detect individuals at risk for schizophrenia-spectrum disorders, given its rapid and easy administration.

Is there a Relationship between Class Size and Student Ratings of Teaching Quality

The relationship between class size and student evaluation of university teaching quality was analyzed. Data from 2,915 university classrooms were collected in classes ranging from 1 to 234 students. Results strongly support conclusions that (a) there is a weak relationship between class size and student ratings of teaching quality, when both statistical significance and effect size are taken into account; and (b) the relationship is a nonlinear one, whose shape is basically determined by the range of class size and the number of different values selected. Theoretical, methodological, and practical implications are discussed.