Dave Bartram

The Great Eight competencies: a criterion-centric approach to validation.

The author presents results of a meta-analysis of 29 validation studies (N=4,861) that uses the Great Eight competency factors (Kurz & Bartram, 2002) as the criterion measurement framework. Predictors of the Great Eight competencies based only on personality scales show moderate to good correlations with line-manager ratings for all 8 of the competencies. On their own, ability tests correlate with 4 of the 8 competencies, and together ability and personality data yield operational validities ranging from 0.20 to 0.44 for the 8 competencies. Operational validities for aggregated predictors with aggregated criteria were estimated to be 0.53. The value of differentiating the criterion space and of relating predictor variables to criterion variables in a one-to-one fashion is discussed.

The role of visual and semantic codes in object naming

Abstract Three experiments investigated the effects of familiarity, practice, and stimulus variability on naming latencies for photographs of objects. Latencies for pictures of objects having the same name decreased most with practice when the same picture was always used to represent a given object (Condition Ps-Ns), less if different views of the same object were used (Condition Pv-Ns), and least if pictures of different objects having the same name were used (Condition Pd-Ns). In all cases, however, the effect of practice was significant. The savings in naming latency associated with practice on Conditions Ps-Ns and Pv-Ns showed almost no transfer to condition Pd-Ns, even though the same responses were being given before and after transfer. However, practice on Condition Ps-Ns transferred completely to Condition Pv-Ns. Name frequency affected latency in all conditions. The frequency effect decreased slightly with practice. These results are related to several alternative models of the coding processes involved in naming objects. It is concluded that at least three types of representation may be necessary: visual codes, nonverbal semantic codes, and name codes. A distinction is made between visual codes that characterize two-dimensional stimuli and those that characterize three-dimensional objects.

Internet Recruitment and Selection: Kissing Frogs to find Princes

The Internet has already had a dramatic impact on the way in which recruitment and selection are carried out in North America, and the impact is increasingly being felt in terms of changes in practice in Europe and Asia-Pacific. The paper presents a picture of the current development of the Internet as a medium in general and as a recruitment and selection medium in particular. The new medium has enabled the widespread adoption of computer-based assessment and it is predicted that it will replace paper as the default medium before very long. A range of issues are raised and discussed. These include security, confidentiality, authentication, control of assessment conditions, control over practice and equality of access. It is argued that as the second generation of users takes over from the first generation, so inequality of skill and access are becoming less and less of an issue. Finally, some potential areas of abuse of the system are noted and a call is made for the development of international standards to protect the rights and interests of test providers, test users and test takers.

Levels of coding in picture-picture comparison tasks.

Two experiments are reported in which subjects had to match pairs of pictures of objects. “Same” pairs could be either identical (Ps), pictures of different views of the same object (Pv), or pictures of different objects having the same name (Pd). With line drawings as stimuli, RTs for Condition Ps were shorter than for Condition Pv, which in turn were shorter than for Condition Pd. Visual similarity had no effect on Pd RTs. However, in Experiment II, where photographs of objects with high-frequency (HF) and low-frequency (LF) names were used, no difference was found between Conditions Ps(HF), Ps(LF) and Condition Pv(HF); and no difference occurred between Conditions Pd(HF), Pd(LF) and Condition Pv(LF), the latter set of conditions being associated with longer RTs than the former. This pattern of results was found with both a .25-sec and a 2-sec ISI. The results are discussed in terms of the levels of coding involved in processing information from picture stimuli. It is concluded that at least two levels are involved in matching photographs of real objects (an object-code level and a nonvisual semantic code level), while a third level may be used in matching tasks involving stylized line drawings (a picture-code level).

Increasing Validity with Forced-Choice Criterion Measurement Formats

The relative validities of forced-choice (ipsative) and Likert rating-scale item formats as criterion measures are examined. While there has been much debate about the relative technical and psychometric merits and demerits of ipsative instruments, the present research focused on the crucial question of whether the use of this format has any practical benefit in terms of improved validity. An analysis is reported from a meta-analysis data set. This demonstrates that higher operational validity coefficients (prediction of line-manager ratings of competencies) are associated with the use of forced-choice (r=.38) rather than rating scale (r=.25) item formats for the criterion measurement instrument when performance is rated by the same line managers on both formats and where the predictor is held constant. Thus the apparent criterion-related validity of a predictor can increase by 50% simply by changing the format of the criterion measurement instrument. The implications of this for practice are discussed.

Test Use in Spain, Portugal and Latin American Countries

Summary: The two main sources of errors in educational and psychological evaluation are the lack of adequate technical and psychometric characteristics of the tests, and especially the failure to properly implement the testing process. The main goal of the present research is to study the situation of test construction and test use in the Spanish-speaking (Spain and Latin American countries) and Portuguese-speaking (Portugal and Brazil) countries. The data were collected using a questionnaire constructed by the European Federation of Professional Psychologists Association (EFPPA) Task Force on Tests and Testing, under the direction of D. Bartram. In addition to the questionnaire, other ad hoc data were also gathered. Four main areas of psychological testing were investigated: Educational, Clinical, Forensic and Work. Key persons were identified in each country in order to provide reliable information. The main results are presented, and some measures that could be taken in order to improve the current tes...

Definition and Assessment of Competences in the Context of the European Diploma in Psychology

The European Diploma in Psychology defines a common European standard for the competences required to practice as a psychologist. This paper describes how that standard was developed and defined, and why it was considered important to bring together the traditional input-based specification of professional competence, in terms of curriculum and training course content, with a more outcome-oriented approach that focuses on the competences that a professional psychologist needs to demonstrate in practice. The paper addresses three specific questions. What are the competences that a psychologist should possess? Are these competences the same for all areas of practice within professional psychology? How can these competences be assessed?

Test Qualifications and Test Use in the UK: The Competence Approach

The paper describes the current British Psychological Society (BPS) approach to the development of professional test user qualifications through the use of a competence-based approach to the certification of test users. The rationale is described and contrasted with previous mechanisms for controlling standards in test use. The overall strategy is to develop more competent test users and to provide better information for them about tests. The latter is being accomplished through the establishment of a Register of Test Users with its associated journal Selection and Development Review; and the publication by BPS Books of regular test reviews and updates. To date, a Certification process has been implemented relating to psychological test use in occupational assessment settings. This comprises qualifications at two levels: “Level A” (tests of ability and aptitude) and “Level B” (personality assessment). The design and implementation of this system has raised many problems - theoretical, practical, political...

Guidelines for Test Users: A Review of National and International Initiatives

Summary: The paper describes why we need guidelines on test use and why, in particular, we need international agreement on what these guidelines should be. The work of the International Test Commission (ITC) is described and the ITCs International Guidelines on Test Use are reviewed. Various other important national initiatives in Britain, Germany, The Netherlands, Sweden, and the United States are described together with the work of the European Federation of Professional Psychologists Associations (EFPPA) Standing Committee on Tests and Testing. While there is considerable agreement on what constitutes good practice in test use, there is wide diversity in the ways in which different countries have attempted to implement good practice or regulate test use. The need for guidelines and test user qualification procedures to be sufficiently flexible to accommodate innovation and change in assessment practices is stressed.

Variations in National Patterns of Testing and Test Use: The ITC/EFPPA International Survey

This study examined a range of issues relating to testing and test use in 36 countries. It also explored attitudes towards issues relating to competence in test use. Respondents included representatives of professional psychological associations, test publishers, and other groups with a national perspective on testing. For the purposes of the study, the domain of testing was divided into four areas: Educational, Clinical, Forensic, and Organizational. While detailed reports on the overall results of the survey have been reported elsewhere (Bartram & Coyne, 1998a, 1998b), the present paper focuses on patterns of similarity and difference between countries. Cluster analysis was used to examine patterns of consistency in responses between countries. The results indicate that the clustering of countries is, for the most part, not attributable to either geographical proximity or to common cultural or language factors. In reviewing these patterns, however, one should note that there is, overall, more consistenc...