William F. Grossnickle

Interrater reliability and agreement of performance ratings: A methodological comparison

This paper demonstrates and compares methods for estimating the interrater reliability and interrater agreement of performance ratings. These methods can be used by applied researchers to investigate the quality of ratings gathered, for example, as criteria for a validity study, or as performance measures for selection or promotional purposes. While estimates of interrater reliability are frequently used for these purposes, indices of interrater agreement appear to be rarely reported for performance ratings. A recommended index of interrater agreement, theT index (Tinsley & Weiss, 1975), is compared to four methods of estimating interrater reliability (Pearsonr, coefficient alpha, mean correlation between raters, and intraclass correlation). Subordinate and superior ratings of the performance of 100 managers were used in these analyses. The results indicated that, in general, interrater agreement and reliability among subordinates were fairly high. Interrater agreement between subordinates and superiors was moderately high; however, interrater reliability between these two rating sources was very low. The results demonstrate that interrater agreement and reliability are distinct indices and that both should be reported. Reasons are discussed as to why interrater reliability should not be reported alone.

An evaluation of three corporate strategies for safety belt use promotion.

Different types of awareness sessions to promote employee safety belt use were evaluated. Across eight different employee safety meetings three dichotomous variables were manipulated: the type of presentation format (lecture vs discussion), the presence or absence of safety belt pledge cards, and the presence or absence of an incentive component. All groups showed a four-fold increase in safety belt use. Five months after the interventions, belt use was still significantly higher than baseline levels. Generalization across employees was demonstrated by a significant increase in safety belt use for those who did not attend the awareness session. The awareness session with a discussion format influenced greater increases in safety belt use than did the lecture-based sessions, but pledge cards and incentives did not increase the impact of these awareness sessions.

The effect of variable-rate cash incentives on safety belt use

Following an awareness campaign targeting the use of safety belts at a large pharmaceutical company, the present research examined the effects of a variable-rate cash-incentive strategy designed to further boost the level of plant-wide safety belt use. The observed percentage of belt use showed a sig- nificant increase during the incentive phase and a subsequent decline during short-term follow-up, although levels during follow-up were significantly higher than during baseline. Belt wearing was found to be essentially un- changed 6 months later during long-term follow-up.

Psychometric properties of the Wife Abuse Inventory

It is estimated that approximately 50% of all married couples experience violence during their relationship (Lewis, 1985). The present study (N = 305) investigated the psychometric properties of the Wife Abuse Inventory (WAI), which was designed to identify women at risk for abuse. The WAI was found to be internally consistent, and evidence was obtained for the validity of the WAI as an indicator of the potential for being abused by ones spouse. A factor analysis indicated that the WAI measures two main factors, emotional and physical abuse, and a moderate correlation between the WAI and the Child Abuse Potential Inventory also was obtained. The WAI has potential for use as a screening inventory in the identification of women at risk.

The impact of recipient list size and priority signs on electronic helping behavior

Abstract The purpose of this study was to examine the effects of recipient list size and priority signs on electronic helping behavior. Participants were sent an e-mailed request to complete a Web-based survey in one of the following conditions: alone, with 1 other person, with 14 others, or with 49 others. A high priority symbol (!) was attached to the e-mail message sent to half of the participants in each of the preceding conditions. We predicted that as the number of the people in the recipient list increased, (a) the number of individuals who completed the surveys would decrease and (b) the time that elapsed between the participant reading the e-mail request and completing the survey would increase. We also predicted that people who were sent the high priority request for help would (a) be particularly inclined to respond and (b) respond more quickly than those in the normal priority condition. Although no statistically significant results were obtained, the data did reveal notable concerns with the commonly used read receipt function. This research has important implications regarding the utility of read receipts, priority symbols, and personalized e-mail messages.

Attitudes toward Masturbation

105 male and 123 female college students were shown photographs of college women varying in attractiveness. Subjects were asked to identify the photograph they considered most attractive, the one they felt was of the woman most likely to be a virgin, and the one they felt was of the woman most likely to masturbate. All subjects selected as most attractive one of the photographs rated as high in attractiveness in prior research. When asked to select the picture of the individual most likely to be a virgin, subjects significantly more frequently selected a less attractive individual. This finding was also true when subjects selected the picture of the individual they believed most likely to masturbate. By the “beauty as good” model, these findings suggest that masturbation and virginity are viewed as negative sexual concepts by college students.

Wife abuse as it affects work behavior in a center for mentally retarded persons

This study examined differences between conjugally abused and nonabused female employees at a state-supported ICF/MR residential facility in relation to the potential for client abuse, absenteeism rates, and performance ratings. The research was conducted at a rural institution for mentally retarded persons at which approximately 72% of the workforce was female. A survey package composed of demographic questions, the Child Abuse Potential Inventory (Milner, 1980), and other self-report questions that concerned abuse was distributed to all permanent female employees. The survey return rate was 31.7%. Analysis of the data indicated a significant difference between CAP scores for emotionally and physically abused women and their nonabused counterparts (p less than .01). When a CAP score of 215 was used as a cut-off, 44% of the abused women were classified into the potential abuser category. No significant differences were revealed between categories of abuse and measures of absenteeism or performance appraisal scores.

Complexity of Effects of Personal Space

Results from two studies failed to replicate the finding that attitude change is a direct function of distance and provided further evidence that personal space is a complex and multidimensional variable. The exact effect of personal space seems dependent upon its interactions with other factors.