Ted A. Baumgartner

Reading Statistics and Research (5th ed.)

The users of this book can be teachers of measurement, research, and statistics courses. Also, this book can be valuable to researchers and consumers of research as a reference. The book is written with an easy-to-understand explanation of a variety of topics and techniques used in research. There are no formulas or computational examples to confuse a person who just wants to know what a technique is or the basic information about a topic. There are review terms, items in the companion website, and exercises inside Research Navigator at the end of each chapter. Chapter 1 explains the typical format of a journal article. This chapter familiarizes the reader with what to expect to see in any journal article. To some degree, Chapter 1 lays the ground work for the rest of the book. Chapters 2 and 3 are about descriptive statistics and bivariate correlation. Graphs, measures of central tendency, measures of variability, standard scores, the correlation coefficient, and different kinds of correlational procedures are covered in these chapters. Chapter 4 is on reliability and validity. Both terms are well-defined, and different approaches to estimating reliability and validity are presented. The author’s comments at the end of the chapter are excellent and quite practical. Chapter 5 covers foundations of inferential statistics. Here is where the concept of inference and sampling techniques is presented. Some common problems in sampling are addressed, such as low response rate and attrition. This chapter lays the groundwork for many of the later chapters.

The Reliability of the Modified Balance Error Scoring System

Objective:Study 1 investigated the intraclass reliability and percent variance associated with each component within the traditional Balance Error Scoring System (BESS) protocol. Study 2 investigated the reliability of subsequent modifications of the BESS. Design:Prospective cross-sectional examination of the traditional and modified BESS protocols. Setting:Schools participating in Georgia High School Athletics Association. Intervention:The modified BESS consisted of 2 surfaces (firm and foam) and 2 stances (single-leg and tandem-leg stance) repeated for a total of three 20-second trials. Participants:Participants consisted of 2 independent samples of high school athletes aged 13 to 19 years. Main Outcome Measures:Percent variance for each condition of the BESS was obtained using GENOVA 3.1. An intraclass reliability coefficient and repeated measures analysis of variance were calculated using SPSS 13.0. Results:Study 1 obtained an intraclass correlation coefficient (r = 0.60) with stance accounting for 55% of the total variance. Removing the double-leg stance increased the intraclass correlation coefficient (r = 0.71). Study 2 found a statistically significant difference between trials 1 and 2 (F(1.65,286) = 4.890, P = 0.013) and intraclass reliability coefficient of r = 0.88 for 3 trials of 4 conditions. Conclusions:The variance associated with the double-leg stance was very small, and when removed, the intraclass reliability coefficient of the BESS increased. Removal of the double-leg stance and addition of 3 trials of 4 conditions provided an easily administered, cost-effective, time-efficient tool that provides reliable objective information for clinicians to base clinical decisions upon.

Confidence Limits for Intraclass Reliability Coefficients

A reliability coefficient (R) calculated using sample data may differ considerably in value from the population reliability coefficient. The reporting of confidence limits for the population reliability coefficient assists a person in interpreting the goodness of a sample reliability coefficient. The study was designed to (a) present the procedures for calculating confidence limits for R, (b) present confidence limits for several values of R and several sample sizes, and (c) describe a program in SPSS (1999) that provides confidence limits and hypothesis testing for Rs. The confidence limits for 3 different values of R, 2 different ANOVA models, 2 different sample sizes, 2 different numbers of repeated measures, and 2 different criterion scores were calculated. The confidence limits became narrower as R increased in value, as sample size increased in value, and as the number of repeated measures increased. For a given (a) R, (b) sample size, (c) number of repeated measures, and (d) criterion score, the confidence limits are the same for a 1-way and a 2-way Analysis of Variance model. Furthermore, when all other factors are equal, the confidence limits tend to be narrower for a criterion score that is a single score rather than a mean score. Based on the confidence limits reported, a sample size of at least 50 is needed to have relatively narrow confidence limits.

ImPact Test-Retest Reliability: Reliably Unreliable?

CONTEXT Computerized neuropsychological testing is commonly used in the assessment and management of sport-related concussion. Even though computerized testing is widespread, psychometric evidence for test-retest reliability is somewhat limited. Additional evidence for test-retest reliability is needed to optimize clinical decision making after concussion. OBJECTIVE To document test-retest reliability for a commercially available computerized neuropsychological test battery (ImPACT) using 2 different clinically relevant time intervals. DESIGN Cross-sectional study. SETTING Two research laboratories. PATIENTS OR OTHER PARTICIPANTS Group 1 (n = 46) consisted of 25 men and 21 women (age = 22.4 ± 1.89 years). Group 2 (n = 45) consisted of 17 men and 28 women (age = 20.9 ± 1.72 years). INTERVENTION(S) Both groups completed ImPACT forms 1, 2, and 3, which were delivered sequentially either at 1-week intervals (group 1) or at baseline, day 45, and day 50 (group 2). Group 2 also completed the Green Word Memory Test (WMT) as a measure of effort. MAIN OUTCOME MEASURES Intraclass correlation coefficients (ICCs) were calculated for the composite scores of ImPACT between time points. Repeated-measures analysis of variance was used to evaluate changes in ImPACT and WMT results over time. RESULTS The ICC values for group 1 ranged from 0.26 to 0.88 for the 4 ImPACT composite scores. The ICC values for group 2 ranged from 0.37 to 0.76. In group 1, ImPACT classified 37.0% and 46.0% of healthy participants as impaired at time points 2 and 3, respectively. In group 2, ImPACT classified 22.2% and 28.9% of healthy participants as impaired at time points 2 and 3, respectively. CONCLUSIONS We found variable test-retest reliability for ImPACT metrics. Visual motor speed and reaction time demonstrated greater reliability than verbal and visual memory. Our current data support a multifaceted approach to concussion assessment using clinical examinations, symptom reports, cognitive testing, and balance assessment.

The Reliability of Distance Run Tests for Children in Grades K—4

The purpose of this study was to determine test-retest reliability for the 1-mile, 3/4-mile, and 1/2-mile distance run/alk tests for children in Grades K-4. Fifty-one intact physical education classes were randomly assigned to one of the three distance run conditions. A total of 1,229 (621 boys, 608 girls) completed the test-retests in the fall (October), with 1,050 of these students (543 boys, 507 girls) repeating the tests in the spring (May). Results indicated that the 1-mile run/walk distance, as recommended for young children in most national test batteries, has acceptable intraclass reliability (.83 less than R less than .90) for both boys and girls in Grades 3 and 4, has minimal (fall) to acceptable (spring) reliability for Grade 2 students (.70 less than R less than .83), but is not reliable for children in Grades K and 1 (.34 less than R less than .56). The 1/2 mile was the only distance meeting minimal reliability standards for boys and girls in Grades K and 1 (.73 less than R less than .82). Results also indicated that reliability estimates remained fairly stable across gender and age groups from the fall to spring testing periods, with the exception of the noticeably improved values for Grade 2 students on the 1-mile run/walk test. Criterion-referenced reliability (P, percent agreement) was also estimated relative to Physical Best and Fitnessgram run/walk standards. Reliability coefficients for all age group standards were acceptable to high (.70 less than P less than .95), except for Fitnessgram standards for 5-year-old girls on the 1-mile test for both fall and spring and for 6-year-old boys and girls on the 1-mile test administered in the spring.

Validity of Field Tests of Upper Body Muscular Strength

The purpose of this study was to determine the validity of five field tests (FTs) of upper body muscular strength and endurance (UBMSE) in 9-10-year-old children. Ninety-four children (38 boys, 56 girls) performed five FTs of UBMSE: pull-ups, flexed arm hang, push-ups, Vermont modified pull-ups (VMPU), and New York modified pull-ups. They also performed three criterion tests (CTs) of strength and three CTs of muscular endurance using a supported weight, set-resistance device. Zero-order correlations between the sum of the standard scores on the three CTs of strength (SUM1RM) and the FTs were nonsignificant. However, when SUM1RM was expressed relative to body weight (SUM1RM.kg-1), significant (p < .01) correlation coefficients were obtained for each FT. Highest correlations with SUM1RM.kg-1 were observed for the VMPU, and this same test yielded the smallest percentage of zero scores. Principal components analysis of the CTs, normalized for body weight, and FTs yielded a factor on which both the FTs and CTs of strength loaded significantly. These data indicate that the five FTs, though invalid as measures of absolute strength and muscular endurance, manifest concurrent and construct validity as measures of weight-relative muscular strength.

Objectivity, reliability, and validity for a revised push-up test protocol

When executing a push-up an individual lowers the body to a down position and then raises it to an up position. A down position often used in recent years is a 90° angle at the elbows (90° push-up) as in FITNESSGRAM. Several researchers have found the interscorer objectivity and stability reliability for 90° push-up scores to be low. The purpose of this study was to (a) estimate the interscorer objectivity and stability reliability for scores from a revised push-up test protocol and (b) obtain evidence concerning the validity of interpretations based on revised push-up test scores. Interscorer objectivity was estimated for a score of one scorer and stability reliability was estimated for a score obtained on 1 day. Four studies were conducted. In the first study, pilot study, the revised push-up test protocol was developed and refined. In the second study, objectivity study, the push-up test was administered once to 49 female and 31 male college-aged students. Two scorers independently scored each student. Interscorer objectivity coefficients of·75 for women and ·88 for men were obtained. In the third study, objectivity and reliability study, the push-up test was administered on each of 2 days to 89 female and 63 male college-aged students. Two scorers independently scored each student on each day. Interscorer objectivity coefficients of ·97 and ·95 for women and ·98 and ·99 for men were obtained. Stability reliability coefficients of ·90 and ·93 for women and ·95 and ·95 for men were obtained. In the fourth study, validity study, validity was estimated using a logical approach, group difference approach, and criterion approach. There were 58 male and 48 female college students in the validity study. The revised push-up test protocol is very similar to protocols presently used and as expected the men scored significantly (p < .01) better than the women on the revised push-up test. The correlation between revised push-up scores and number of bench press executions with a percentage of the body weight was·80 for women and ·87 for men. The interscorer objectivity and stability reliability coefficients are very acceptable. Sufficient validity evidence was provided that the revised push-up scores relate to the amount of arm and shoulder girdle strength and endurance a person has to move the body weight.

Objectivity and Reliability of the 90° Push-Up Test

The purpose of this study was to determine the objectivity and stability reliability of the 90° push-up (90PU) test for elementary, high school (HS), and college-age students. One hundred fifty-six elementary and 70 HS students performed the 90PU test on 2 different days. Of those students, 49 elementary students were videotaped for the objectivity portion of the study. Eighty-four college students performed the 90PU on 2 different days, and each day they were scored by 2 to 4 judges. For the elementary girls and boys, the objectivity coefficients were .46 and .75, respectively, and the stability reliability coefficients ranged from .50 to .86 for the elementary and HS students. For the college students, the objectivity coefficients were between .16 and .91, with 6 of the 16 coefficients above .70. The reliability coefficients were between .22 and .87, with 5 of the 7 coefficients greater than .70. Several administration problems with the 90PU test were identified during the testing.

Estimating the Stability Reliability of a Score

At a recent conference devoted to the measurement of physical activity, a number of speakers reported stability reliability estimates using a Pearson correlation coefficient. Most of these correlation coefficients were calculated using test–retest data to estimate stability reliability. The number of days between the test and the retest varied from several days to many months. Using the Pearson correlation coefficient is incorrect and the large number of days between the test and the retest is also incorrect. Apparently instructors of measurement, statistics, and/or research techniques courses have not been teaching the correct information in their courses or some people are not enrolling in measurement, statistics, and/or research techniques courses. Addressed in this article are the correlation coefficient issue, the number of days between the test and the retest issue, and additional issues and recommendations.

Objectivity, Reliability, and Validity of the Bent-Knee Push-Up for College-Age Women

The revised push-up test has been found to have good validity but it produces many zero scores for women. Maybe there should be an alternative to the revised push-up test for college-age women. The purpose of this study was to determine the objectivity, reliability, and validity for the bent-knee push-up test (executed on hands and knees) for college-age women and to determine the relationship between the revised push-up test (executed on hands and toes) and bent-knee push-up test scores. College-age women (N = 87) participated in this study. The bent-knee push-up test was administered to all the participants the 1st day. Two raters were used to determine interscorer objectivity for approximately half of the participants. On the 2nd day, half the participants did the bent-knee push-up test again to determine stability reliability of the scores. The other half of the participants were administered the revised push-up test. On the 3rd day of testing, all participants were administered the bench press test using 40% of their body weight to determine the criterion validity for both of the push-up tests. The interscorer objectivity coefficient for the bent-knee push-up scores was .997. A stability reliability coefficient of .83 was obtained. The correlation between the bent-knee push-up and revised push-up scores was .75. The correlation between the bent-knee push-up and bench press scores was .67. The correlation between the revised push-up and bench press scores was .68. Both tests appear effective, however, the bent-knee test is probably more appropriate with lower strength level college-age women.