Howard P. Davis

The effect of trial size on statistical power.

Many research studies produce results that falsely support a null hypothesis due to a lack of statistical power. The purpose of this research was to demonstrate selected relationships between single subject (SS) and group analyses and the importance of data reliability (trial size) on results. A computer model was developed and used in conjunction with Monte Carlo procedures to study the effects of sample size (subjects and trials), within- and between-subject variability, and subject performance strategies on selected statistical evaluation procedures. The inherent advantages of the approach are control and replication. Selected results are presented in this paper. Group analyses on subjects using similar performance strategies identified 10, 5, and 3 trials for sample sizes of 5, 10, and 20, respectively, as necessary to achieve statistical power values greater than 90% for effect sizes equal to one standard deviation of the condition distribution. SS analyses produced results exhibiting considerably less power than the group results for corresponding trial sizes, indicating how much more difficult it is to detect significant differences using a SS design. These results should be of concern to all investigators especially when interpreting nonsignificant findings.

The effect of trial size and variability on statistical power

A computer model was developed, validated, and used in conjunction with Monte Carlo procedures to study the effects of sample size (subjects and trials), mean differences, and subject variability on statistical power. Also examined were the differences between single subject (SS) and group results. Mean differences were varied from 1/4 to 4 times the distribution SD resulting in improved power values. Mean group F results ranged from 63.6% to 100% while SS results were poorer, especially for the smaller mean differences (16.8%-100%). Subject variability was examined for a Simple model and two Complex (MOD1 and MOD4) models. MOD1 produced group results similar to the corresponding Simple model with an overall mean of 78.2% and a Complex/Simple (C/S) ratio of 0.99. The more variable model (MOD4) produced fewer significant results (52.9%) and a lesser C/S ratio (0.82). The SS results were more dramatic. The percentages of significant values were less (38.1% and 33.9%) and the C/S ratios favored the Complex models (1.48 and 3.17). Both sample size and trial size had a major impact on the results. In summary, these results provide additional insight into the interactive effects and importance of the factors investigated, especially in the area of SS experiments.

Validity of lower extremity strength and power utilizing a new closed chain dynamometer

PURPOSE The purpose of this study was to compare selected variables measured on a traditional isokinetic dynamometer (Cybex II) with a new lower extremity, closed chain dynamometer (Omnikinetic, OmK). METHODS Twelve subjects (6 male, 6 female, age = 28+/-5 yr, mean +/- SD) performed Cybex II knee flexion and extension at 1.05, 3.14, and 5.23 rad x s(-1). A maximal effort of 10 repetitions of lower extremity concentric extension and eccentric flexion at 36% of subjects 1-RM was performed on the OmK. Crank power and joint (ankle, knee, and hip) kinetics were recorded as a mean of 10 repetitions. RESULTS t-Tests revealed right versus left leg differences (P < 0.05) for Cybex II peak torque flexion at 5.23 rad x s(-1), and OmK knee and hip peak power and hip root mean square power (RMS) power. Cybex peak knee torques were related (Pearson r values 0.78-0.92, P < 0.01) to OmK peak knee torques. Cybex average power was related to OmK knee power (Pearson r values 0.71-0.96, P < 0.01) and OmK crank power (r = 0.62-0.94, P < .01). Correlations tended to be stronger comparing the OmK with the fastest (5.23 rad x s(-1)) Cybex II speed. CONCLUSIONS These results suggest that the OmK knee and crank kinetic data are comparable to Cybex It isokinetic dynamometry. The ability to evaluate lower extremity joint exercise at a subjects maximal movement speed, in addition to the use of a closed-chain, multi-joint motion, may allow for the OmK to provide a more global evaluation of lower extremity kinetics during seated concentric-extension, eccentric-flexion exercise.

A new bilateral closed chain assessment technique: methods and error analysis.

PURPOSE The purpose of this paper is to present the Omnikinetic methodology for clinical evaluation of lower extremity function, to characterize its sensitivity to errors, and to present typical data for an assessment protocol. METHODS A 5-bar, 2-degree of freedom linkage was used to model the geometry of the crank, pedal, and lower extremity. Two-degree force transducers at the pedal were used to calculate center of pressure and force applied at the foot. A Newton-Euler inverse dynamic model was used to calculate net joint torques and powers bilaterally of the ankle, knee, and hip. Ten subjects performed a high velocity evaluation protocol which served as the control. Error sensitivity was determined by adding instrumental error, hip translation, and segmental length errors to the collected data and comparing the outcome to the control. RESULTS All variables associated with instrumental error had mean errors under 4%. Mean errors associated with violations of the fixed hip assumption were under 15% for all variables. Mean errors associated with anthropometric measures were divided into two types: relative error (overall length unchanged, ratios of segments changed) and absolute (overall length changed, ratios of segment lengths unchanged). Relative anthropometric mean errors were under 5%. Absolute anthropometric mean errors were under 12%. CONCLUSION The Omnikinetic is a new tool for bilateral lower extremity evaluation that enables the whole lower extremity to be evaluated at the joint level. Instrumental accuracy was excellent. The instrument was most sensitive to violations of the fixed hip position assumption over the last 20 degrees of knee extension.

Effects of position and speed on joint torques and knee shear after ACL injury.

PURPOSE The purpose of this study was to assess the effect of joint position and movement speed on hip (HT) and knee (KT) extensor torque, HT/KT ratio (HKR), and anterior tibial shear (S) in presurgical-ACL deficient (PRE; N = 15), post-ACL reconstructed (POST; N = 15), and uninjured controls (CON; N = 15). METHODS Measurements were recorded on a semirecumbent variable resistance, closed-chain dynamometer. Tests were conducted at 1 and 1.5 Hz and maximum speed at 33% and 50% 1RM. HT, KT, and S were recorded during the extension phase of the cycle (85 degrees -25 degrees of knee flexion). RESULTS KT was greatest when the knee was more flexed, whereas HT dominated when it was more extended. This suggests that the hamstrings, as a component of the hip extensors, may generate considerable propulsive force during knee extension, which may help counter anterior tibial shear. S increased whereas KT decreased, suggesting that the quadriceps continue to generate shear force despite the decreasing mechanical advantage producing KT. Increasing knee extension speed significantly decreased S in the POST and CON groups. The correlations between KT and S changed from significantly negative to significantly positive as the knee extended from 85 degrees to 25 degrees in the POST and CON groups. This may be related to the orientation of the patellar tendon, relative to the tibial longitudinal axis that shifts from a posterior to anterior direction, as the knee extends. CONCLUSIONS Joint position and movement speeds affect the magnitude of hip and knee torques and anterior tibial shear. Reducing the magnitude of S during heavy loads may be a normal phenomenon, and POST surgical subjects may retain or regain this function by 1 yr after surgery.

Work in progress: Help in finding evaluation instruments for engineering education innovations

This paper describes the first-year progress of a TUES Type 2 project funded by the National Science Foundation (NSF) that supports scholarly innovation in engineering education. The project, known as the Appraisal System for Superior Engineering Education Evaluation-instrument Sharing and Scholarship (ASSESS), builds on a precursor NSF CCLI Type 1 project, the Inventory of Evaluation Tools for Engineering Education Projects. The paper reports on the process and framework for developing ASSESS as a sustainable library of superior evaluation instruments that promote scholarly innovation in engineering education. The online database supports information storage, retrieval, and input as well as expert review and user feedback. Adoption research currently being conducted will determine factors that influence broad utilization of the database. Feedback is invited from the community to produce a user-friendly website that enables engineering educators to locate and implement tools that can be used to credibly evaluate the impacts of engineering education innovations.

Search engine for engineering education assessment instruments

Adoption research has guided the development of a classification and rating system for assessment instruments valuable to the engineering education community. The ASSESS website enables users to search for instruments that measure outcomes of interest and that possess desired characteristics. The website classifies instruments by outcomes assessed, technical features, instrument format, and administration parameters. Instrument descriptions and ratings are displayed in forms useful to both experienced and inexperienced users seeking instruments for specific needs. The user community contributes to ASSESS by rating instruments and suggesting others for development or inclusion. ASSESS developers invite user input to make ASSESS highly valuable to the engineering education community.

Workshop: Assessing professional skills for ABET

The Integrated Design Engineering Assessment and Learning System (IDEALS) is a set of assessment instruments that focus on aspects of professional development such as leadership, ethics, project management and communication skills within the context of capstone engineering design projects. These instruments have been piloted across a broad spectrum of diverse engineering programs and are available in a web format that facilitates data collection from students and feedback from instructional staff. This knowledge has been synthesized into a set of engaging and transferable modules that include learning objectives, supporting resource materials, preparatory tasks, lesson plans for faculty, and recommendations for timely follow-up using the online assessments. In addition, the instruments enhance student development through both formative and summative feedback and can be used to provide evidence for demonstrating the achievement of ABET student outcomes and program specific criteria.

Methodology for Selection, Sequencing, and Deployment of Activities in a Capstone Design Course Using the TIDEE Web-Based Assessment System

Assessment of design process, design products, team process, and professional practice are natural fits in an engineering capstone design course. In order for instructors and students to fully experience the value of capstone course assessment activities, the activities must not only be carefully developed but must also be deployed in an appropriate manner. Course designers must choose an optimal set of assignments based on local needs, while balancing time intensive design project activities with professional growth experiences. Instructors must facilitate the complete cycle of usage of a single assignment in order to ensure that the value is understood before and after completion of the assessment. This paper introduces guidelines for achieving effectiveness in selecting, timing, and sequencing assessment activities, preparing for activity deployment, and implementing a facilitation plan. Additionally this paper reports on the feedback from students and faculty using the system that highlights the importance of naturalistically integrating assessment.Copyright

Work in progress - web-based implementation of capstone engineering design assessments

Assessment of student performance in capstone engineering design is vital to student learning and to program accreditation and improvement. Performances important to capstone design courses include studentspsila growth in personal capacity for design, development of teamwork, development of design processes, and delivery of solution assets. The Transferable Integrated Design Engineering Education (TIDEE) consortium is creating and testing a capstone design assessment system to measure these student performances and provide rapid personalized feedback. Broad adoption of this assessment system requires ease of use by both students and instructors. This paper describes design features of a Web-based implementation of the TIDEE capstone design assessment system.