Edward M. Winter

Metrics of meaningfulness as opposed to sleights of significance

For some 80 years, statistical analyses have been predicated on the testing of hypotheses and evaluated by the probabilities of outcomes. This reflects Karl Popper’s principle of falsifiability that states: before something can be accepted, the opposite has to be shown to be untenable. By convention, two principal hypotheses are usually stated: a research hypothesis – i.e. “A influences B” – and a null hypothesis – i.e. “A has no influence on B”. It is the latter that is tested. Only when the null hypothesis is shown to be untenable can the research hypothesis be accepted. Usually, a probability of 0.05 is used as a cut-off. Probabilities equal to or less than 0.05 are considered to be statistically significant, whereas those greater than 0.05 are not. However, it is well established that the pass-orfail nature of hypothesis testing has serious, perhaps fundamental flaws (Cumming & Finch, 2001). Even Gosset (1876–1937) and Fisher (1890–1962) whose work on distributions provided the basis for our understanding recognised these flaws. Overpowered studies can produce outcomes that are “statistically significant” but that have little practical meaning and similarly, outcomes that are “not statistically significant” can be practically or clinically meaningful. Ambiguity of the word “significant” has been considered in a previous editorial (Winter, 2008). Among others, Jacob Cohen (1988) addressed the contradiction and proposed an alternative metric of meaningfulness, effect size. An effect size expresses a difference between groups or change within groups as a fraction of the variability between participants. Usually, this denominator is the standard deviation. Effect sizes can be evaluated as trivial (0–0.19), small (0.20–0.49), medium (0.50–0.79) and large (0.80 and greater) (Cohen, 1992). Similarly, confidence intervals of difference/change (Cumming & Finch, 2001) can evaluate outcomes on the basis of their inclusion of zero, i.e. no effect. These metrics allow investigators meaningfully to answer the twofold question, “does the intervention work, and how well?”. The intervention could be a therapy in medicine or a training programme in sport and physical activity. A second key question is, “how precise is our estimate of how well the intervention works?”. While there could be a large difference in mean performance times between groups after an intervention, a low sample size or large individual variation might still result in uncertainty that our intervention will work. Consequently, we need a metric that captures this imprecision: the confidence interval. The confidence interval represents a plausible range of values within which the true (but unknown) population value lies (Cumming, 2012). The greatest likelihood will arise from effects with narrow confidence intervals and therefore high precision. A third key question is, “is the intervention cost-effective?”. This is perhaps harder to answer but in both sport and physical activity and medically related settings, the question has to be considered. Another way to evaluate the effectiveness of an intervention is by way of the minimum clinically (or practically) important difference. This difference should be stated before a study commences and expresses the smallest change in the principal outcome measure that must occur if the intervention is to be considered effective. It is usually taken to be equivalent to an effect size of 0.20. Increasingly, journals no longer accept manuscripts whose outcomes are evaluated solely on the basis of probabilities. At its meeting on 14 January 2014, the Journal of Sports Sciences Editorial Board decided to do likewise. Authors must at least accompany conventional P values with metrics such as effect sizes, confidence intervals of difference/change and minimum clinically or practically important difference. This editorial announces the change. Our instructions for authors have been amended accordingly. Journal of Sports Sciences, 2014 Vol. 32, No. 10, 901–902, http://dx.doi.org/10.1080/02640414.2014.895118

You’ve told me what you have found, but you haven’t told me the so-what

It is just over two years since Professor Brian Whipp PhD DSc passed away on 20 October 2011. His contributions to the physiology of exercise were profound and appropriately, they have been enduring. His work on tolerance to exercise advanced not only our knowledge and understanding of underlying mechanisms but also continued long-standing traditions characterised by the work of A. V. Hill of ingenuity. Brian played lead roles in the development of breath-by-breath gas-analysis systems that today, tend to be taken for granted. Moreover, Brian’s personal qualities inspired many. He was fastidious, probing and exacting. He was also immensely supportive and many can count themselves fortunate to have benefitted from his tutelage. It is perhaps fitting then to mark the anniversary of Brian’s passing by recounting one of his many sayings. We have chosen the one that heads this editorial. In research, the process is driven by the research question. The question should be a good one. So how do we know if it is “good”? There are perhaps two metrics (Winter, 2011) against which “goodness” can be judged. Answering the question should (1) advance knowledge and understanding or (2) change practice. If we can do both, we really have done well. In the context of advancing knowledge, care has to be taken when claiming “newness”. This has been highlighted in previous editorials (Nevill, 2001; Winter, 2008). Moreover, preparations in the UK for the REF2014 must include statements about the “impact” that work has made. Principally although not necessarily, this is evaluated in the context either of how outcomes of research have changed or are likely to change practice. So, it is important to report not only what a study has found but also, the implications of the finding, or of course, findings. Outcomes should be stated clearly, simply and unequivocally. Hence, Brian’s “so-what?” We would do well to heed Brian’s advice.

Modelling the Influence of Age, Body Size and Sex on Maximum Oxygen Uptake in Older Humans

The purpose of this study was to describe the influence of body size and sex on the decline in maximum oxygen uptake (V̇O2,max) in older men and women. A stratified random sample of 152 men and 146 women, aged 55‐86 years, was drawn from the study population. Influence of age on V̇O2,max, independent of differences in body mass (BM) or fat‐free mass (FFM), was investigated using the following allometric model: V̇O2,max= BMb (or FFMb) exp(a + (c × age) + (d × sex)) [epsilon]. The model was linearised and parameters identified using standard multiple regression. The BM model explained 68.8% of the variance in V̇O2,max. The parameters (± s.e.e., standard error of the estimate) for lnBM (0.563 ± 0.070), age (‐0.0154 ± 0.0012), sex (0.242 ± 0.024) and the intercept (‐1.09 ± 0.32) were all significant (P < 0.001). The FFM model explained 69.3% of the variance in V̇O2,max, and the parameters (± s.e.e) lnFFM (0.772 ± 0.090), age (‐0.0159 ± 0.0012) and the intercept (‐1.57 ± 0.36) were significant (P < 0.001), while sex (0.077 +/− 0.038) was significant at P = 0.0497. Regardless of the model used, the age‐associated decline was similar, with a relative decline of 15% per decade (0.984 exp(age)) in V̇O2,max in older humans being estimated. The study has demonstrated that, for a randomly drawn sample, the age‐related loss in V̇O2,max is determined, in part, by the loss of fat‐free body mass. When this factor is accounted for, the loss of V̇O2,max across age is similar in older men and women.

Modelling the Influence of Fat‐Free Mass and Physical Activity on the Decline in Maximal Oxygen Uptake with Age in Older Humans

The purpose of this study was to use an allometric model (maximal oxygen uptake (V̇O2,max) = FFMb1 × PAb2 × exp(b0 + b3 age + b4 sex) ×ε) to determine the influence of fat‐free mass (FFM), physical activity (PA), sex and age on V̇O2,max in older men (n = 152) and women (n = 146) aged 55‐86 years. V̇O2,max was measured during a fatigue‐limited treadmill test, FFM was determined from skinfold thickness and physical activity by the Minnesota Leisure Time Physical Activity questionnaire. The model was linearised by taking the natural logarithm of V̇O2,max, FFM and physical activity. Variables were selected using multiple linear regression (P < 0.05). The sex variable was not significant (P = 0.062). The model explained 72.1% of the variance in V̇O2,max. Significant individual coefficients were incorporated into the model yielding the following expression: V̇O2,max= FFM0.971 × PA0.026 × exp(‐2.48 – 0.015age). Therefore, FFM and physical activity were significant factors contributing to the changes in V̇O2,max with age. In addition, controlling for FFM and physical activity abolished sex differences in V̇O2,max. The rate of decline in V̇O2,max (after accounting for FFM and physical activity) with age, was approximately 15% per decade.

Effects of a lifestyle intervention on endothelial function in men on long-term androgen deprivation therapy for prostate cancer

Background:Treatment of prostate cancer with androgen deprivation therapy (ADT) is associated with metabolic changes that have been linked to an increase in cardiovascular risk.Methods:This randomised controlled trial investigated the effects of a 12-week lifestyle intervention that included supervised exercise training and dietary advice on markers of cardiovascular risk in 50 men on long-term ADT recruited to an on-going study investigating the effects of such a lifestyle intervention on quality of life. Participants were randomly allocated to receive the intervention or usual care. Cardiovascular outcomes included endothelial function (flow-mediated dilatation (FMD) of the brachial artery), blood pressure, body composition and serum lipids. Additional outcomes included treadmill walk time and exercise and dietary behaviours. Outcomes were assessed before randomisation (baseline), and 6, 12 and 24 weeks after randomisation.Results:At 12 weeks, the difference in mean relative FMD was 2.2% (95% confidence interval (CI) 0.1–4.3, P=0.04) with an effect size of 0.60 (95% CI <0.01–1.18) favouring the intervention group. Improvements in skeletal muscle mass, treadmill walk time and exercise behaviour also occurred in the intervention group over that duration (P<0.05). At 24 weeks, only the difference in treadmill walk time was maintained.Conclusions:This study demonstrates that lifestyle changes can improve endothelial function in men on long-term ADT for prostate cancer. The implications for cardiovascular health need further investigation in larger studies over longer duration.

Use and misuse of the term “significant”

From time to time, the Journal includes editorials that are intended to provide soft-touch communications on matters of interest and perhaps importance to those whose interests are in sport and exercise science – or indeed, elsewhere. This inclusion considers the use and possible misuse of the term ‘‘significant’’ in three possible contexts: first, precision of measurement; second, statistical analyses; and third, meaningfulness. As regards precision of measurement and decimal places, stature for example, measured on a stadiometer that records to the nearest millimetre, can be expressed in centimetres to one decimal place or in metres to three decimal places. For instance, 181.4 cm. This means that the instrument can read to one part in 1814 – a precision of 0.05%. Similarly, in the context of cycle ergometry, it is tempting to report the result of an assessment of maximal intensity exercise as, say, 905.5 W. However, now we have a problem: the suggestion is that recording instrumentation can read to one part in 9055 (i.e. 0.01%). Considering the likely characteristics of the type of ergometer and accompanying sensors used, this is highly improbable and even rounding up to the nearest integer still implies unrealistically high precision of 0.1%. After all, test – retest differences in performance can be up to 10%. It can be argued that reporting values of maximum oxygen uptake in ml kg min such as 63.2 ml kg min (i.e. to one decimal place – three significant figures) is unwarranted; integers would suffice because even at ‘‘63’’, precision is still 1.6%. Reporting to two decimal places – four significant figures – is definitely unjustifiable. The use of an appropriate number of significant figures and hence decimal places has to be considered carefully because it indicates underlying precision of measurement. Successful authors can and do report appropriately. The word ‘‘significant’’ also occurs in the context of statistical analyses, in particular the outcomes of such analyses. In the case of means, it is not uncommon to see something like, ‘‘mean A was statistically significantly different compared with mean B (P5 0.05)’’. Much of this is redundant and besides, it is imprecise because no direction is indicated. In the part of the methods section of a manuscript that describes statistical analyses, the alpha value that signifies statistical significance is stated. Usually, but not always, this is 0.05. Having done so, means then become either greater or less than each other or they do not differ. So, as appropriate, our previous statement becomes ‘‘A was greater/less than B (P1⁄4 . . .)’’. This is an improvement because it is both clearer and shorter. The actual probability of the outcome should be reported; conveniently, this is stated routinely on commercial packages’ print outs. Of course, an error term should accompany each mean. Regrettably, it is not unknown for misguided authors to report P1⁄4 0.000; 50.001 yes, 0.000 no. The latter implies that there was no probability that the outcome could have occurred by chance alone. This, arguably, is impossible. Those of us of a certain age can remember having to go to probability tables in the backs of textbooks to seek out the likes of t, r, and F tables to determine whether, for given degrees of freedom, results were ‘‘significant’’. Similarly, necessary verification of the normality of distributions, homogeneity of variances, compound symmetry, and the like that such tables require (Nevill, 2000) – and that were often only assumed – can be undertaken easily. Because of the routine statement of actual probabilities of outcomes that now occurs, it is no longer necessary to seek out tables. Of course, such simple retention or rejection of the tested null hypothesis is giving way to confidence-interval and practicalor clinical-meaningfulness-of-outcome approaches – but this is not for debate here. This moves us on neatly to the final context of ‘‘significant’’: meaningfulness. In the discussion section of manuscripts we can have what amounts to a double whammy. For instance, the expression ‘‘this is a significant finding’’ is ambiguous. Is the ‘‘significance’’ statistical or does it imply something that is practically meaningful? To avoid confusion, words such as notable, meaningful or influential are preferable. They indicate the importance of the outcome and hence what was probably intended. Journal of Sports Sciences, March 2008; 26(5): 429 – 430

The Effectiveness of a Leg-Kicking Training Program on Performance and Physiological Measures of Competitive Swimmers

This study investigated the adaptations in leg muscle metabolism of swimmers following a six-week, leg-kicking swimming training program Fifteen male competitive swimmers were randomly assigned to an experimental group (E; n=8) and a control group (C; n=7). E swimmers performed normal leg-kicking training three times per week, whereas C swimmers performed reduced leg-kicking training (20% and 4% of weekly training distance, respectively). Before and after the training program, all swimmers performed a 200 m leg-kicking and a 400 m full-stroke freestyle time trial and a dry-land exercise test during which peak oxygen uptake, oxygen uptake at 60 W and exercise intensity at ventilatory threshold were measured. After training, there were improvements in leg-kicking time in 200 m (s; −6.0 ± 2.0%, p = 0.044), oxygen uptake at 60 Watts (L·min−1; −20.4 ± 3.0%, p = 0.035) and exercise intensity at ventilatory threshold (Watts; +28.0 ± 5.0%, p = 0.023) in E swimmers, whereas time in 400 m and peak oxygen uptake remained unchanged (p > 0.05). There were no changes in any of the measures for C swimmers (P > 0.05). These results suggest that normal leg-kicking swimming training positively affects the conditioning of the legs, but does not improve aerobic power during the dry-land, leg-kicking exercise test or middle-distance, full-stroke, swimming performance.

Terms and nomenclature

In 1963, B. K. Forscher had a letter published in Science entitled ‘‘Chaos in the brickyard’’, in which he expressed concerns about the proliferation of meaningless studies that did not adhere to the high ideas of accuracy and sound theory building of science. This resonates with our similar concerns about terms and nomenclature that interfere with advancements in sport and exercise science. The introduction in 1960 of the Système International d’Unités should have established correct applications of mechanical constructs to describe and quantify the performance of exercise. In spite of attempts from authors such as Adamson and Whitney (1971), Knuttgen (1978), Rodgers and Cavanagh (1984), and Winter and Fowler (2009) to ensure that such applications occur, there are many instances where they do not. For example, in the context of body size, distinctions between weight and mass are frequently not made. Mass is the amount of matter in a body and weight is the force exerted by that body that arises from gravitational attraction. Body weight of humans should be reported in newtons (N) yet frequently it is misreported in the unit of mass (kg). If you want to lose weight, go to the moon and you will weigh about a sixth of what you weigh on earth. However, it is probably mass you want to lose and while you might well do so on the long extraterrestrial journey, there are simpler ways to achieve that requirement. Similarly, the terms ‘‘work’’, ‘‘power’’, and ‘‘efficiency’’ are often misused. In humans, the expenditure of energy does not always result in motion. For instance, recruited muscles can be active isometrically. Examples are suspension and balance in gymnastics, the scrum in both codes of rugby, and fine motor tasks involved in archery and shooting. Even so, the expenditure of energy can be considerable; for instance, try performing a Maltese Cross in gymnastics. Similarly, activities of daily living require isometric muscle actions exemplified by the tightening of screw-tops on jars or holding objects during domestic tasks. When motion does occur, mechanical work is done and for nutritional and other reasons, quantification of the amount of work that is performed can be valuable. Of more use, however, is the rate at which work is done (i.e. power output). But for reasons outlined by Knudson (2009), ‘‘power’’ is probably the second most abused term in sport and exercise – the first is likely ‘‘efficiency’’ (Winter, 2009) – and there are still legions who genuflect at the altar of vague calculations of power and efficiency oblivious to the misapplications that are being made. In projectile activities such as vertical or horizontal jumping where the body is the projectile or in throwing an implement or striking a ball, the decisive factor in determining performance is the velocity of the projectile as appropriate, at take-off, release or departure. This velocity is exactly determined by the preceding impulse that is imparted to the projectile (i.e. the force–time integral). Newton’s second law that is written as the impulse–momentum relationship clearly demonstrates and encapsulates the relationship. Here ‘‘velocity’’ is appropriate because it encompasses both magnitude and direction. For activity of long duration where laudable attempts are made to identify limits of tolerance to exercise, for nearly 50 years the term ‘‘critical power’’ has been used to describe these limits (Monod & Scherer, 1965). Unfortunately, ‘‘critical power’’ has limited applicability; it is an inappropriate use of the term ‘‘power’’. It cannot be used with activities such as running or swimming where performance is quantified either as time (t) or speed (m s); neither is the unit of power (i.e. the watt). The term ‘‘critical power’’ cannot be used when isometric muscle activity occurs because no external mechanical work is done, and hence there cannot be power output. Even in activities such as cycling where it is possible to assess a mean external power output on a bicycle or ergometer, if pedalling rate is increased, ‘‘critical power’’ is adversely affected. The reason is simple: more energy is required to move the limbs, and hence less is available for useful external output. The distinction between these internal and external Journal of Sports Sciences, July 2011; 29(10): 999–1000

Validity of a Squash-Specific Test of Multiple-Sprint Ability

Wilkinson, M, McCord, A, and Winter, EM. Validity of a squash-specific test of multiple-sprint ability. J Strength Cond Res 24(12): 3381-3386, 2010-We examined the validity and reproducibility of a squash-specific multiple-sprint test. Eight male squash and 8 male soccer players performed Bakers 8 × 40-m sprints and a squash-specific-multiple-sprint test on separate days. The sum of individual sprint times in each test was recorded. Six squash and 6 soccer players repeated the tests 7 days later to assess reproducibility using intraclass correlation. In addition, 2 England Squash coaches independently ranked the squash players using knowledge of the player and recent performances in local leagues. Performance on the squash-specific (r = 0.97 and 0.90) and Bakers test (r = 0.95 and 0.83) was reproducible in squash and soccer players, respectively, and did not differ on Bakers test (mean ± SD 72.9 ± 3.9 and 72.9 ± 2.8 seconds for squash and soccer players, p = 0.969, effect size = 0.03). Squash players (232 ± 32 seconds) outperformed soccer players (264 ± 14 seconds) on the squash-specific test (p = 0.02, effect size = 1.39). Performance on Bakers and the squash-specific test were related in squash players (r = 0.98, p < 0.001) but not in soccer players (r = −0.08, p = 0.87). Squash-player rank correlated with performance on the squash-specific (ρ = 0.79, p = 0.02) but not the Bakers test (ρ = 0.55, p = 0.16). The squash-specific test discriminated between groups with similar non-sport-specific multiple-sprint ability and in squash players. In conjunction with the relationship between test performances, the results suggest that the squash-specific test is a valid and reproducible measure of multiple-sprint ability in squash players and could be used for assessing and tracking training-induced changes in multiple-sprint ability.

Effects of an acute bout of aerobic exercise on immediate and subsequent three-day food intake and energy expenditure in active and inactive pre-menopausal women taking oral contraceptives ☆

UNLABELLED This study examined the effects of an acute bout of exercise of low-intensity on food intake and energy expenditure over four days in women taking oral contraceptives. Twenty healthy, active (n = 10) and inactive (n = 10) pre-menopausal women taking oral contraceptives completed two conditions (exercise and control), in a randomised, crossover fashion. The exercise experimental day involved cycling for one hour at an intensity equivalent to 50% of maximum oxygen uptake and two hours of rest. The control condition comprised three hours of rest. Participants arrived at the laboratory fasted overnight; breakfast was standardised and an ad libitum pasta lunch was consumed on each experimental day. Participants kept a food diary to measure food intake and wore an Actiheart to measure energy expenditure for the remainder of the experimental days and over the subsequent 3 days. There was a condition effect for absolute energy intake (exercise vs. CONTROL 3363 ± 668 kJ vs. 3035 ± 752 kJ; p = 0.033, d = 0.49) and relative energy intake (exercise vs. CONTROL 2019 ± 746 kJ vs. 2710 ± 712 kJ; p <0.001, d = -1.00) at the ad libitum lunch. There were no significant differences in energy intake over the four days in active participants and there was a suppression of energy intake on the first day after the exercise experimental day compared with the same day of the control condition in inactive participants (mean difference = -1974 kJ; 95% CI -1048 to -2900 kJ, p = 0.002, d = -0.89). There was a group effect (p = 0.001, d = 1.63) for free-living energy expenditure, indicating that active participants expended more energy than inactive participants during this period. However, there were no compensatory changes in daily physical activity energy expenditure. These results support the use of low-intensity aerobic exercise as a method to induce a short-term negative energy balance in inactive women taking oral contraceptives.