John Ludbrook

MULTIPLE COMPARISON PROCEDURES UPDATED

1. A common statistical flaw in articles submitted to or published in biomedical research journals is to test multiple null hypotheses that originate from the results of a single experiment without correcting for the inflated risk of type 1 error (false positive statistical inference) that results from this. Multiple comparison procedures (MCP) are designed to minimize this risk. The present review focuses on pairwise contrasts, the most common sort of multiple comparisons made by biomedical investigators.

Statistical Techniques For Comparing Measurers And Methods Of Measurement: A Critical Review

1. Clinical and experimental pharmacologists and physiologists often wish to compare two methods of measurement, or two measurers.

Why Permutation Tests are Superior to t and F Tests in Biomedical Research

Abstract A survey of 252 prospective, comparative studies reported in five, frequently cited biomedical journals revealed that experimental groups were constructed by randomization in 96% of cases and by random sampling in only 4%. The median group sizes ranged from 4 to 12. In the randomized studies in which measurements were made on a continuous scale, comparisons of location were made by t or F tests in 84% of cases, and by nonparametric, rank-order, tests in the remainder. Because randomization rather than random sampling is the norm in biomedical research and because group sizes are usually small, exact permutation or randomization tests for differences in location should be preferred to t or F tests.

SPECIAL ARTICLE COMPARING METHODS OF MEASUREMENT

1. The purpose of comparing two methods of measurement of a continuous biological variable is to uncover systematic differences not to point to similarities.

Definition of ambulatory blood pressure targets for diagnosis and treatment of hypertension in relation to clinic blood pressure: prospective cohort study.

Background Twenty-four hour ambulatory blood pressure thresholds have been defined for the diagnosis of mild hypertension but not for its treatment or for other blood pressure thresholds used in the diagnosis of moderate to severe hypertension. We aimed to derive age and sex related ambulatory blood pressure equivalents to clinic blood pressure thresholds for diagnosis and treatment of hypertension. Methods We collated 24 hour ambulatory blood pressure data, recorded with validated devices, from 11 centres across six Australian states (n=8575). We used least product regression to assess the relation between these measurements and clinic blood pressure measured by trained staff and in a smaller cohort by doctors (n=1693). Results Mean age of participants was 56 years (SD 15) with mean body mass index 28.9 (5.5) and mean clinic systolic/diastolic blood pressure 142/82 mm Hg (19/12); 4626 (54%) were women. Average clinic measurements by trained staff were 6/3 mm Hg higher than daytime ambulatory blood pressure and 10/5 mm Hg higher than 24 hour blood pressure, but 9/7 mm Hg lower than clinic values measured by doctors. Daytime ambulatory equivalents derived from trained staff clinic measurements were 4/3 mm Hg less than the 140/90 mm Hg clinic threshold (lower limit of grade 1 hypertension), 2/2 mm Hg less than the 130/80 mm Hg threshold (target upper limit for patients with associated conditions), and 1/1 mm Hg less than the 125/75 mm Hg threshold. Equivalents were 1/2 mm Hg lower for women and 3/1 mm Hg lower in older people compared with the combined group. Conclusions Our study provides daytime ambulatory blood pressure thresholds that are slightly lower than equivalent clinic values. Clinic blood pressure measurements taken by doctors were considerably higher than those taken by trained staff and therefore gave inappropriate estimates of ambulatory thresholds. These results provide a framework for the diagnosis and management of hypertension using ambulatory blood pressure values.

Confidence in Altman–Bland plots: A critical review of the method of differences

1. Altman and Bland argue that the virtue of plotting differences against averages in method‐comparison studies is that 95% confidence limits for the differences can be constructed. These allow authors and readers to judge whether one method of measurement could be substituted for another.

Different outcomes of the Wilcoxon-Mann-Whitney test from different statistics packages

Abstract The Wilcoxon—Mann—Whitney test enjoys great popularity among scientists comparing two groups of observations, especially when measurements made on a continuous scale are non-normally distributed. Triggered by different results for the procedure from two statistics programs, we compared the outcomes from 11 PC-based statistics packages. The findings were that the delivered p values ranged from significant to nonsignificant at the 5% level, depending on whether a large-sample approximation or an exact permutation form of the test was used and, in the former case, whether or not a correction for continuity was used and whether or not a correction for ties was made. Some packages also produced pseudo-exact p values, based on the null distribution under the assumption of no ties. A further crucial point is that the variant of the algorithm used for computation by the packages is rarely indicated in the output or documented in the Help facility and the manuals. We conclude that the only accurate form of the Wilcoxon—Mann—Whitney procedure is one in which the exact permutation null distribution is compiled for the actual data.

Linear regression analysis for comparing two measurers or methods of measurement: But which regression?

1. There are two reasons for wanting to compare measurers or methods of measurement. One is to calibrate one method or measurer against another; the other is to detect bias. Fixed bias is present when one method gives higher (or lower) values across the whole range of measurement. Proportional bias is present when one method gives values that diverge progressively from those of the other.

Cold Hypersensitivity in Raynaud's Phenomenon

We have sought evidence that local cooling enhances the adrenergic neurotransmitter mechanism in resistance vessels in the hands of subjects with Raynauds phenomenon. The method used was to examine the magnitude and time-course of the reflex vasoconstriction in the hand which follows application of ice to the neck. One hand of each subject was kept at 26°C and the other at 36°C, while the general thermal environment was varied. Enhancement was found in three groups of subjects: those with idiopathic, early age-onset, Raynauds disease; those with Raynauds phenomenon of late age-onset; and those with systemic scleroderma. It was absent in normal subjects. The effect could not be accounted for by structural abnormalities of resistance vessels, blood abnormalities, nor by interference with norepinephrine reuptake or dissipation at the neuro-effector junction.

Intracisternal naloxone and cardiac nerve blockade prevent vasodilatation during simulated haemorrhage in awake rabbits.

1. Acute haemorrhage was simulated in five unanaesthetized rabbits, by inflating a cuff on the inferior vena cava so that cardiac output fell by 8.3% of its resting level per minute. Simulated haemorrhage was performed after sham treatment, after graded doses of intravenous and intracisternal naloxone, and after cardiac nerve blockade with intrapericardial procaine. 2. After sham treatment, the haemodynamic response to simulated haemorrhage was biphasic. During the first phase, systemic vascular conductance fell steadily, heart rate rose steadily, and arterial pressure fell only slightly. A second decompensatory phase began abruptly when cardiac output had fallen to approximately 55% of its resting level. Vascular conductance rose steeply, heart rate fell slowly, and arterial pressure fell precipitately. 3. Treatment with naloxone (intravenous, 0.04‐0.4 mg kg‐1; intracisternal, 0.2‐2 micrograms kg‐1) did not affect either phase of the haemodynamic response to simulated haemorrhage. 4. After treatment with larger doses of naloxone (intravenous, 4‐8 mg kg‐1; intracisternal, 4‐69 micrograms kg‐1), the first phase was unaffected, but the second phase no longer occurred. Throughout simulated haemorrhage, systemic vascular conductance fell steadily, heart rate rose, and arterial pressure was well maintained. The dose of intracisternal naloxone which prevented the second phase was 90‐900 times less than the corresponding intravenous dose. The second phase was also prevented by cardiac nerve blockade. 5. We conclude that an endogenous opiate mechanism is responsible for the haemodynamic decompensation that occurs when cardiac output falls to a critical level. The mechanism is located within the central nervous system. It is triggered by a signal from the heart.