Michelle A. Giuffrida

Type II error and statistical power in reports of small animal clinical trials

OBJECTIVE To describe reporting of key methodological elements associated with type II error in published reports of small animal randomized controlled trials (RCTs) and to determine the statistical power in a subset of RCTs with negative results. DESIGN Descriptive literature survey. SAMPLE Reports of parallel-group clinical RCTs published in 11 English-language veterinary journals from 2005 to 2012. PROCEDURES Predefined criteria were used to identify trial primary outcomes and classify results as negative or positive. Details of sample size determination and use of confidence intervals in results reporting were recorded. For each 2-group RCT with negative results, the statistical power to detect 25% and 50% relative differences in outcome was calculated. RESULTS Of 238 RCTs, 42 (18%) stated a primary outcome, 52 (22%) reported a sample size calculation, and 18 (9%) included a confidence interval around the observed treatment effect. Reports of only 2 (0.8%) RCTs included all 3 elements. Among 103 two-group RCTs with negative results, only 14 (14%) and 40 (39%) were sufficiently powered (β < 0.20) to detect 25% and 50% relative differences in outcome between treatments, respectively. CONCLUSIONS AND CLINICAL RELEVANCE The present survey found that small animal RCTs with negative results were often underpowered to detect moderate-to-large effect sizes between study groups. Information needed for critical appraisal was missing from most reports. The potential for clinicians to base treatment decisions on inappropriate interpretations of RCTs was worrisome. Design and reporting of small animal RCTs must be improved.

Blinding terminology used in reports of randomized controlled trials involving dogs and cats

OBJECTIVE To review blinding terminology used in published reports of veterinary clinical randomized controlled trials (RCTs) and to determine how practicing veterinarians interpret blinding terminology. DESIGN Retrospective literature review and prospective veterinarian survey. SAMPLE 195 parallel-group clinical RCTs published from June 2004 to June 2010 in 11 peer-reviewed journals; 21 practicing veterinarians at a university-based small animal teaching hospital. PROCEDURES Journals were hand searched to identify eligible reports. Details concerning trial methodology were recorded. Veterinarians provided information regarding position, experience, and personal interpretation of blinding terminology via an anonymous questionnaire. RESULTS Blinding was reported or inferred in 131 reports of RCTs, yet complete descriptions of who was blinded were present in only 42 (32.1%) reports. Studies for which blinding was reported with the terms single or double blinded were less likely to contain clear descriptions of the role of blinded study personnel, compared with studies reported as blinded or in which blinding was inferred through trial methodology. Veterinarians did not agree on how to interpret the terms single, double, and triple blinded when reading the report of an RCT. CONCLUSIONS AND CLINICAL RELEVANCE Blinding was commonly used as a means of reducing bias associated with collection and interpretation of data in reports of veterinary RCTs. However, most reports of blinding methodology were incomplete and there was no consistency in how blinding terminology was used by authors or interpreted by veterinarians. Ambiguous reporting hinders the ability of practitioners to assess the validity of trial results and make informed decisions about applying study findings to their patient populations.

In veterinary trials reporting and communication regarding randomisation procedures is suboptimal

To evaluate randomisation mechanisms in the veterinary literature, all trials defined as ‘randomised’ were extracted from five leading veterinary journals for the year 2013. Three blinded investigators evaluated (1) if the random sequence generation was actually non-random, and (2) whether method (CONSORT item 8A) and (3) type of randomisation (CONSORT item 8B) were reported. Trialists were contacted via email to establish (1) willingness to respond to questions on randomisation procedures, (2) whether reporting of randomisation improved following a suggestion to use the CONSORT 2010 guideline. Seven per cent ((95 per cent CI 2 to 12 per cent); 8/114) of the trials defined as ‘randomised’ explicitly used methods that are considered non-random. Almost half of the trials (49 per cent (40 to 59 per cent); 52/106) did not report any mechanism of randomisation. Only 13 trials (12.3 per cent (6 to 19 per cent); 13/106) reported both items. 39 of 114 (34.2 per cent) trialists contacted were willing to respond to further questions on randomisation mechanisms; 4 (3.5 per cent) trialists were unwilling and 71 (62.3 per cent) trialists did not respond. Email correspondence resulted in a mean clarification of 0.7 items (95 per cent CI 0.4 to 1.0) for the 15 trials for trialists that replied. Improved adherence to CONSORT guidelines and trialists communication is imperative to increase the quality of published evidence in veterinary medicine and to reduce research waste.To evaluate randomisation mechanisms in the veterinary literature, all trials defined as ‘randomised’ were extracted from five leading veterinary journals for the year 2013. Three blinded investigators evaluated (1) if the random sequence generation was actually non-random, and (2) whether method (CONSORT item 8A) and (3) type of randomisation (CONSORT item 8B) were reported. Trialists were contacted via email to establish (1) willingness to respond to questions on randomisation procedures, (2) whether reporting of randomisation improved following a suggestion to use the CONSORT 2010 guideline. Seven per cent ((95 per cent CI 2 to 12 per cent); 8/114) of the trials defined as ‘randomised’ explicitly used methods that are considered non-random. Almost half of the trials (49 per cent (40 to 59 per cent); 52/106) did not report any mechanism of randomisation. Only 13 trials (12.3 per cent (6 to 19 per cent); 13/106) reported both items. 39 of 114 (34.2 per cent) trialists contacted were willing to respond to further questions on randomisation mechanisms; 4 (3.5 per cent) trialists were unwilling and 71 (62.3 per cent) trialists did not respond. Email correspondence resulted in a mean clarification of 0.7 items (95 per cent CI 0.4 to 1.0) for the 15 trials for trialists that replied. Improved adherence to CONSORT guidelines and trialists communication is imperative to increase the quality of published evidence in veterinary medicine and to reduce research waste.

Short-term clinical outcome of laparoscopic liver biopsy in dogs: 106 cases (2003-2013)

OBJECTIVE To describe the operative technique, complications, and conversion rates for laparoscopic liver biopsy (LLB) in dogs and evaluate short-term clinical outcome for dogs that underwent the procedure. DESIGN Retrospective case series. ANIMALS 106 client-owned dogs. PROCEDURES Medical records were reviewed to identify dogs that underwent an LLB with a single-port or multiport technique at either of 2 veterinary teaching hospitals from August 2003 to September 2013. Demographic and laboratory data, preoperative administration of fresh frozen plasma, procedural and diagnostic information, intraoperative complications, and survival to discharge were recorded. The LLB specimens were obtained with 5-mm laparoscopic biopsy cup forceps and a grasp-and-twist technique. RESULTS Prior to surgery, 25 of 94 (27%) dogs had coagulopathy (prothrombin time or partial thromboplastin time greater than the facility reference ranges, regardless of platelet count). Twenty-one dogs were thrombocytopenic, 14 had ascites, and 14 received fresh frozen plasma transfusion before surgery. In all cases, biopsy samples collected were of sufficient size and quality for histopathologic evaluation. Two dogs required conversion to an open laparotomy because of splenic laceration during initial port placement. One hundred one of 106 dogs survived to discharge; 5 were euthanized during hospitalization owing to progression of liver disease and poor prognosis. CONCLUSIONS AND CLINICAL RELEVANCE Single-port and multiport LLB were found to be effective, minimally invasive diagnostic techniques with a low rate of complications. Results suggested LLB can be safely used in dogs with underlying coagulopathies and advanced liver disease.

Defining the primary research question in veterinary clinical studies.

A thoughtful, clearly defined research question should be the foundation of any clinical trial or research study. The research question helps determine key study methods, and defining a specific research question helps avoid problems with inadequate sample size, inappropriate design, or multiple statistical comparisons. Rationales and strategies for formulating research questions and using them to define study protocols are discussed, with a focus on application in clinical trials.

Use of routine histopathology and factor VIII-related antigen/von Willebrand factor immunohistochemistry to differentiate primary hemangiosarcoma of bone from telangiectatic osteosarcoma in 54 dogs.

Hemangiosarcoma (HSA) of bone and telangiectatic osteosarcoma (tOSA) can appear similar histologically, but differ in histogenesis (malignant endothelial cells versus osteoblasts), and may warrant different treatments. Immunohistochemistry (IHC) for endothelial cell marker factor VIII-related antigen/von Willebrand factor (FVIII-RAg/vWF) is a well-documented ancillary test to confirm HSA diagnoses in soft tissues, but its use in osseous HSA is rarely described. Archived samples of 54 primary appendicular bone tumours previously diagnosed as HSA or tOSA were evaluated using combination routine histopathology (RHP) and IHC. Approximately 20% of tumours were reclassified on the basis of FVIII-RAg/vWF immunoreactivity, typically from an original diagnosis of tOSA to a reclassified diagnosis of HSA. No sample with tumour osteoid clearly identified on RHP was immunopositive for FVIII-RAg/vWF. RHP alone was specific but not sensitive for diagnosis of HSA, compared with combination RHP and IHC. Routine histopathological evaluation in combination with FVIII-RAg/vWF IHC can help differentiate canine primary appendicular HSA from tOSA.

Psychometric properties of the Canine Symptom Assessment Scale, a multidimensional owner-reported questionnaire instrument for assessment of physical symptoms in dogs with solid tumors

OBJECTIVE To describe development and initial psychometric testing of the Canine Symptom Assessment Scale (CSAS), a multidimensional owner-reported questionnaire instrument, in a population of dogs with solid tumors enrolled in clinical trials. DESIGN Questionnaire development and validation study. ANIMALS 238 client-owned dogs with solid tumors. PROCEDURES A 14-symptom questionnaire was developed. Symptoms were defined as subjective physical disturbances dogs experienced during the course of daily living as assessed through proxy reports of pet owners. For each symptom, owners reported frequency and severity of the symptom and extent of distress caused by the symptom for the dog and the owner. Questionnaire content, symptom prevalence and dimensionality, internal consistency, and factor structure were examined. Construct and criterion validity were examined via comparison with the Canine Brief Pain Inventory (CBPI). RESULTS Symptom prevalence was high, with pain and lack of energy reported in most dogs. Severity, versus frequency, was most highly correlated with both dog and owner distress. Two symptoms were removed from consideration because of poor performance. Analysis of the remaining 12 symptoms revealed that they could be grouped into 3 factors: malaise, anxiety, and digestive upset. The CSAS factor and total scores demonstrated predictable relationships with quality of life and pain scores as measured by the CBPI, including a significant association between increasing symptom burden and decreasing quality of life. The Cronbach α for the CSAS was 0.77. CONCLUSIONS AND CLINICAL RELEVANCE The 12-item CSAS was a psychometrically sound owner-reported instrument for assessment of symptom frequency and characteristics in client-owned dogs with solid tumors. Potential applications include clinical research and practice settings.

An Observational Study of Abstracts Presented at the American College of Veterinary Surgeon Annual Meetings (2001-2008) and Their Subsequent Full-Text Publication.

OBJECTIVE To determine the frequency of abstracts presented at American College of Veterinary Surgeons (ACVS) meetings from 2001 to 2008 that were published as complete articles, to identify abstract characteristics associated with final full-text publication, and to examine consistency of information between abstracts and final full-text publications. STUDY DESIGN Observational bibliographic study. METHODS Abstracts were retrieved from published proceedings. Published articles were retrieved from bibliographic databases. Features of abstract and article authorship, design, and content were recorded. Regression analysis identified abstract features associated with article publication, and evaluated consistency between abstracts and final publications. RESULTS Seven hundred eighty-two of 1078 (73%) abstracts were published as complete articles. Median time to publication was 1 year; 90% were published within 3 years. Abstracts originating from academic institutions were published more often than abstracts from practice or industry sites (odds ratio 2.61, 95% confidence interval 1.68-4.05). Compared to their conference abstracts, 49% of articles contained major inconsistences including changes in study design, interventions, outcomes, sample size, and results. For each year elapsed between presentation and publication, the odds of major inconsistency increased 2.4 times (odds ratio 2.36, 95% confidence interval 1.57-3.55) for retrospective studies and 1.4 times (odds ratio 1.35, 95% confidence interval 1.17-1.56) for other study designs. Changes in study title and authorship were frequent, particularly in publications that contained major inconsistencies. CONCLUSION ACVS abstracts were promptly and reliably published, but final full-text publications often differed substantially from the original abstracts.

Primary splenic torsion in dogs: 102 cases (1992–2014)

OBJECTIVE To determine the percentage of dogs surviving to hospital discharge and identify factors associated with death prior to hospital discharge among dogs undergoing surgery because of primary splenic torsion (PST). DESIGN Retrospective case series. ANIMALS 102 client-owned dogs. PROCEDURES Medical records of dogs with a confirmed diagnosis of PST that underwent surgery between August 1992 and May 2014 were reviewed. History, signalment, results of physical examination and preoperative bloodwork, method of splenectomy, concurrent surgical procedures, perioperative complications, duration of hospital stay, splenic histopathologic findings, and details of follow-up were recorded. Best-fit multivariate logistic regression was performed to identify perioperative factors associated with survival to hospital discharge. RESULTS 93 of the 102 (91.2%) dogs survived to hospital discharge. German Shepherd Dogs (24/102 [23.5%]), Great Danes (15/102 [14.7%]), and English Bulldogs (12/102 [11.8%]) accounted for 50% of cases. Risk factors significantly associated with death prior to hospital discharge included septic peritonitis at initial examination (OR, 32.4; 95% confidence interval [CI], 2.1 to 502.0), intraoperative hemorrhage (OR, 22.6; 95% CI, 1.8 to 289.8), and postoperative development of respiratory distress (OR, 35.7; 95% CI, 2.7 to 466.0). Histopathologic evidence of splenic neoplasia was not found in any case. CONCLUSIONS AND CLINICAL RELEVANCE Results suggested that the prognosis for dogs undergoing splenectomy because of PST was favorable. Several risk factors for death prior to discharge were identified, including preexisting septic peritonitis, intraoperative hemorrhage, and postoperative development of respiratory distress.

Development and psychometric testing of the Canine Owner-Reported Quality of Life questionnaire, an instrument designed to measure quality of life in dogs with cancer

OBJECTIVE To describe development and initial psychometric testing of an owner-reported questionnaire designed to standardize measurement of general quality of life (QOL) in dogs with cancer. DESIGN Key-informant interviews, questionnaire development, and field trial. SAMPLE Owners of 25 dogs with cancer for item development and pretesting and owners of 90 dogs with cancer for reliability and validity testing. PROCEDURES Standard methods for development and testing of questionnaire instruments intended to measure subjective states were used. Items were generated, selected, scaled, and pretested for content, meaning, and readability. Response items were evaluated with exploratory factor analysis and by assessing internal consistency (Cronbach α) and convergence with global QOL as determined with a visual analog scale. Preliminary tests of stability and responsiveness were performed. RESULTS The final questionnaire-which was named the Canine Owner-Reported Quality of Life (CORQ) questionnaire-contained 17 items related to observable behaviors commonly used by owners to evaluate QOL in their dogs. Several items pertaining to physical symptoms performed poorly and were omitted. The 17 items were assigned to 4 factors-vitality, companionship, pain, and mobility-on the basis of the items they contained. The CORQ questionnaire and its factors had high internal consistency (Cronbach α = 0.68 to 0.90) and moderate to strong correlations (r = 0.49 to 0.71) with global QOL as measured on a visual analog scale. Preliminary testing indicated good test-retest reliability and responsiveness to improvements in overall QOL. CONCLUSIONS AND CLINICAL RELEVANCE The CORQ questionnaire was a valid, reliable owner-reported questionnaire that measured general QOL in dogs with cancer and showed promise as a clinical trial outcome measure for quantifying changes in individual dog QOL occurring in response to cancer treatment and progression.