Claes Enøe

Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown

The performance of a new diagnostic test is frequently evaluated by comparison to a perfect reference test (i.e. a gold standard). In many instances, however, a reference test is less than perfect. In this paper, we review methods for estimation of the accuracy of a diagnostic test when an imperfect reference test with known classification errors is available. Furthermore, we focus our presentation on available methods of estimation of test characteristics when the sensitivity and specificity of both tests are unknown. We present some of the available statistical methods for estimation of the accuracy of diagnostic tests when a reference test does not exist (including maximum likelihood estimation and Bayesian inference). We illustrate the application of the described methods using data from an evaluation of a nested polymerase chain reaction and microscopic examination of kidney imprints for detection of Nucleospora salmonis in rainbow trout.

Infection, excretion and seroconversion dynamics of porcine circovirus type 2 (PCV2) in pigs from post-weaning multisystemic wasting syndrome (PMWS) affected farms in Spain and Denmark.

Longitudinal case-control studies were performed in post-weaning multisystemic wasting syndrome (PMWS) affected farms from Denmark and Spain using similar designs. Fourteen independent batches of 100-154 pigs per batch were monitored from birth to PMWS outbreak occurrence. Pigs displaying PMWS-like signs and matched healthy cohorts were euthanized during the clinical outbreak. PMWS was diagnosed according to internationally accepted criteria and pigs were classified as: (i) PMWS cases, (ii) wasted non-PMWS cases and (iii) healthy pigs. Porcine circovirus type 2 (PCV2) quantitative PCR (qPCR) and serology techniques were applied to analyse longitudinally collected sera and/or nasal and rectal swabs. Results showed that PCV2 load increased in parallel to waning maternal antibody levels, reaching the maximum viral load concurrent with development of clinical signs. PMWS affected pigs had higher PCV2 prevalence and/or viral load than healthy pigs in all collected samples at necropsy (p<0.0001-0.05) and even in sera and nasal swabs at the sampling prior to PMWS outbreak (p<0.01-0.05). Danish farms had a higher PCV2 prevalence in young piglets as well as an earlier PMWS presentation compared to Spanish farms. PMWS diagnoses were confirmed by laboratory tests in only half of pigs clinically suspected to suffer from PMWS. Positive and significant correlations were found among PCV2 viral loads present in sera, nasal swabs, rectal swabs and lymphoid tissues (R=0.289-0.827, p<0.0001-0.01), which indicates that nasal and rectal swabs were suitable indicators of PCV2 excretion. Sensitivity and/or specificity values observed from both tests used separately or combined suggested that qPCR and/or serology tests are not apparently able to substitute histopathology plus detection of PCV2 in tissues for the individual PMWS diagnosis within PMWS affected farms. However, qPCR appears to be a potential reliable technique to diagnose PMWS on a population basis.

Prevalence of subclinical Salmonella enterica infection in Danish finishing pig herds.

Our aim was to determine the Salmonella enterica prevalence in 96 randomly selected Danish pig herds, based on serological examination of blood samples and bacteriological examination of faecal samples (collected simultaneously from the same pens). For comparison, 39 high-seroprevalence herds were included in the study. The representativeness of the selected herds was assessed, based on descriptive statistics of herd size and type. Totals of 1330 pen samples and 6814 blood samples were examined.The results from the meat-juice screening in the Danish S. enterica Control Programme were available for 3372 meat-juice samples from 91 of the 96 randomly selected herds and 1195 meat-juice samples from 37 of the 39 high-seroprevalence herds. Of the 96 randomly selected herds, 23 herds had no positive pen samples (out of 10), no positive blood samples (out of 50) and no positive meat-juice samples (out of approximately 30-40 samples in 6 months). Ten herds had one or more positive meat-juice samples but were otherwise negative. S. Typhimurium was isolated from 30 of the 39 high-seroprevalence herds. Our conclusions were: (1) The within-herd seroprevalence among the 96 randomly selected Danish pig herds was low (average within-herd seroprevalence=2%, maximum=32%). (2) Among the 39 high-seroprevalence herds (recently assigned level 3 in the S. enterica Control Programme), S. enterica was isolated from 77% of the herds when 10 pen samples were examined bacteriologically. (3) Seropositivity tended to be related to the presence of S. Typhimurium.

Inter-laboratory and inter-assay comparison on two real-time PCR techniques for quantification of PCV2 nucleic acid extracted from field samples.

Several real-time PCR assays for quantification of PCV2 DNA (qPCR) have been described in the literature, and different in-house assays are being used by laboratories around the world. A general threshold of 10(7) copies of PCV2 per millilitre serum for postweaning multisystemic wasting syndrome (PMWS) diagnosis has been suggested. However, neither inter-laboratory nor inter-assay comparisons have been published so far. In the present study, two different qPCR probe assays used routinely in two laboratories were compared on DNA extracted from serum, nasal and rectal swabs. Results showed a significant linear association between the assays (p<0.0001), and a systematic difference of 1.4 log10 copies of PCV2 per millilitre of sample (p<0.0001). This difference indicated that the assay from laboratory 1 yielded a higher output than the one from laboratory 2. Results also showed that there was no linear association between the amount of PCV2 DNA and the amount of total DNA, neither in nasal (p=0.86) nor in rectal (p=0.78) swabs, suggesting that normalizing of PCV2 DNA load in swab samples to total DNA concentration is not suitable. The present exploratory study highlights the need for the performance of ring trials on qPCV2 protocols between laboratories. Meanwhile, the proposed thresholds for PMWS diagnosis should only be considered reliable for each particular laboratory and each particular assay.

The need for built-in validation of surveillance data so that changes in diagnostic performance of post-mortem meat inspection can be detected

Time trends in animal-disease surveillance often are evaluated on the basis of crude estimates of apparent prevalence. In addition to possible changes in the true prevalence of the condition, changes in apparent prevalence over time might reflect changes in sensitivity and/or specificity of the diagnostic classification used. To illustrate this, comparative post-mortem meat inspection data from four Danish slaughter plants sampled in 1993-1994 and 1997-1998 were used to obtain latent-class model estimates of the sensitivity and specificity of traditional and extended post-mortem meat inspection of visceral and parietal chronic pleuritis (CP), respectively.True prevalence of CP was estimated for each study period and slaughter plant by latent-class models. Estimated sensitivities of traditional post-mortem meat (TPM) inspection ranged from 28.8 to 61.4% (1993-1994) and 39.2 to 87.3% (1997-1998). An increase in sensitivity with time was seen for all slaughter plants. Estimated sensitivities of extended post-mortem meat (EPM) inspection ranged from 85.7 to 94.8% (1993-1994) and 73.8 to 93.0% (1997-1998). All estimated specificities were >93.3%. The possible association of the estimated true prevalence of CP with time (1993-1994 versus 1997-1998) was investigated with a logistic-regression model with random effects. A slight, but non-significant decrease in the odds of CP from 1994 to 1998 was found (odds ratio=0.9). In this and similar situations, one should consider conducting ongoing double-classification of samples of units followed by statistical estimation of true prevalences, sensitivities and specificities, so that decisions can be based on such estimates rather than on crude apparent prevalences.

Infectious and rearing-system related risk factors for chronic pleuritis in slaughter pigs

Chronic pleuritis (CP) in Danish pigs for slaughter is by far the most frequent finding at the routine post-mortem meat inspection. An initial investigation published in 1990 demonstrated infectious and management-related risk factors. Serological testing for additional infectious agents, as well as the need to consider the effect of disease clustering at the herd level, required a re-analysis of the data. Our re-analysis used a representative sample of 4,800 pigs originating from 623 Danish herds. Each pig was examined for the presence of CP and progressive atrophic rhinitis (PAR). The gender of the pig, the weight of the carcass, and the herd of origin were also recorded. Individual blood samples were examined for seropositivity for Actinobacillus pleuropneumoniae (AP) serotypes 2, 6, 7, 12, Haemophilus parasuis, Mycoplasma hyopneumoniae (MYC) and swine influenza (SI). Herd-level information retrieved through a questionnaire included health status, production type, herd size (i.e. pigs per year) and vaccination procedures. Associations between CP and infectious, individual and herd-related factors were investigated by logistic regression with random effects. Among pigs from herds with conventional health status, seropositivity for AP serotypes 2 and 6, and MYC had odds ratios (ORs) of CP of 9.0, 1.6 and 1.8, respectively. Neither seropositivity for AP serotype 7 nor SI were associated with CP by themselves, but interacted: OR of CP of 5.3 (1.8) when present at the same time among pigs exhibiting (not exhibiting) PAR. An association of PAR with CP was found, and PAR interacted with AP serotype 7: OR=10.0 (4.3) when both factors were present among pigs exposed (non-exposed) to SI. The OR (0.97) for an increase of carcass weight by 1 kg was negligible. In pigs from specific pathogen-free (SPF) herds, seropositivity for MYC and herd size were associated with CP. Moreover, for a herd size of 1,000 pigs, CP was associated with exposure to MYC by an OR of 3.3 (decreasing to 1.9 when the herd size was increased by 1,000). Farrow-to-finish as opposed to finishing herd had an OR of CP of 3.2. In conventional herds, seropositivity for AP serotype 2 and MYC were associated with 51% and 29% of the occurrence of CP. In SPF herds, farrow-to-finish as opposed to finishing herds was associated with 47% of the occurrence of CP. Seropositivity for MYC was associated with 33% (39%) of the occurrence of CP in herds with a size > (< or =) 1,500 pigs.

Comparing control strategies against foot-and-mouth disease: Will vaccination be cost-effective in Denmark?

Recent outbreaks of foot-and-mouth disease (FMD) in Europe have highlighted the need for assessment of control strategies to optimise control of the spread of FMD. Our objectives were to assess the epidemiological and financial impact of simulated FMD outbreaks in Denmark and the effect of using ring depopulation or emergency vaccination to control these outbreaks. Two stochastic simulation models (InterSpreadPlus (ISP) and the modified Davis Animal Disease Simulation model (DTU-DADS)) were used to simulate the spread of FMD in Denmark using different control strategies. Each epidemic was initiated in one herd (index herd), and a total of 5000 index herds were used. Four types of control measures were investigated: (1) a basic scenario including depopulation of detected herds, 3 km protection and 10 km surveillance zones, movement tracing and a three-day national standstill, (2) the basic scenario plus depopulation in ring zones around detected herds (Depop), (3) the basic scenario plus protective vaccination within ring zones around detected herds, and (4) the basic scenario plus protective vaccination within ring zones around detected herds. Disease spread was simulated through direct animal movements, medium-risk contacts (veterinarians, artificial inseminators or milk controllers), low-risk contacts (animal feed and rendering trucks, technicians or visitors), market contacts, abattoir trucks, milk tanks, or local spread. The two simulation models showed different results in terms of the estimated numbers. However, the tendencies in terms of recommendations of strategies were similar for both models. Comparison of the different control strategies showed that, from an epidemiological point of view, protective vaccination would be preferable if the epidemic started in a cattle herd in an area with a high density of cattle, whereas if the epidemic started in an area with a low density of cattle or in other species, protective vaccination or depopulation would have almost the same preventive effect. Implementing additional control measures either 14 days after detection of the first infected herd or when 10 herds have been diagnosed would be more efficient than implementing additional control measures when more herds have been diagnosed. Protective vaccination scenarios would never be cost-effective, whereas depopulation or suppressive vaccination scenarios would most often be recommended. Looking at the median estimates of the cost-benefit analysis, depopulation in zones would most often be recommended, although, in extreme epidemics, suppressive vaccination scenarios could be less expensive. The vast majority of the costs and losses associated with a Danish epidemic could be attributed to export losses.

Estimation of sensitivity, specificity and predictive values of two serologic tests for the detection of antibodies against Actinobacillus pleuropneumoniae serotype 2 in the absence of a reference test (gold standard)

Latent-class models were used to determine the sensitivity, specificity and predictive values of a polyclonal blocking enzyme-linked immunosorbent assay (ELISA) and a modified complement-fixation test (CFT) when there was no reference test. The tests were used for detection of antibodies against Actinobacillus pleuropneumoniae serotype 2 in a survey of respiratory diseases in Danish finishing pigs. The estimates were obtained by maximum-likelihood and also by a Bayesian method (implemented with Gibbs sampling). Possible dependence of diagnostic errors was investigated by comparing models where independence was assumed to models allowing for conditional dependence, given the true disease status. No strong evidence of conditional dependence in either test sensitivity or specificity was found. Assuming independence, maximum-likelihood estimates and 95% confidence intervals of the sensitivity and specificity of the ELISA were 100% and 92.8% (90.1-95.5%) and the corresponding values of the CFT were 90.6% (85.8-95.4%) and 98.6% (98.0-99.3%), respectively. Bayesian estimates and posterior 95% credible intervals of the sensitivity and specificity of the ELISA were 99.7% (98.7-100%) and 92.7% (89.9-95.3%) and of the CFT were 90.6% (86.0-95.3%) and 98.7% (98.0-99.3%). The sensitivity and specificity of a combined test, where the CFT is subsequently applied to the pig sera that test positive in the ELISA, were estimated at 90.2% (85.6-95.0%) and 99.9% (99.8-100%), respectively. The cost of the combined test was less than the cost of the use of the CFT alone, at prevalences <54%. Prevalences and predictive values and their 95% limits were estimated in six sub-samples of data. The estimates of sensitivity and specificity obtained in the present investigation generally validate those reported from other sources.

Decisions on control of foot-and-mouth disease informed using model predictions

The decision on whether or not to change the control strategy, such as introducing emergency vaccination, is perhaps one of the most difficult decisions faced by the veterinary authorities during a foot-and-mouth disease (FMD) epidemic. A simple tool that may predict the epidemic outcome and consequences would be useful to assist the veterinary authorities in the decision-making process. A previously proposed simple quantitative tool based on the first 14 days outbreaks (FFO) of FMD was used with results from an FMD simulation exercise. Epidemic outcomes included the number of affected herds, epidemic duration, geographical size and costs. The first 14 days spatial spread (FFS) was also included to further support the prediction. The epidemic data was obtained from a Danish version (DTU-DADS) of a pre-existing FMD simulation model (Davis Animal Disease Spread - DADS) adapted to model the spread of FMD in Denmark. The European Union (EU) and Danish regulations for FMD control were used in the simulation. The correlations between FFO and FFS and the additional number of affected herds after day 14 following detection of the first infected herd were 0.66 and 0.82, respectively. The variation explained by the FFO at day 14 following detection was high (P-value<0.001). This indicates that the FFO may take a part in the decision of whether or not to intensify FMD control, for instance by introducing emergency vaccination and/or pre-emptive depopulation, which might prevent a catastrophic situation. A significant part of the variation was explained by supplementing the model with the FFS (P-value<0.001). Furthermore, the type of the index-herd was also a significant predictor of the epidemic outcomes (P-value<0.05). The results of the current study suggest that national veterinary authorities should consider to model their national situation and to use FFO and FFS to help planning and updating their contingency plans and FMD emergency control strategies.

Meta-analysis on the efficacy of foot-and-mouth disease emergency vaccination

The objectives of this study were to provide a summary quantification of the efficacy of FMD emergency vaccination based on a systematic review and a meta-analysis of available literature, and to further discuss the suitability of this review and meta-analysis to summarize and further interpret the results. Peer-reviewed, symposium, and unpublished studies were considered in the analysis. Clinical protection and virological protection against FMD were used as parameters to assess the efficacy of emergency vaccination. The clinical protection was estimated based on the appearance of clinical signs including FMD lesions and fever, while the virological protection parameter was estimated based on the outcome of laboratory tests that were used to diagnose FMD infection. A meta-analysis relative risk was calculated per protection parameter. Results of the meta-analyses were examined using publication bias tests. In total, 31 studies were included in the analyses, of which 26 were peer-reviewed studies, 1 was a symposium study and 4 were unpublished studies. Cattle, swine and sheep were well protected against clinical disease and FMD infection following the use of emergency vaccine. Fortunately, no significant bias that would alter the conclusions was encountered in the analysis. Meta-analysis can be a useful tool to summarize literature results from a systematic review of the efficacy of FMD emergency vaccination.