Thomas A. McWalter

HIV incidence in rural South Africa: comparison of estimates from longitudinal surveillance and cross-sectional cBED assay testing.

Background The BED IgG-Capture Enzyme Immunoassay (cBED assay), a test of recent HIV infection, has been used to estimate HIV incidence in cross-sectional HIV surveys. However, there has been concern that the assay overestimates HIV incidence to an unknown extent because it falsely classifies some individuals with non-recent HIV infections as recently infected. We used data from a longitudinal HIV surveillance in rural South Africa to measure the fraction of people with non-recent HIV infection who are falsely classified as recently HIV-infected by the cBED assay (the long-term false-positive ratio (FPR)) and compared cBED assay-based HIV incidence estimates to longitudinally measured HIV incidence. Methodology/Principal Findings We measured the long-term FPR in individuals with two positive HIV tests (in the HIV surveillance, 2003–2006) more than 306 days apart (sample size n = 1,065). We implemented four different formulae to calculate HIV incidence using cBED assay testing (n = 11,755) and obtained confidence intervals (CIs) by directly calculating the central 95th percentile of incidence values. We observed 4,869 individuals over 7,685 person-years for longitudinal HIV incidence estimation. The long-term FPR was 0.0169 (95% CI 0.0100–0.0266). Using this FPR, the cross-sectional cBED-based HIV incidence estimates (per 100 people per year) varied between 3.03 (95% CI 2.44–3.63) and 3.19 (95% CI 2.57–3.82), depending on the incidence formula. Using a long-term FPR of 0.0560 based on previous studies, HIV incidence estimates varied between 0.65 (95% CI 0.00–1.32) and 0.71 (95% CI 0.00–1.43). The longitudinally measured HIV incidence was 3.09 per 100 people per year (95% CI 2.69–3.52), after adjustment to the sex-age distribution of the sample used in cBED assay-based estimation. Conclusions/Significance In a rural community in South Africa with high HIV prevalence, the long-term FPR of the cBED assay is substantially lower than previous estimates. The cBED assay performs well in HIV incidence estimation if the locally measured long-term FPR is used, but significantly underestimates incidence when a FPR estimate based on previous studies in other settings is used.

A New General Biomarker-based Incidence Estimator

Background: Estimating disease incidence from cross-sectional surveys, using biomarkers for “recent” infection, has attracted much interest. Despite widespread applications to HIV, there is currently no consensus on the correct handling of biomarker results classifying persons as “recently” infected long after the infections occurred. Methods: We derive a general expression for a weighted average of recent incidence that—unlike previous estimators—requires no particular assumption about recent infection biomarker dynamics or about the demographic and epidemiologic context. This is possible through the introduction of an explicit timescale T that truncates the period of averaging implied by the estimator. Results: The recent infection test dynamics can be summarized into 2 parameters, similar to those appearing in previous estimators: a mean duration of recent infection and a false-recent rate. We identify a number of dimensionless parameters that capture the bias that arises from working with tractable forms of the resulting estimator and elucidate the utility of the incidence estimator in terms of the performance of the recency test and the population state. Estimation of test characteristics and incidence is demonstrated using simulated data. The observed confidence interval coverage of the test characteristics and incidence is within 1% of intended coverage. Conclusions: Biomarker-based incidence estimation can be consistently adapted to a general context without the strong assumptions of previous work about biomarker dynamics and epidemiologic and demographic history.

HIV incidence estimation using the BED capture enzyme immunoassay: systematic review and sensitivity analysis.

Background: HIV incidence estimates are essential for understanding the evolution of the HIV epidemic and the impact of interventions. Tests for recent HIV infection allow incidence estimation based on a single cross-sectional survey. The BED IgG-Capture Enzyme Immunoassay (BED assay) is a commercially available and widely used test for recent HIV infection. Methods: In a systematic literature search for BED assay studies, we identified 1181 unique studies, 1138 of which were excluded based on titles or abstracts. We conducted reviews of the 43 remaining publications and a further 23 studies identified on conference Web sites or by colleagues. Thirty-nine articles were included in the final review. We investigated the sensitivity of incidence values to various estimation methods and parameter choices. Results: BED assay surveys have been conducted on 5 continents in general populations and high-risk groups, using 1 or more of 10 distinct incidence formulae. Most studies used estimators that do not account for assay imperfection. Those studies that correct for assay imperfection commonly do not use locally valid assay parameters. Incidence estimates were very sensitive to methodological and parameter choices. Most confidence intervals provided good assessment of uncertainty due to counting error, but only a few incorporated parameter uncertainty. Conclusions: BED assay surveys can produce valid HIV incidence estimates, but many studies have not sufficiently accounted for assay imperfection. Future studies should (1) report all information necessary for incidence point and uncertainty estimation, (2) use an unbiased estimator with locally valid assay calibration parameters, and (3) compute confidence intervals that take into account parameter uncertainty.

Relating recent infection prevalence to incidence with a sub-population of assay non-progressors

We present a new analysis of relationships between disease incidence and the prevalence of an experimentally defined state of ‘recent infection’. This leads to a clean separation between biological parameters (properties of disease progression as reflected in a test for recent infection), which need to be calibrated, and epidemiological state variables, which are estimated in a cross-sectional survey. The framework takes into account the possibility that details of the assay and host/pathogen chemistry leave a (knowable) fraction of the population in the recent category for all times. This systematically addresses an issue which is the source of some controversy about the appropriate use of the BED assay for defining recent HIV infection. The analysis is, however, applicable to any assay that forms the basis of a test for recent infection. Analysis of relative contributions of error arising variously from statistical considerations and simplifications of general expressions indicate that statistical error dominates heavily over methodological bias for realistic epidemiological and biological scenarios.

A Comparison of Biomarker Based Incidence Estimators

Background Cross-sectional surveys utilizing biomarkers that test for recent infection provide a convenient and cost effective way to estimate HIV incidence. In particular, the BED assay has been developed for this purpose. Controversy surrounding the way in which false positive results from the biomarker should be handled has lead to a number of different estimators that account for imperfect specificity. We compare the estimators proposed by McDougal et al., Hargrove et al. and McWalter & Welte. Methodology/Principal Findings The three estimators are analyzed and compared. An identity showing a relationship between the calibration parameters in the McDougal methodology is shown. When the three estimators are tested under a steady state epidemic, which includes individuals who fail to progress on the biomarker, only the McWalter/Welte method recovers an unbiased result. Conclusions/Significance Our analysis shows that the McDougal estimator can be reduced to a formula that only requires calibration of a mean window period and a long-term specificity. This allows simpler calibration techniques to be used and shows that all three estimators can be expressed using the same set of parameters. The McWalter/Welte method is applicable under the least restrictive assumptions and is the least prone to bias of the methods reviewed.

A Simplified Formula for Inferring HIV Incidence from Cross-Sectional Surveys Using a Test for Recent Infection.

Editor: The paper of McDougal et al., previously published in this journal,1 is becoming a standard reference used for the estimation of HIV incidence from applications of the BED IgG-capture enzyme immunoassay (BED assay) to cross-sectional blood samples.2,3 Their approach provides an estimate for an annual risk of infection in a hypothetical cohort, using an estimate for the true proportion, Pt, of “recent infections” among HIV-seropositive individuals. The estimate Pt is in turn derived from the proportion, P0, of seropositive individuals in a survey who test below a threshold value for normalized BED optical density (OD-n).4 The condition of being below the OD-n threshold is declared to be an imperfect test for recent infection. True “recent infection” is defined as having been infected for less than a period ω, where ω is the mean time individuals spend below the OD-n threshold. Since it is well known that not all individuals progress to a given threshold, even after arbitrarily long times, ω needs to be carefully defined as the mean threshold crossing time among those who do progress. It is also known that during late stage illness, or under the influence of antiretroviral therapy, individuals may regress to OD-n values below the recency threshold. It is further plausible, and indeed appears to be the case,5,6 that the parameters characterizing progression through the BED-defined states of infection vary regionally. These complications have caused doubt about the prospects for using the BED assay as a robustly characterizable test for recent infection for the purposes of estimating HIV incidence, as reflected in a UNAIDS statement in 20067 recommending it not be used for this purpose. Hence, new assays, or combinations of assays (such as a BED and an antibody avidity test), are being developed to provide more robust tests for recent infection. The fraction of individuals that progresses atypically through an assay-defined class of “recently infected” may thus be reduced, but is unlikely to be zero. Therefore, the methodology developed to deal with this problem for the BED assay appears, at face value, to be immediately transferable, requiring only minor modification (namely in the values of its parameters) to be applicable to other imperfect tests for recent infection. We argue that several subtle points need to be addressed to ensure that incidence inferences based on imperfect tests for recent infection are not unnecessarily limited, or even in error, and we do this by a critique of the original application. The interindividual variability of BED OD-n progression is captured in the McDougal model by three parameters: The sensitivity (σ) of the BED assay as a test for the condition of being “recently infected,” as defined above. The short-term specificity (ρ1) of the BED assay as a test for the condition of being “recently infected,” when restricted to persons who have been infected for a time between ω and 2ω. The long-term specificity (ρ2) of the BED assay as a test for the condition of being “recently infected,” when restricted to persons who have been infected for a time longer than 2ω. Using data from a major epidemiological and demographic surveillance study in South Africa,8,9 we and our collaborators are currently comparing various approaches to HIV incidence estimation using the BED assay.5,10 Given the long intervals between follow-up visits in this study (about a year), it was not possible to calibrate the McDougal formula in its published form. Calibration of σ and ρ1 requires a follow-up interval of at most ω (which is of the order of half a year1). While trying to address this issue, we discovered that a simplification of the McDougal formula is possible. In their paper, the key result relating Pt to the calibration parameters is given by (1) As is shown by McWalter and Welte in a separate short note,11 the above equation can be simplified using the following identity: (2) This identity is derived using no more assumptions than are used by McDougal et al. to derive their formula; these assumptions are, however, have been stated with greater precision.11 The idea that these parameters might be related was inspired by the analysis of the incidence estimation problem previously undertaken.12 Inserting the identity into (1) gives (3) This means that in order to estimate incidence, it is only necessary to calibrate the long-term specificity ρ2 (to estimate Pt) and the window period ω (to convert Pt to an annual risk of infection). Unlike σ and ρ1, these can both be inferred from infrequent follow-up. Incidentally, using the values of σ, ρ1, and ρ2 previously reported,1 we find that (4) which manifests the combined fluctuations in the estimates of σ, ρ1, ρ2, and ω. Although ω is superficially absent in the identity, it enters as the period over which the other parameters are defined. The appropriately simplified form10 is amenable to calibration using data obtained with long intervals of follow-up.10 This seems to us to be an important point, as many demographic and epidemiological surveillance studies we are aware of, or expect to see implemented, are characterized by follow-up intervals of the order of a year, almost ideal for calibrating the reduced formula and clearly inadequate for calibrating the previously published form. There is likely to be substantial data of this sort available. On the other hand, the cost of obtaining short interval follow-up data is high and the opportunities for doing so are rare. Note that even given an appropriate data set for estimating σ, ρ1, and ρ2, the use of the naive formula, for the purpose of systematically quantifying uncertainty due to imperfect calibration, would require additional specification of nontrivial covariances implied by the identity.5 The attraction of using a test for recent infection for HIV surveillance, program evaluation, and policy making lies in the fact that it allows HIV incidence estimation from cross-sectional blood samples. Cross-sectional HIV status information alone, however, does not allow estimation of the calibration parameters. These must be estimated in separate studies, involving follow-up of an intensity comparable to a prospective observation of incidence. Only after this has been done can the more efficient cross-sectional survey be employed on a suitably similar population. The more robust and locally validated the calibration parameters are, the more informative cross-sectional surveys can be. Therefore it is important that the necessary parameters be calibrated as widely and thoroughly as possible, using such data as is available. The parameters of the simplified formula are independent and can be estimated from comparatively long interval follow-up data, while the parameters used by McDougal et al.1 have nontrivial correlation and require short intervals of follow-up.

Seroconverting blood donors as a resource for characterising and optimising recent infection testing algorithms for incidence estimation.

Introduction Biomarker-based cross-sectional incidence estimation requires a Recent Infection Testing Algorithm (RITA) with an adequately large mean recency duration, to achieve reasonable survey counts, and a low false-recent rate, to minimise exposure to further bias and imprecision. Estimating these characteristics requires specimens from individuals with well-known seroconversion dates or confirmed long-standing infection. Specimens with well-known seroconversion dates are typically rare and precious, presenting a bottleneck in the development of RITAs. Methods The mean recency duration and a ‘false-recent rate’ are estimated from data on seroconverting blood donors. Within an idealised model for the dynamics of false-recent results, blood donor specimens were used to characterise RITAs by a new method that maximises the likelihood of cohort-level recency classifications, rather than modelling individual sojourn times in recency. Results For a range of assumptions about the false-recent results (0% to 20% of biomarker response curves failing to reach the threshold distinguishing test-recent and test-non-recent infection), the mean recency duration of the Vironostika-LS ranged from 154 (95% CI: 96–231) to 274 (95% CI: 234–313) days in the South African donor population (n = 282), and from 145 (95% CI: 67–226) to 252 (95% CI: 194–308) days in the American donor population (n = 106). The significance of gender and clade on performance was rejected (p−value = 10%), and utility in incidence estimation appeared comparable to that of a BED-like RITA. Assessment of the Vitros-LS (n = 108) suggested potentially high false-recent rates. Discussion The new method facilitates RITA characterisation using widely available specimens that were previously overlooked, at the cost of possible artefacts. While accuracy and precision are insufficient to provide estimates suitable for incidence surveillance, a low-cost approach for preliminary assessments of new RITAs has been demonstrated. The Vironostika-LS and Vitros-LS warrant further analysis to provide greater precision of estimates.