P. Sean Walsh

Genotyping over 100,000 SNPs on a pair of oligonucleotide arrays

We present a genotyping method for simultaneously scoring 116,204 SNPs using oligonucleotide arrays. At call rates >99%, reproducibility is >99.97% and accuracy, as measured by inheritance in trios and concordance with the HapMap Project, is >99.7%. Average intermarker distance is 23.6 kb, and 92% of the genome is within 100 kb of a SNP marker. Average heterozygosity is 0.30, with 105,511 SNPs having minor allele frequencies >5%.

Analytical Performance Verification of a Molecular Diagnostic for Cytology-Indeterminate Thyroid Nodules

OBJECTIVE Our objective was to verify the analytical performance of the Afirma gene expression classifier (GEC) in the classification of cytologically indeterminate thyroid nodule fine-needle aspirates (FNAs). DESIGN Analytical performance studies were designed to characterize the stability of RNA in FNAs during collection and shipment, analytical sensitivity as applied to input RNA concentration and malignant/benign FNA mixtures, analytical specificity (i.e. potentially interfering substances) as tested on blood and genomic DNA, and assay performance studies including intra-nodule, intraassay, inter-assay, and inter-laboratory reproducibility. RESULTS RNA content within FNAs preserved in FNAProtect is stable for up to 6 d at room temperature with no changes in RNA yield (P = 0.58) or quality (P = 0.56). FNA storage and shipping temperatures were found to have no significant effect on GEC scores (P = 0.55) or calls (100% concordance). Analytical sensitivity studies demonstrated tolerance to variation in RNA input (5-25 ng) and to the dilution of malignant FNA material down to 20%. Analytical specificity studies using malignant samples mixed with blood (up to 83%) and genomic DNA (up to 30%) demonstrated negligible assay interference with respect to false-negative calls, although benign FNA samples mixed with relatively high proportions of blood demonstrated a potential for false-positive calls. The test is reproducible from extraction through GEC result, including variation across operators, runs, reagent lots, and laboratories (sd of 0.158 for scores on a >6 unit scale). CONCLUSIONS Analytical sensitivity, analytical specificity, robustness, and quality control of the GEC were successfully verified, indicating its suitability for clinical use.

Does Addition of BRAF V600E Mutation Testing Modify Sensitivity or Specificity of the Afirma Gene Expression Classifier in Cytologically Indeterminate Thyroid Nodules

OBJECTIVE The purpose of this study was to determine the frequency of BRAF mutation in cytologically indeterminate thyroid nodules and to investigate whether adding the BRAF test improves diagnostic accuracy of the Afirma Gene Expression Classifier (GEC). DESIGN BRAF V600E mutational status was determined for DNA extracted from cytologically benign (n = 40), indeterminate (n = 208), and malignant (n = 48) fine-needle aspiration specimens previously categorized by GEC as molecularly Benign or Suspicious. Analytical performance of the BRAF assay was assessed to establish reproducibility and limits of detection. Molecular testing results were correlated with blinded expert histopathological diagnoses. RESULTS The BRAF assay detected mutations reproducibly to 2.5% mutant allele frequency. The prevalence of BRAF mutations in cytologically benign specimens was 2 of 40 (5.0%, 95% confidence interval [CI], 0-16) and in cytologically malignant specimens was 36 of 48 (75.0%, 95% CI, 60-86). In the cytologically indeterminate category, 10.1% of specimens were BRAF+: 2 of 95 were subcategorized as atypia of undetermined significance or follicular lesion of undetermined significance (2.1%, 95% CI, 0-7); 1 of 70 as follicular neoplasm or suspicious for follicular neoplasm (1.4%, 95% CI, 0-9); and 18 of 43 as suspicious for malignancy (41.9%, 95% CI, 27-58). All BRAF+ specimens were classified as Suspicious by the GEC. CONCLUSIONS BRAF mutations are uncommon in nodules with atypia of undetermined significance or follicular lesion of undetermined significance or follicular neoplasm or suspicious for follicular neoplasm cytology. Most cytologically indeterminate nodules that proved to be malignant were also BRAF-, and all nodules that were false-negative by GEC were also BRAF-. Similarly, all BRAF+ specimens were also GEC Suspicious. Neither GEC test sensitivity nor specificity was improved by addition of BRAF mutation testing.

Usual Interstitial Pneumonia Can Be Detected in Transbronchial Biopsies Using Machine Learning

Rationale: Usual interstitial pneumonia (UIP) is the histopathologic hallmark of idiopathic pulmonary fibrosis. Although UIP can be detected by high‐resolution computed tomography of the chest, the results are frequently inconclusive, and pathology from transbronchial biopsy (TBB) has poor sensitivity. Surgical lung biopsy may be necessary for a definitive diagnosis. Objectives: To develop a genomic classifier in tissue obtained by TBB that distinguishes UIP from non‐UIP, trained against central pathology as the reference standard. Methods: Exome enriched RNA sequencing was performed on 283 TBBs from 84 subjects. Machine learning was used to train an algorithm with high rule‐in (specificity) performance using specimens from 53 subjects. Performance was evaluated by cross‐validation and on an independent test set of specimens from 31 subjects. We explored the feasibility of a single molecular test per subject by combining multiple TBBs from upper and lower lobes. To address whether classifier accuracy depends upon adequate alveolar sampling, we tested for correlation between classifier accuracy and expression of alveolar‐specific genes. Results: The top‐performing algorithm distinguishes UIP from non‐UIP conditions in single TBB samples with an area under the receiver operator characteristic curve (AUC) of 0.86, with specificity of 86% (confidence interval = 71‐95%) and sensitivity of 63% (confidence interval = 51‐74%) (31 test subjects). Performance improves to an AUC of 0.92 when three to five TBB samples per subject are combined at the RNA level for testing. Although we observed a wide range of type I and II alveolar‐specific gene expression in TBBs, expression of these transcripts did not correlate with classifier accuracy. Conclusions: We demonstrate proof of principle that genomic analysis and machine learning improves the utility of TBB for the diagnosis of UIP, with greater sensitivity and specificity than pathology in TBB alone. Combining multiple individual subject samples results in increased test accuracy over single sample testing. This approach requires validation in an independent cohort of subjects before application in the clinic.

Machine learning from concept to clinic: reliable detection of BRAF V600E DNA mutations in thyroid nodules using high-dimensional RNA expression data.

The promise of personalized medicine will require rigorously validated molecular diagnostics developed on minimally invasive, clinically relevant samples. Measurement of DNA mutations is increasingly common in clinical settings but only higher-prevalence mutations are cost-effective. Patients with rare variants are at best ignored or, at worst, misdiagnosed. Mutations result in downstream impacts on transcription, offering the possibility of broader diagnosis for patients with rare variants causing similar downstream changes. Use of such signatures in clinical settings is rare as these algorithms are difficult to validate for commercial use. Validation on a test set (against a clinical gold standard) is necessary but not sufficient: accuracy must be maintained amidst interfering substances, across reagent lots and across operators. Here we report the development, clinical validation, and diagnostic accuracy of a pre-operative molecular test (Afirma BRAF) to identify BRAF V600E mutations using mRNA expression in thyroid fine needle aspirate biopsies (FNABs). FNABs were obtained prospectively from 716 nodules and more than 3,000 features measured using microarrays. BRAF V600E labels for training (n=181) and independent test (n=535) sets were established using a sensitive quantitative PCR (qPCR) assay. The resulting 128-gene linear support vector machine was compared to qPCR in the independent test set. Clinical sensitivity and specificity for malignancy were evaluated in a subset of test set samples (n=213) with expert-derived histopathology. We observed high positive- (PPA, 90.4%) and negative (NPA, 99.0%) percent agreement with qPCR on the test set. Clinical sensitivity for malignancy was 43.8% (consistent with published prevalence of BRAF V600E in this neoplasm) and specificity was 100%, identical to qPCR on the same samples. Classification was accurate in up to 60% blood. A double-mutant still resulting in the V600E amino acid change was negative by qPCR but correctly positive by Afirma BRAF. Non-diagnostic rates were lower (7.6%) for Afirma BRAF than for qPCR (24.5%), a further advantage of using RNA in small sample biopsies. Afirma BRAF accurately determined the presence or absence of the BRAF V600E DNA mutation in FNABs, a collection method directly relevant to solid tumor assessment, with performance equal to that of an established, highly sensitive DNA-based assay and with a lower non-diagnostic rate. This is the first such test in thyroid cancer to undergo sufficient analytical and clinical validation for real-world use in a personalized medicine context to frame individual patient risk and inform surgical choice.

Performance of a genomic sequencing classifier for the preoperative diagnosis of cytologically indeterminate thyroid nodules

Importance Use of next-generation sequencing of RNA and machine learning algorithms can classify the risk of malignancy in cytologically indeterminate thyroid nodules to limit unnecessary diagnostic surgery. Objective To measure the performance of a genomic sequencing classifier for cytologically indeterminate thyroid nodules. Design, Setting, and Participants A blinded validation study was conducted on a set of cytologically indeterminate thyroid nodules collected by fine-needle aspiration biopsy between June 2009 and December 2010 from 49 academic and community centers in the United States. All patients underwent surgery without genomic information and were assigned a histopathology diagnosis by an expert panel blinded to all genomic information. There were 210 potentially eligible thyroid biopsy samples with Bethesda III or IV indeterminate cytopathology that constituted a cohort previously used to validate the gene expression classifier. Of these, 191 samples (91.0%) had adequate residual RNA for validation of the genomic sequencing classifier. Algorithm development and independent validation occurred between August 2016 and May 2017. Exposures Thyroid nodule surgical histopathology diagnosis by an expert panel blinded to all genomic data. Main Outcomes and Measures The primary end point was measurement of genomic sequencing classifier sensitivity, specificity, and negative and positive predictive values in biopsies from Bethesda III and IV nodules. The secondary end point was measurement of classifier performance in biopsies from Bethesda II, V, and VI nodules. Results Of the 183 included patients, 142 (77.6%) were women, and the mean (range) age was 51.7 (22.0-85.0) years. The genomic sequencing classifier had a sensitivity of 91% (95% CI, 79-98) and a specificity of 68% (95% CI, 60-76). At 24% cancer prevalence, the negative predictive value was 96% (95% CI, 90-99) and the positive predictive value was 47% (95% CI, 36-58). Conclusions and Relevance The genomic sequencing classifier demonstrates high sensitivity and accuracy for identifying benign nodules. Its 36% increase in specificity compared with the gene expression classifier potentially increases the number of patients with benign nodules who can safely avoid unnecessary diagnostic surgery.

Analytical performance of Envisia: a genomic classifier for usual interstitial pneumonia

BackgroundClinical guidelines specify that diagnosis of interstitial pulmonary fibrosis (IPF) requires identification of usual interstitial pneumonia (UIP) pattern. While UIP can be identified by high resolution CT of the chest, the results are often inconclusive, making surgical lung biopsy necessary to reach a definitive diagnosis (Raghu et al., Am J Respir Crit Care Med 183(6):788–824, 2011). The Envisia genomic classifier differentiates UIP from non-UIP pathology in transbronchial biopsies (TBB), potentially allowing patients to avoid an invasive procedure (Brown et al., Am J Respir Crit Care Med 195:A6792, 2017). To ensure patient safety and efficacy, a laboratory developed test (LDT) must meet strict regulatory requirements for accuracy, reproducibility and robustness. The analytical characteristics of the Envisia test are assessed and reported here.MethodsThe Envisia test utilizes total RNA extracted from TBB samples to perform Next Generation RNA Sequencing. The gene count data from 190 genes are then input to the Envisia genomic classifier, a machine learning algorithm, to output either a UIP or non-UIP classification result. We characterized the stability of RNA in TBBs during collection and shipment, and evaluated input RNA mass and proportions on the limit of detection of UIP. We evaluated potentially interfering substances such as blood and genomic DNA. Intra-run, inter-run, and inter-laboratory reproducibility of test results were also characterized.ResultsRNA content within TBBs preserved in RNAprotect is stable for up to 14 days with no detectable change in RNA quality. The Envisia test is tolerant to variation in RNA input (5 to 30 ng), with no impact on classifier results. The Envisia test can tolerate dilution of non-UIP and UIP classification signals at the RNA level by up to 60% and 20%, respectively. Analytical specificity studies utilizing UIP and non-UIP samples mixed with genomic DNA (up to 30% relative input) demonstrated no impact to classifier results. The Envisia test tolerates up to 22% of blood contamination, well beyond the level observed in TBBs. The test is reproducible from RNA extraction through to Envisia test result (standard deviation of 0.20 for Envisia classification scores on > 7-unit scale).ConclusionsThe Envisia test demonstrates the robust analytical performance required of an LDT. Envisia can be used to inform the diagnoses of patients with suspected IPF.