A Public Website for the Automated Assessment and Validation of SARS-CoV-2 Diagnostic PCR Assays
Po-E Li, Adán Myers y Gutiérrez, Karen Davenport, Mark Flynn, Bin Hu, Chien-Chi Lo, Elais Player Jackson, Migun Shakya, Yan Xu, Jason Gans, Patrick S. G. Chain
A Public Website for the Automated Assessment and Validation of SARS-CoV-2 Diagnostic PCR Assays
Po-E Li , Adán Myers y Gutiérrez , Karen Davenport, Mark Flynn, Bin Hu, Chien-Chi Lo, Elais Player Jackson, Migun Shakya, Yan Xu, Jason Gans*, and Patrick S. G. Chain* Bioscience Division, Los Alamos National Laboratory, Los Alamos, New Mexico Contributed equally to this work. *To whom correspondence should be addressed.
Abstract
Summary:
Polymerase chain reaction-based assays are the current gold standard for detecting and diagnosing SARS-CoV-2. However, as SARS-CoV-2 mutates, we need to constantly assess whether existing PCR-based assays will continue to detect all known viral strains. To enable the continuous monitoring of SARS-CoV-2 assays, we have developed a web-based assay validation algorithm that checks existing PCR-based assays against the ever-expanding genome databases for SARS-CoV-2 using both thermodynamic and edit-distance metrics. The assay screening results are displayed as a heatmap, showing the number of mismatches between each detection and each SARS-CoV-2 genome sequence. Using a mismatch threshold to define detection failure, assay performance is summarized with the true positive rate (recall) to simplify assay comparisons.
Availability: https://covid19.edgebioinformatics.org/
Contact:
Jason Gans ([email protected]) and Patrick Chain ([email protected]) Introduction
Many aspects of the control, management and treatment responses to the global COVID-19 pandemic require accurate detection of its causative agent, SARS-CoV-2. To address this challenge, research groups around the world have developed Polymerase Chain Reaction (PCR)-based assays to detect SARS-CoV-2 genomic RNA (Supplementary Tables S1). The impact of SARS-CoV-2 genetic drift on the ability of PCR-based assays to successfully detect target sequences is a concern. To address this concern, we have developed a web-based application that monitors existing SARS-CoV-2 PCR-based assays that are in use around the world and provides a visual summary of assay performance. Both the acquisition of new genomes and the assay validation process is automated, so that assays are checked and displayed daily to give near real-time results. Implementation
The core of the validation algorithm is the ThermonucleotideBLAST (Gans and Wolinsky, 2008) in silico
PCR screening tool. Publicly available assays are used as queries in ThermonucleotideBLAST and searched against a target database of SARS-CoV-2 genomes from the Global Initiative on Sharing All Influenza Data (GISAID) (Shu and McCauley, 2017) and Genbank (Clark et al., 2016). Sequences are downloaded daily from these databases and filtered to exclude any that are less than 29 kilobases or are pangolin-SARS and bat- SARS. For sequences found in both databases, only the GenBank version is retained. True positives are defined as any assay-target pairwise alignment that contains in 0, 1, or 2 mismatches (in the oligonucleotide with the most mismatches). False negatives are defined as an assay/target combination that has either (a) one or more oligo/target pairwise alignments with 3 or more mismatches, or (b) one or more predicted oligo/target melting temperatures less than 40°C. Since all of the included assays are intended to detect SARS-CoV-2 and false positives are not predicted, assay performance is quantified by the recall (defined as the number of true positives divided by the sum of true positives and false negatives).
Figure 1. Visualization of in silico evaluation of diagnostic assays. (A) Dashboard including a bar chart and table with per-assay recall and mismatch counts; (B) Phylogenetic tree created from high-quality genomes color-labeled by continent; (C) A heatmap display of assay assessment per assay per genome; (D) Assay details and statistics of genomes with mismatches; (E) Detailed assay evaluation results, including alignments and thermodynamic information.
Per-assay recall values are summarized in the dashboard (Fig. 1A). The assays with the best recall rates are shown in a bar chart, which also displays detailed mismatch counts. The total mismatch and failure results are summarized in the per-assay table of aggregated data. Selecting any bar in the chart or assay in the table will display additional information on the distribution of targets with mismatches (Fig. 1D). A phylogenetic tree (Fig. 1B), created using PhaME (Shakya et al. 2020), is derived from genomes defined by GISAID as high-quality (<1% Ns and <0.05% unique mutations). The leaves on the tree are represented by the genome labels and color-coded by geographic location. Mousing over the genome labels displays metadata associated with the sample. Identical SARS-CoV-2 sequences are clustered and represented as collapsed branches in the tree. The heatmap (Fig. 1C), color-coded to indicate the number of mismatches, shows analysis of every combination of assay and SARS-CoV-2 genome sequence. Selecting an individual cell of the heatmap displays detailed pairwise alignment information (Fig. 1E). This visualization is rendered using a custom PhyD3 phylogenetic tree viewer (Kreft et al., 2017).
3 Discussion
Few other public resources exist for assessing the performance of PCR-based SARS-CoV-2 assays. GISAID, one of the primary repositories for SARS-CoV-2 genomes, provides a high-level summary of PCR-based assay performance for registered users. However, this information is provided in the form of a static image with only a limited amount of information. The virological.org website provides static tables summarizing the high-level performance of PCR assays that have been periodically uploaded (Holland et al., 2020). Unlike these resources, the web-based application presented here provides a more detailed and interactive view of molecular assay performance that is updated regularly with recently deposited genomes (>31K as of May 22, 2020). The heatmap-phylogeny view reveals patterns in predicted assay performance, including mismatches for the Charité RdRP assays (Vogels et al., 2020, Corman, et al.,
Acknowledgments
The authors declare no conflict of interest. Hosting of edgebioinformatics.org is provided by Cyverse, which is supported by the National Science Foundation under Award Numbers DBI-0735191, DBI-1265383, and DBI-1743442. We acknowledge the authors and originators of sequences from the submitting laboratories who have contributed to the GISAID database.
Funding
This research was supported by LANL (20200732ER), by DTRA (CB10152 and CB10623) and by the DOE Office of Science (KP160101), through the National Virtual Biotechnology Laboratory, a consortium of DOE national laboratories focused on response to COVID-19, with funding provided by the Coronavirus CARES Act.
References et al. (2015) GenBank.
Nucleic Acids Res. , , D67–D72. Corman, V. et al. Nucleic Acids Res. , , 1–5. Holland, M. et al. (2020) BioLaboro: A bioinformatics system for detecting molecular assay signature erosion and designing new assays in response to emerging and reemerging pathogens. bioRxiv , 2020.04.08.031963. Kreft, Ł. et al. (2017) PhyD3: a phylogenetic tree viewer with extended phyloXML support for functional genomics data visualization. Bioinformatics , , 2946–2947. Shakya, M. et al. (2020) Standardized phylogenetic and molecular evolutionary analysis applied to species across the microbial tree of life. Sci. Rep. , , 1723. Shu, Y. and McCauley, J. (2017) GISAID: Global initiative on sharing all influenza data - from vision to reality. Euro Surveill. , . Vogels, C.B., et al . (2020) Analytical sensitivity and efficiency comparisons of SARS-COV-2 qRT-PCR assays. medRxiv . upp l e m e n t a r y T a b l e S : P ub li c l y a v a il ab l e a ss a ys and t he s equen c e s o f t he a ss a y o li gonu c l eo t i de s N a m e - Fo r w a r d - ’ ’ - R eve r se - ’ ’ - P r ob e - ’ S ou r ce ( S ee h tt p s : // . w ho .i n t / e m e r g e n c i es / d i seases / no ve l - c o r on av i r u s - / t ec hn i ca l - gu i d a n ce /l a bo r a t o r y - gu i d a n ce ) CDC - - n C o V _ N GACCCCAAAATCAGCGAAAT
TCTGGTTACTGCCAGTTGAATCTG
ACCCCGCATTACGTTTGGTGGACC h tt p s :// . c d c . go v / c o r ona v i r u s / - n c o v / l ab / r t - p c r- pane l - p r i m e r- p r obe s . h t m l CDC - - n C o V _ N TTACAAACATTGGCCGCAAA
GCGCGACATTCCGAAGAA
ACAATTTGCCCCCAGCGCTTC h tt p s :// . c d c . go v / c o r ona v i r u s / - n c o v / l ab / r t - p c r- pane l - p r i m e r- p r obe s . h t m l CDC - - n C o V _ N GGGAGCCTTGAATACACCAAAA
TGTAGCACGATTGCAGCATTG A Y CACATTGGCACCCGCAATCCTG h tt p s :// . c d c . go v / c o r ona v i r u s / - n c o v / l ab / r t - p c r- pane l - p r i m e r- p r obe s . h t m l C h i na - CDC - O R F CCCTGTGGGTTTTACACTTAA
ACGATTGTGCATCAGCTGA
CCGTCTGCGGTATGTGGAAAGGTTATGG h tt p s :// . ne j m . o r g / do i / . / N E J M oa2001017 C h i na - CDC - N GGGGAACTTCTCCTGCTAGAAT
CAGACATTTTGCTCTCAAGCTG
TTGCTGCTGCTTGACAGATT h tt p s :// . ne j m . o r g / do i / . / N E J M oa2001017 C ha r i t e - R d R P _ P GTGA R ATGGTCATGTGTGGCGG CA R ATGTTAAA S ACACTATTAGCATA CCAGGTGG W AC R TCATC M GGTGATGC h tt p s :// . w ho . i n t/ do cs / de f au l t - s ou r c e / c o r ona v i r u s e / p r o t o c o l - v - . pd f ? s f v r s n = a9e f c _2 C ha r i t e - R d R P _ P GTGA R ATGGTCATGTGTGGCGG CA R ATGTTAAA S ACACTATTAGCATA
CAGGTGGAACCTCATCAGGAGATGC h tt p s :// . w ho . i n t/ do cs / de f au l t - s ou r c e / c o r ona v i r u s e / p r o t o c o l - v - . pd f ? s f v r s n = a9e f c _2 C ha r i t e - E _ S a r be c o ACAGGTACGTTAATAGTTAATAGCGT
ATATTGCAGCAGTACGCACACA
ACACTAGCCATCCTTACTGCGCTTCG h tt p s :// . w ho . i n t/ do cs / de f au l t - s ou r c e / c o r ona v i r u s e / p r o t o c o l - v - . pd f ? s f v r s n = a9e f c _2 H K U - O R F - n s p TGGGG Y TTTAC R GGTAACCT
AAC R CGCTTAACAAAGCACTC
TAGTTGTGATGC W ATCATGACTAG h tt p s :// . w ho . i n t/ do cs / de f au l t - s ou r c e / c o r ona v i r u s e / pe i r i s - p r o t o c o l - - - . pd f ? s f v r s n = a f c H K U - N TAATCAGACAAGGAACTGATTA
CGAAGGTGTGACTTCCATG
GCAAATTGTGCAATTTGCGG h tt p s :// . w ho . i n t/ do cs / de f au l t - s ou r c e / c o r ona v i r u s e / pe i r i s - p r o t o c o l - - - . pd f ? s f v r s n = a f c W H - N I C - N CGTTTGGTGGACCCTCAGAT
CCCCACTGCGTTCTCCATT
CAACTGGCAGTAACCA h tt p s :// . w ho . i n t/ do cs / de f au l t - s ou r c e / c o r ona v i r u s e / c on v en t i ona l -r t - p c r- f o ll o w ed - b y - s equen c i ng - f o r- de t e c t i on - o f - n c o v -r i r l - na t - i n s t - hea l t h - t. pd f ? s f v r s n = c N II D _2019 - n C O V _ N AAATTTTGGGGACCAGGAAC TGGCACCTGTGTAGGTCAAC ATGTCGCGCATTGGCATGGA h tt p s :// . j s t age . j s t. go . j p / a r t i c l e / y o k en / ad v pub / / ad v pub_ JJ I D . . / _pd f n C o V _ I P ATGAGCTTAGTCCTGTTG
CTCCCTTTGTTGTGTTGT
AGATGTCTTGTGCTGCCGGTA h tt p s :// . w ho . i n t/ do cs / de f au l t - s ou r c e / c o r ona v i r u s e / r ea l - t i m e -r t - p c r- a ss a ys - f o r- t he - de t e c t i on - o f - s a r s - c o v - - i n s t i t u t - pa s t eu r- pa r i s . pd f ? s f v r s n = f c b6_2 n C o V _ I P GGTAACTGGTATGATTTCG
CTGGTCAAGGTTAATATAGG
TCATACAAACCACGCCAGG h tt p s :// . w ho . i n t/ do cs / de f au l t - s ou r c e / c o r ona v i r u s e / r ea l - t i m e -r t - p c r- a ss a ys - f o r- t he - de t e c t i on - o f - s a r s - c o v - - i n s t i t u t - pa s t eu r- pa r i s . pd f ? s f v r s n = f c b6_2 i s c u ss i o n A dd e n d u m W h il e t h e r e a r e m a ny s ou r ce s o f e m p i r i ca l a ss a y v a li d a ti on , on l y a f e w c u rr e n t r e s ou r ce s e x i s t t h a t p r e s e n t p r e d i c ti v e a ss e ss m e n t s . O n li n e r e po s it o r i e s a nd m e ss a g e g r oup s no w o ff e r t h e m ea n s f o r p r e s e n ti ng a nd s h a r i ng i n f o r m a ti on r e l a t e d t o t h e p e rf o r m a n ce o f c u rr e n t a ss a y s . T h i s i n f o r m a ti on ca n t a k e t h e f o r m o f s t a ti c a n a l y s e s ( W h i c h p r e s u m a b l y e m p l oy i n s ili c o p r e d i c ti v e m od e l s f o r p e rf o r m a n ce a ss e ss m e n t ) o r s i m p l y s h a r i ng o f i n f o r m a ti on ( o f t e n i n m a nu s c r i p t f o r m a t ) i n a w o r k s p ace o r t op i c g r oup c h a t . G I S A I D ( S hu , e t a l ., ) , i n a dd iti on t o b e i ng a r e po s it o r y f o r S A R - C o V - g e no m e s , s upp li e s b i o i n f o r m a ti c a n a l y s e s o f t h e d a t a b a s e a nd a s u mm a r y o f a ss a y p e rf o r m a n ce . T h e a ss a y a ss e ss m e n t f o r m a t i s a s t a ti c s li d e w it h s u mm a r y i n f o r m a ti on . T h e n a t u r e o f t h e c on t e n t s upp li e d h e r e h a s v a r i e d ov e r ti m e . E a r l y , t h e s li d e s ho w e d s e l ec t e d a ss a y s a nd li s t e d g e no m e s w it h m i s m a t c h e s t o t h e a ss a y ( a l s o i nd i ca ti ng t h e s p ec i f i c m i s m a t c h e s ) . A s t h e nu m b e r o f g e no m e s h a s g r o w n , t h e li s t o f g e no m e s on t h i s s li d e g r e w , w h il e l o s i ng t h e s p ec i f i c m i s m a t c h i n f o r m a ti on . T h i s w a s s ub s e qu e n tl y r e p l ace d w it h a li s t o f a ss a y s a nd t h e i r m i s m a t c h e s w it h r e s p ec t t o a r e f e r e n ce g e no m e . T h e c u rr e n t d i s p l a y s ho w s a h i s t og r a m o f t h e p e r ce n t o f g e no m e s w it h a m u t a ti on i n t h e p r i m e r r e g i on f o r eac h a ss a y a nd a no t h e r h i s t og r a m w it h t h e p e r ce n t o f g e no m e s w it h a m u t a ti on i n t h e f i v e ’ t e r m i n a l nu c l e o ti d e s . T h i s a n a l y s i s i s d e v e l op e d fr o m B L A S T N s ea r c h e s . I n g e n e r a l , G I S A I D o ff e r s u s e f u l a n a l y s i s , bu t t h e c on t e n t h a s v a r i e d ov e r ti m e a nd t h e p r e s e n t a ti on i s s t a ti c . U pd a t e s t o t h e a n a l y s i s a r e fr e qu e n t bu t unp r e d i c t a b l e . I n c on t r a s t , v i r o l og i ca l . o r g p r ov i d e s a f o r u m f o r d i s c u ss i on o f v i r u s b i o i n f o r m a ti c s a n a l y s i s . I n f o r m a ti on m a y b e p r e s e n t e d i n a s t a ti c f o r m a t , s u c h a s a m a nu s c r i p t . W h il e t h i s e n a b l e s c onv e r s a ti on on t h e t op i c , t h e a n a l y s i s i s s t a ti c , a nd i n f o r m a ti on i s on l y upd a t e d a s o f t e n a s po s t e r s c hoo s e t o up l o a d t o t h e s it e . O u r w e b - b a s e d a pp li ca ti on p r ov i d e s a n a lt e r n a ti v e t h a t i s i n t e r ac ti v e a nd upd a t e d d a il y w it h r ece n tl y d e po s it e d g e no m e s . O u r a n a l y s i s u s e s a n i n s ili c o a l go r it h m t h a t i s p r e d i c ti v e o f a ss a y p e rf o r m a n ce . A g l a n ce a t t h e h ea t m a p w ill qu i c k l y r e v ea l p a tt e r n s a nd t r e nd s i n t h e p e rf o r m a n ce o f t h e a ss a y s . I t i s i mm e d i a t e l y e v i d e n t t h a t t w o o f t h e C h a r it é a ss a y s f o r R d R P a r e i m p e rf ec t m a t c h e s t o a ll s t r a i n s o f S A R S - C o V - ( F i g . S ) . F i gu r e s S a nd S s ho w t h e a li gn m e n t s o f t h e p r ob e a nd p r ob e a ss a y (r e s p ec ti v e l y ) t o T u r k e y / - A nk a r a g e no m e a s a n e x a m p l e . I t i s e v i d e n t t h a t bo t h a ss a y s h a v e on e c on s i s t e n t m i s m a t c h i n t h e r e v e r s e p r i m e r a nd t h e P a ss a y h a s t w o m i s m a t c h e s i n t h e p r ob e . T h e s e m i s m a t c h e s o cc u r on a ll g e no m e s v i s u a li ze d . T h e a li gn m e n t s ho w s t h a t t h e C h a r it é r e v e r s e p r i m e r h a s a G o r a C oppo s it e a T i n t h e g e no m e . E v i d e n tl y , t h e C h a r it é a ss a y h a s b ee n d e s i gn e d w it h a d e g e n e r a t e b a s e a t t h i s po s iti on w h e r e t h e S A R S - C o V - g e no m e a pp ea r s t o c on s i s t e n tl y h a v e a T . T h i s d e s i gn a pp ea r s t o b e a r e li c o f t h e f ac t t h a t t h i s p r i m e r w a s i n iti a ll y d e s i gn e d f o r t e s ti ng f o r S A R S o r b a t - S A R S c o r on a v i r u s e s ( V og e l s e t a l ., ) . A no t h e r p a tt e r n t h a t i s e v i d e n t fr o m t h e h ea t m a p ( F i g . S ) i s t h a t t h e N a ss a y fr o m U S A C D C h a s a nu m b e r o f s t r a i n s w it h a s i ng l e m i s m a t c h a s w e ll . T h i s i s a T b a s e i n t h e f o r w a r d p r i m e r t h a t m i s m a t c h e s w it h a G on t h e t a r g e t g e no m e ( V og e l s e t a l ., ) ( F i g S ) . W h il e m o s t s t r a i n s o f v i r u s do no t h a v e t h i s m i s m a t c h w it h t h e a ss a y a t t h i s b a s e , a ce r t a i n nu m b e r o f s t r a i n s a pp ea r t o h a v e ac qu i r e d t h i s m u t a ti on . T h e t r ee r e v ea l s t h a t t h e r e i s a c l a d e w it h s t r a i n s t h a t h a v e ac qu i r e d t h i s m u t a ti on r e s u lti ng i n t h e m i s m a t c h t o t h e N a ss a y ( F i g . S ) . M o s t o f t h e s e s t r a i n s o r i g i n a t e fr o m A s i a a nd O cea n i a , w it h a f e w i n N o r t h A m e r i ca n a nd E u r op e . T h e m a j o r it y o f t h e s e s t r a i n s w e r e c o ll ec t e d i n ea r l y M a r c h , c o rr obo r a ti ng a c o mm on o r i g i n f o r t h i s c l a d e . T h e r e m a i nd e r o f s t r a i n s w it h t h i s m u t a ti on do no t a pp ea r t o s h a r e a n obv i ou s l y d e li n ea t e d c l a d e ( F i g . S ) , bu t m a ny o f t h e s e o r i g i n a t e fr o m A s i a a nd O cea n i a a nd m a ny h a v e a c o ll ec ti on d a t e o f l a t e F e b r u a r y a nd ea r l y M a r c h ( w it h s o m e a s ea r l y a s J a nu a r y ) s ugg e s ti ng a n ea r li e r o r i g i n f o r t h e m u t a ti on i n t h i s g r oup . O t h e r s m a ll c l u s t e r s o f m u t a ti on s a bound . T h e r e i s a s m a ll c l a d e o f s e qu e n ce s t h a t ca u s e f a il u r e o f t h e N a ss a y du e t o m i s m a t c h e s on t h e p r ob e ( F i g . S ) . T h e s e b a s e s ( A CC ) a r e on t h e f i r s t t h r ee b a s e s o f t h e p r ob e oppo s it e T GA ( F i g . S ) . W h il e C - A i s a lit e r a l m i s m a t c h , t h e r e m a i n i ng t w o b a s e s s hou l d no t p a i r du e t o t h e t h e r m odyn a m i c un f a vo r a b ilit y o f t h e i r po s iti on on t h e e nd o f a n o li go . T h e m a j o r it y o f t h e s e s e qu e n ce s o r i g i n a t e i n O cea n i a i n ea r l y M a r c h . T h e r e do e s no t a pp ea r t o b e n ea r by c l a d e s w it h m i s m a t c h e s , s o t h i s g r oup a pp ea r s t o h a v e e m e r g e d f a i r l y r a p i d l y . T h i s i s t h e t yp e o f d e v e l op m e n t t h a t w ill po t e n ti a ll y l ea d t o ob s o l e s ce n ce o f a ss a y s , a nd s o t h e a b ilit y t o t r ac k t h e s e t yp e s o f e v e n t s i s p a r ti c u l a r l y v a l u a b l e . H e r e w e h a v e p r e s e n t e d a w e b - b a s e d t oo l f o r v i s u a l a nd qu a n tit a ti v e p r e d i c ti v e a ss e ss m e n t a nd v a li d a ti on o f P CR a ss a y s t h a t a r e d e s i gn e d t o d e t ec t S A R S - C o V - . T h e t oo l w e h a v e d e s c r i b e d p r ov i d e s a n ea s y - t o -r ea d v i s u a l r e p r e s e n t a ti on o f a ss a y p e rf o r m a n ce w it h r e s p ec t t o t h e t a r g e t g e no m e s a nd t h e e vo l u ti on a r y r e l a ti on s h i p b e t w ee n t h e g e no m e s . T h i s w e b a pp li ca ti on w ill t hu s o ff e r u tilit y t o po li c y m a k e r s , h ea lt h o ff i c i a l s , a nd o t h e r s t a k e ho l d e r s i n m a k i ng d ec i s i on s a bou t C OV I D - t e s ti ng a nd t r ac k i ng o f t h e S A R S - C o V - v i r u s . F i gu r e S . F i gu r e S F i gu r e S F i gu r e S F i gu r e S F i gu r e S F i gu r e S F i gu r e S e f e r e n ce s C D C ( ) - N ov e l C o r on a v i r u s ( - n C o V ) R ea l - T i m e R T - P CR D i a gno s ti c P a n e l F o r E m e r g e n c y U s e O n l y : I n s t r u c ti on s f o r U s e . C e n t e r s f o r D i s ea s e C on t r o l a nd P r e v e n ti on . A cce ss e d : M a y , . h tt p s :// . f d a . gov / m e d i a / / do w n l o a d C l a r k , K . e t a l . ( ) G e n B a nk . N u c l e i c A c i d s R e s . , , D D . C o r m a n , V . e t a l . ( ) D i a gno s ti c d e t ec ti on o f - n C o V by r ea l - ti m e R T - P CR . P r o t o c o l . A cce ss e d : M a y , . h tt p s :// . w ho . i n t/ do c s / d e f a u lt - s ou r ce / c o r on a v i r u s e / p r o t o c o l - v2 - . pd f G a n s , J . D . a nd W o li n s ky , M . ( ) I m p r ov e d a ss a y - d e p e nd e n t s ea r c h i ng o f nu c l e i c ac i d s e qu e n ce d a t a b a s e s . N u c l e i c A c i d s R e s . , , . H o ll a nd , M . e t a l . ( ) B i o L a bo r o : A b i o i n f o r m a ti c s s y s t e m f o r d e t ec ti ng m o l ec u l a r a ss a y s i gn a t u r e e r o s i on a nd d e s i gn i ng n e w a ss a y s i n r e s pon s e t o e m e r g i ng a nd r ee m e r g i ng p a t hog e n s . b i o R x i v , . . . . K r e f t , Ł . e t a l . ( ) P hy D : a phy l og e n e ti c t r ee v i e w e r w it h e x t e nd e d phy l o X M L s uppo r t f o r f un c ti on a l g e no m i c s d a t a v i s u a li za ti on . B i o i n f o r m a ti c s , , . S h a ky a , M . e t a l . ( ) S t a nd a r d i ze d phy l og e n e ti c a nd m o l ec u l a r e vo l u ti on a r y a n a l y s i s a pp li e d t o s p ec i e s ac r o ss t h e m i c r ob i a l t r ee o f li f e . S c i . R e p . , , . S hu , Y . a nd M c C a u l e y , J . ( ) G I S A I D : G l ob a l i n iti a ti v e on s h a r i ng a ll i n f l u e n za d a t a - fr o m v i s i on t o r ea lit y . E u r o Su r ve ill . , . v i r o l og i ca l . o r g . W e b s it e : v i r o l og i ca l . o r g V og e l s , C . B ., e t a l . ( ) A n a l y ti ca l s e n s iti v it y a nd e ff i c i e n c y c o m p a r i s on s o f S A R S - C OV - q R T - P CR a ss a y s . m e d R x i vv