Krzysztof Fidelis

Critical assessment of methods of protein structure prediction (CASP)—Round 6

This article is an introduction to the special issue of the journal PROTEINS, dedicated to the ninth Critical Assessment of Structure Prediction (CASP) experiment to assess the state of the art in protein structure modeling. The article describes the conduct of the experiment, the categories of prediction included, and outlines the evaluation and assessment procedures. Methods for modeling protein structure continue to advance, although at a more modest pace than in the early CASP experiments. CASP developments of note are indications of improvement in model accuracy for some classes of target, an improved ability to choose the most accurate of a set of generated models, and evidence of improvement in accuracy for short “new fold” models. In addition, a new analysis of regions of models not derivable from the most obvious template structure has revealed better performance than expected. Proteins 2011;

A modified definition of Sov, a segment‐based measure for protein secondary structure prediction assessment

We present a measure for the evaluation of secondary structure prediction methods that is based on secondary structure segments rather than individual residues. The algorithm is an extension of the segment overlap measure Sov, originally defined by Rost et al. (J Mol Biol 1994;235:13–26). The new definition of Sov corrects the normalization procedure and improves Sovs ability to discriminate between similar and dissimilar segment distributions. The method has been comprehensively tested during the second Critical Assessment of Techniques for Protein Structure Prediction (CASP2). Here, we describe the underlying concepts, modifications to the original definition, and their significance. Proteins 1999;34:220–223. Published 1999 Wiley‐Liss, Inc.

Critical assessment of methods of protein structure prediction—Round VII

This paper is an introduction to the supplemental issue of the journal PROTEINS, dedicated to the seventh CASP experiment to assess the state of the art in protein structure prediction. The paper describes the conduct of the experiment, the categories of prediction included, and outlines the evaluation and assessment procedures. Highlights are improvements in model accuracy relative to that obtainable from knowledge of a single best template structure; convergence of the accuracy of models produced by automatic servers toward that produced by human modeling teams; the emergence of methods for predicting the quality of models; and rapidly increasing practical applications of the methods. Proteins 2007.

Processing and Analysis of CASP3 Protein Structure Predictions

Livermore Prediction Center provides basic infrastructure for the CASP (Critical Assessment of Structure Prediction) experiments, including prediction processing and verification servers, a system of prediction evaluation tools, and interactive numerical and graphical displays. Here we outline the essentials of our approach, with discussion of the superposition procedures, definitions of basic measures, and descriptions of new methods developed to analyze predictions. Our primary focus is on the evaluation of threedimensional models and secondary structure predictions. To put the results of the three prediction experiments held to date on the same footing, the latest CASP3 evaluation criteria were retrospectively applied to both CASP1 and CASP2 predictions. Finally, we give an overview of our website (http://PredictionCenter.llnl.gov), which makes the target structures, predictions, and the evaluation system accessible to the community. Proteins Suppl 1999;3:22–29. Published 1999 Wiley‐Liss, Inc.

Critical assessment of methods of protein structure prediction (CASP) - round x: Critical Assessment of Structure Prediction

This article is an introduction to the special issue of the journal PROTEINS, dedicated to the tenth Critical Assessment of Structure Prediction (CASP) experiment to assess the state of the art in protein structure modeling. The article describes the conduct of the experiment, the categories of prediction included, and outlines the evaluation and assessment procedures. The 10 CASP experiments span almost 20 years of progress in the field of protein structure modeling, and there have been enormous advances in methods and model accuracy in that period. Notable in this round is the first sustained improvement of models with refinement methods, using molecular dynamics. For the first time, we tested the ability of modeling methods to make use of sparse experimental three‐dimensional contact information, such as may be obtained from new experimental techniques, with encouraging results. On the other hand, new contact prediction methods, though holding considerable promise, have yet to make an impact in CASP testing. The nature of CASP targets has been changing in recent CASPs, reflecting shifts in experimental structural biology, with more irregular structures, more multi‐domain and multi‐subunit structures, and less standard versions of known folds. When allowance is made for these factors, we continue to see steady progress in the overall accuracy of models, particularly resulting from improvement of non‐template regions. Proteins 2014; 82(Suppl 2):1–6.

Progress over the first decade of CASP experiments.

CASP has now completed a decade of monitoring the state of the art in protein structure prediction. The quality of structure models produced in the latest experiment, CASP6, has been compared with that in earlier CASPs. Significant although modest progress has again been made in the fold recognition regime, and cumulatively, progress in this area is impressive. Models of previously unknown folds again appear to have modestly improved, and several mixed α/β structures have been modeled in a topologically correct manner. Progress remains hard to detect in high sequence identity comparative modeling, but server performance in this area has moved forward. Proteins 2005;61:225–236.

A Sliding Clamp Model for the Rad1 Family of Cell Cycle Checkpoint Proteins

We explored the possibility that the Rad1 family members have more distant homology with other proteins of Rad1 family were embedded into the Rad1 Mm sequence using BLOCKs, and this cobbled sequence was used as the probe in a PSI-BLAST search. An initial weak We have identified an unexpected structural similarity match was amplified after several iterations, and PCNA between six members of the Rad1 cell cycle checkpoint (S. cerevisiae, and other species) was retrieved as a family and the DNA sliding clamp protein PCNA. Like candidate homolog with nearly end-to-end sequence its prokaryotic structural homolog, the ␤ subunit of DNA alignment. However, PCNA and the probe sequence polymerase III, PCNA facilitates genome replication by were only 15% identical, making it uncertain whether encircling the DNA helix and tethering DNA polymerase these protein families are truly related. to its substrate. Our Rad1 model predicts that eukaryotic We therefore sought independent verification of this cells contain a second PCNA-like structure, one that hypothesis using the methods of fold recognition and may be critical to the mechanisms coupling DNA repair comparative modeling. Using an empirically derived fit-and DNA synthesis to a mitotic checkpoint. ness function, fold recognition (threading) evaluates the The Rad1 family of proteins, including Rec1 of Usti-compatibility of a new sequence with templates in the lago maydis and its distant relatives Rad1 sp (S. pombe) library of known folds. Threading was performed for and Rad17 sc (S. cerevisiae), function in both DNA repair each of the six homologs separately. In each case, only and cell cycle control (Lydall and Weinert, 1995; Onel the PCNA fold was always among the top ranking hits. Sequence comparisons reveal established false positive threshold (Fischer and Eisen-conserved blocks of amino acids between these six berg, 1996). proteins, but overall there is less than 30% identity be-The hypothesis was further explored by building and tween any two sequences. Similarity to other known evaluating an all atom model of Rad1 Mm. Using the PSI-proteins is not obvious, and a common biochemical BLAST alignment as a guide, Rad1 Mm amino acid side chains were placed on the peptide backbone of the function remains unclear. Rad1 homologs include those from mouse (Mm_Rad1), fruit fly (Dm_Rad1), smut fungus (Um_Rec1), and fission yeast (Sp_Rad1). PCNA sequences are from budding yeast (Sc_PCNA), malaria parasite (Pf_PCNA), bac-ulovirus (Ac_PCNA), and human (Hs_PCNA). Rad1 family sequence alignment with PCNA proteins relied on Rad1 Mm modeling …

Evaluation of disorder predictions in CASP9

Lack of stable three‐dimensional structure, or intrinsic disorder, is a common phenomenon in proteins. Naturally, unstructured regions are proven to be essential for carrying function by many proteins, and therefore identification of such regions is an important issue. CASP has been assessing the state of the art in predicting disorder regions from amino acid sequence since 2002. Here, we present the results of the evaluation of the disorder predictions submitted to CASP9. The assessment is based on the evaluation measures and procedures used in previous CASPs. The balanced accuracy and the Matthews correlation coefficient were chosen as basic measures for evaluating the correctness of binary classifications. The area under the receiver operating characteristic curve was the measure of choice for evaluating probability‐based predictions of disorder. The CASP9 methods are shown to perform slightly better than the CASP7 methods but not better than the methods in CASP8. It was also shown that capability of most CASP9 methods to predict disorder decreases with increasing minimum disorder segment length. Proteins 2011;.

Evaluation of template-based models in CASP8 with standard measures

The strategy for evaluating template‐based models submitted to CASP has continuously evolved from CASP1 to CASP5, leading to a standard procedure that has been used in all subsequent editions. The established approach includes methods for calculating the quality of each individual model, for assigning scores based on the distribution of the results for each target and for computing the statistical significance of the differences in scores between prediction methods. These data are made available to the assessor of the template‐based modeling category, who uses them as a starting point for further evaluations and analyses. This article describes the detailed workflow of the procedure, provides justifications for a number of choices that are customarily made for CASP data evaluation, and reports the results of the analysis of template‐based predictions at CASP8. Proteins 2009.

Assessment of Progress Over the CASP Experiments

The quality of structure models produced in the CASP5 experiment has been compared with that in earlier CASPs. The most significant progress is in the fold recognition regime, where the development of meta‐servers has allowed more accurate consensus models to be generated. In contrast to this, there is little evidence of progress in producing more accurate comparative models, particularly those based on sequence identities > 30%. For comparative models based on low‐sequence identity and for fold recognition models, accuracy depends primarily on the fraction of the target structure that is similar to an available template, and the quality of the alignment. Overall, these results indicate that there are still no effective methods of improving model quality beyond that obtained by successfully copying a template structure. For models of proteins with previously unknown folds, there appears to be a pause in the previous consistent improvement. There is some evidence that more groups are producing top‐quality models, however. Although specific progress between successive experiments is sometimes difficulty to identify, over the history of all the CASPs there has been steady, if sometimes slow, progress in all modeling regimes. Proteins 2003;53:585–595.