Volker A. Eyrich

EVA: continuous automatic evaluation of protein structure prediction servers

UNLABELLED Evaluation of protein structure prediction methods is difficult and time-consuming. Here, we describe EVA, a web server for assessing protein structure prediction methods, in an automated, continuous and large-scale fashion. Currently, EVA evaluates the performance of a variety of prediction methods available through the internet. Every week, the sequences of the latest experimentally determined protein structures are sent to prediction servers, results are collected, performance is evaluated, and a summary is published on the web. EVA has so far collected data for more than 3000 protein chains. These results may provide valuable insight to both developers and users of prediction methods. AVAILABILITY http://cubic.bioc.columbia.edu/eva. CONTACT eva@cubic.bioc.columbia.edu

EVA: evaluation of protein structure prediction servers

EVA (http://cubic.bioc.columbia.edu/eva/) is a web server for evaluation of the accuracy of automated protein structure prediction methods. The evaluation is updated automatically each week, to cope with the large number of existing prediction servers and the constant changes in the prediction methods. EVA currently assesses servers for secondary structure prediction, contact prediction, comparative protein structure modelling and threading/fold recognition. Every day, sequences of newly available protein structures in the Protein Data Bank (PDB) are sent to the servers and their predictions are collected. The predictions are then compared to the experimental structures once a week; the results are published on the EVA web pages. Over time, EVA has accumulated prediction results for a large number of proteins, ranging from hundreds to thousands, depending on the prediction method. This large sample assures that methods are compared reliably. As a result, EVA provides useful information to developers as well as users of prediction methods.

EVA: Large‐scale analysis of secondary structure prediction

EVA is a web‐based server that evaluates automatic structure prediction servers continuously and objectively. Since June 2000, EVA collected more than 20,000 secondary structure predictions. The EVA sets sufficed to conclude that the field of secondary structure prediction has advanced again. Accuracy increased substantially in the 1990s through using evolutionary information taken from the divergence of proteins in the same structural family. Recently, the evolutionary information resulting from improved searches and larger databases has again boosted prediction accuracy by more than 4% to its current height around 76% of all residues predicted correctly in one of the three states: helix, strand, or other. The best current methods solved most of the problems raised at earlier CASP meetings: All good methods now get segments right and perform well on strands. Is the recent increase in accuracy significant enough to make predictions even more useful? We believe the answer is affirmative. What is the limit of prediction accuracy? We shall see. All data are available through the EVA web site at {cubic.bioc.columbia.edu/eva/}. The raw data for the results presented are available at {eva}/sec/bup_common/2001_02_22/. Proteins 2001;Suppl 5:192–199.

Protein tertiary structure prediction using a branch and bound algorithm

We report a new method for predicting protein tertiary structure from sequence and secondary structure information. The predictions result from global optimization of a potential energy function, including van der Waals, hydrophobic, and excluded volume terms. The optimization algorithm, which is based on the αBB method developed by Floudas and coworkers (Costas and Floudas, J Chem Phys 1994;100:1247–1261), uses a reduced model of the protein and is implemented in both distance and dihedral angle space, enabling a side‐by‐side comparison of methodologies. For a set of eight small proteins, representing the three basic types—all α, all β, and mixed α/β—the algorithm locates low‐energy native‐like structures (less than 6Å root mean square deviation from the native coordinates) starting from an unfolded state. Serial and parallel implementations of this methodology are discussed. Proteins 1999;35:41–57.

CAFASP3 in the spotlight of EVA

We have analysed fold recognition, secondary structure and contact prediction servers from CAFASP3. This assessment was carried out in the framework of the fully automated, web‐based evaluation server EVA. Detailed results are available at http://cubic.bioc.columbia.edu/eva/cafasp3/. We observed that the sequence‐unique targets from CAFASP3/CASP5 were not fully representative for evaluating performance. For all three categories, we showed how careless ranking might be misleading. We compared methods from all categories to experts in secondary structure and contact prediction and homology modellers to fold recognisers. While the secondary structure experts clearly outperformed all others, the contact experts appeared to outperform only novel fold methods. Automatic evaluation servers are good at getting statistics right and at using these to discard misleading ranking schemes. We challenge that to let machines rule where they are best might be the best way for the community to enjoy the tremendous benefit of CASP as a unique opportunity for brainstorming. Proteins 2003;53:548–560.

New tools and expanded data analysis capabilities at the protein structure prediction center

We outline the main tasks performed by the Protein Structure Prediction Center in support of the CASP7 experiment and provide a brief review of the major measures used in the automatic evaluation of predictions. We describe in more detail the software developed to facilitate analysis of modeling success over and beyond the available templates and the adopted Java‐based tool enabling visualization of multiple structural superpositions between target and several models/templates. We also give an overview of the CASP infrastructure provided by the Center and discuss the organization of the results web pages available through http://predictioncenter.org. Proteins 2007.

EVAcon: a protein contact prediction evaluation service

Here we introduce EVAcon, an automated web service that evaluates the performance of contact prediction servers. Currently, EVAcon is monitoring nine servers, four of which are specialized in contact prediction and five are general structure prediction servers. Results are compared for all newly determined experimental structures deposited into PDB (∼5–50 per week). EVAcon allows for a precise comparison of the results based on a system of common protein subsets and the commonly accepted evaluation criteria that are also used in the corresponding category of the CASP assessment. EVAcon is a new service added to the functionality of the EVA system for the continuous evaluation of protein structure prediction servers. The new service is accesible from any of the three EVA mirrors: PDG (CNB-CSIC, Madrid) (); CUBIC (Columbia University, NYC) (); and Sali Lab (UCSF, San Francisco) ().

META-PP: single interface to crucial prediction servers

The META-PP server (http://cubic.bioc.columbia.edu/meta/) simplifies access to a battery of public protein structure and function prediction servers by providing a common and stable web-based interface. The goal is to make these powerful and increasingly essential methods more readily available to nonexpert users and the bioinformatics community at large. At present META-PP provides access to a selected set of high-quality servers in the areas of comparative modelling, threading/fold recognition, secondary structure prediction and more specialized fields like contact and function prediction.

Protein structure prediction using a combination of sequence‐based alignment, constrained energy minimization, and structural alignment

We present a novel approach to protein structure prediction in which fold recognition techniques are combined with ab initio folding methods. Based on the predicted secondary structure, one of two different protocols is followed. For mostly α‐proteins, global optimization and sampling of a statistical energy function is used to generate many low‐energy structures; these structures are then screened against a fold library. Any structural matches are then selected for further refinement. For proteins predicted to have significant β‐content, sequence and secondary structure‐based alignment is used to identify candidate templates; spatial constraints are then extracted from these templates and used, along with the statistical energy function, in the global sampling and optimization program. Successes and failures of both protocols are discussed. Proteins 2001;Suppl 5:133–139.

System for accepting server predictions in CASP6

We describe the new CASP system for collecting and verifying predictions generated by servers. The system was developed to ensure reliable execution of the server assessment part of CASP, with particular emphasis on data consistency. Following the principle that predictions should not be modified by anyone but their authors and to allow a later meaningful assessment, submissions are now verified for correctness of format and contents within the strict 48 hour CASP deadlines for this type of submission. This article also provides an overview of the rules governing server participation in CASP6 and some statistics pertaining to servers in CASP6. Proteins 2005;Suppl 7:24–26.