Paola Gramatica

QSAR Modeling: Where have you been? Where are you going to?

Quantitative structure-activity relationship modeling is one of the major computational tools employed in medicinal chemistry. However, throughout its entire history it has drawn both praise and criticism concerning its reliability, limitations, successes, and failures. In this paper, we discuss (i) the development and evolution of QSAR; (ii) the current trends, unsolved problems, and pressing challenges; and (iii) several novel and emerging applications of QSAR modeling. Throughout this discussion, we provide guidelines for QSAR development, validation, and application, which are summarized in best practices for building rigorously validated and externally predictive QSAR models. We hope that this Perspective will help communications between computational and experimental chemists toward collaborative development and use of QSAR models. We also believe that the guidelines presented here will help journal editors and reviewers apply more stringent scientific standards to manuscripts reporting new QSAR studies, as well as encourage the use of high quality, validated QSARs for regulatory decision making.

Joint algal toxicity of 16 dissimilarly acting chemicals is predictable by the concept of independent action

For a predictive assessment of the aquatic toxicity of chemical mixtures, two competing concepts are available: concentration addition and independent action. Concentration addition is generally regarded as a reasonable expectation for the joint toxicity of similarly acting substances. In the opposite case of dissimilarly acting toxicants the choice of the most appropriate concept is a controversial issue. In tests with freshwater algae we therefore studied the extreme situation of multiple exposure to chemicals with strictly different specific mechanisms of action. Concentration response analyses were performed for 16 different biocides, and for mixtures containing all 16 substances in two different concentration ratios. Observed mixture toxicity was compared with predictions, calculated from the concentration response functions of individual toxicants by alternatively applying both concepts. The assumption of independent action yielded accurate predictions, irrespective of the mixture ratio or the effect level under consideration. Moreover, results even demonstrate that dissimilarly acting chemicals can show significant joint effects, predictable by independent action, when combined in concentrations below individual NOEC values, statistically estimated to elicit insignificant individual effects of only 1%. The alternative hypothesis of concentration addition resulted in overestimation of mixture toxicity, but differences between observed and predicted effect concentrations did not exceed a factor of 3.2. This finding complies with previous studies, which indicated near concentration-additive action of mixtures of dissimilarly acting substances. Nevertheless, with the scientific objective to predict multi-component mixture toxicity with the highest possible accuracy, concentration addition obviously is no universal solution. Independent action proves to be superior where components are well known to interact specifically with different molecular target sites, and provided that reliable statistical estimates of low toxic effects of individual mixture constituents can be given. With a regulatory perspective, however, fulfilment of both conditions may be regarded as an extraordinary situation, and hence concentration addition may be defendable as a pragmatic and precautionary default assumption.

Predicting the joint algal toxicity of multi-component s-triazine mixtures at low-effect concentrations of individual toxicants.

Herbicidal s-triazines are widespread contaminants of surface waters. They are highly toxic to algae and other primary producers in aquatic systems. This results from their specific interference with photosynthetic electron transport. Risk assessment for aquatic biota has to consider situations of simultaneous exposure to various of these toxicants. In tests with freshwater algae we predicted and determined the toxicity of multiple mixtures of 18 different s-triazines. The toxicity parameter was the inhibition of reproduction of Scenedesmus vacuolatus. Concentration-response analyses were performed for single toxicants and for mixtures containing all 18 s-triazines in two different concentration ratios. Experiments were designed to allow a valid statistical description of the entire concentration-response relationships, including the low concentration range down to EC1. Observed effects and effect concentrations of mixtures were compared to predictions of mixture toxicity. Predictions were calculated from the concentration-response functions of individual s-triazines by applying the concepts of concentration addition and independent action (response addition) alternatively. Predictions based on independent action tend to underestimate the overall toxicity of s-triazine mixtures. In contrast, the concept of concentration addition provides highly accurate predictions of s-triazine mixture toxicity, irrespective of the effect level under consideration and the concentration ratio of the mixture components. This also holds true when the mixture components are present in concentrations below their individual NOEC values. Concentrations statistically estimated to elicit non-significant effects of only 1% still contribute to the overall toxicity. When present in a multi-component mixture they can co-operate to give a severe joint effect. Applicability of the findings obtained with s-triazines to mixtures of other contaminants in aquatic systems and consequences for risk assessment procedures are discussed.

Real external predictivity of QSAR models: how to evaluate it? Comparison of different validation criteria and proposal of using the concordance correlation coefficient.

The main utility of QSAR models is their ability to predict activities/properties for new chemicals, and this external prediction ability is evaluated by means of various validation criteria. As a measure for such evaluation the OECD guidelines have proposed the predictive squared correlation coefficient Q(2)(F1) (Shi et al.). However, other validation criteria have been proposed by other authors: the Golbraikh-Tropsha method, r(2)(m) (Roy), Q(2)(F2) (Schüürmann et al.), Q(2)(F3) (Consonni et al.). In QSAR studies these measures are usually in accordance, though this is not always the case, thus doubts can arise when contradictory results are obtained. It is likely that none of the aforementioned criteria is the best in every situation, so a comparative study using simulated data sets is proposed here, using threshold values suggested by the proponents or those widely used in QSAR modeling. In addition, a different and simple external validation measure, the concordance correlation coefficient (CCC), is proposed and compared with other criteria. Huge data sets were used to study the general behavior of validation measures, and the concordance correlation coefficient was shown to be the most restrictive. On using simulated data sets of a more realistic size, it was found that CCC was broadly in agreement, about 96% of the time, with other validation measures in accepting models as predictive, and in almost all the examples it was the most precautionary. The proposed concordance correlation coefficient also works well on real data sets, where it seems to be more stable, and helps in making decisions when the validation measures are in conflict. Since it is conceptually simple, and given its stability and restrictiveness, we propose the concordance correlation coefficient as a complementary, or alternative, more prudent measure of a QSAR model to be externally predictive.

Structure/Response Correlations and Similarity/Diversity Analysis by GETAWAY Descriptors. 2. Application of the Novel 3D Molecular Descriptors to QSAR/QSPR Studies

In a previous paper the theory of the new molecular descriptors called GETAWAY (GEometry, Topology, and Atom-Weights AssemblY) was explained. These descriptors have been proposed with the aim of matching 3D-molecular geometry, atom relatedness, and chemical information. In this paper prediction ability in structure-property correlations of GETAWAY descriptors has been tested extensively by analyzing the regressions of these descriptors for selected properties of some reference compound classes. Moreover, the general performance of the new descriptors in QSAR/QSPR has been evaluated with respect to other well-known sets of molecular descriptors.

Real External Predictivity of QSAR Models. Part 2. New Intercomparable Thresholds for Different Validation Criteria and the Need for Scatter Plot Inspection

The evaluation of regression QSAR model performance, in fitting, robustness, and external prediction, is of pivotal importance. Over the past decade, different external validation parameters have been proposed: Q(F1)(2), Q(F2)(2), Q(F3)(2), r(m)(2), and the Golbraikh-Tropsha method. Recently, the concordance correlation coefficient (CCC, Lin), which simply verifies how small the differences are between experimental data and external data set predictions, independently of their range, was proposed by our group as an external validation parameter for use in QSAR studies. In our preliminary work, we demonstrated with thousands of simulated models that CCC is in good agreement with the compared validation criteria (except r(m)(2)) using the cutoff values normally applied for the acceptance of QSAR models as externally predictive. In this new work, we have studied and compared the general trends of the various criteria relative to different possible biases (scale and location shifts) in external data distributions, using a wide range of different simulated scenarios. This study, further supported by visual inspection of experimental vs predicted data scatter plots, has highlighted problems related to some criteria. Indeed, if based on the cutoff suggested by the proponent, r(m)(2) could also accept not predictive models in two of the possible biases (location, location plus scale), while in the case of scale shift bias, it appears to be the most restrictive. Moreover, Q(F1)(2) and Q(F2)(2) showed some problems in one of the possible biases (scale shift). This analysis allowed us to also propose recalibrated, and intercomparable for the same data scatter, new thresholds for each criterion in defining a QSAR model as really externally predictive in a more precautionary approach. An analysis of the results revealed that the scatter plot of experimental vs predicted external data must always be evaluated to support the statistical criteria values: in some cases high statistical parameter values could hide models with unacceptable predictions.

QSARINS: A new software for the development, analysis and validation of QSAR MLR models

QSARINS (QSAR‐INSUBRIA) is a new software for the development and validation of multiple linear regression Quantitative Structure‐Activity Relationship (QSAR) models by Ordinary Least Squares method and Genetic Algorithm for variable selection. This program is mainly focused on the external validation of QSAR models. Various tools for explorative analysis of the datasets by Principal Component Analysis, prereduction of input molecular descriptors, splitting of datasets in training and prediction sets, detection of outliers and interpolated or extrapolated predictions, internal and external validation by different parameters, consensus modeling and various plots for visualizations are implemented. QSARINS is a user‐friendly platform for QSAR modeling in agreement with the OECD Principles and for the analysis of the reliability of the obtained predicted data. The Insubria Persistent Bioaccumulative and Toxic (PBT) Index model for the prediction of the cumulative behavior of new chemicals as PBTs is implemented. Additionally, QSARINS allows the user to validate single models, predeveloped using also different software.

Combinatorial QSAR Modeling of Chemical Toxicants Tested against Tetrahymena pyriformis

Selecting most rigorous quantitative structure-activity relationship (QSAR) approaches is of great importance in the development of robust and predictive models of chemical toxicity. To address this issue in a systematic way, we have formed an international virtual collaboratory consisting of six independent groups with shared interests in computational chemical toxicology. We have compiled an aqueous toxicity data set containing 983 unique compounds tested in the same laboratory over a decade against Tetrahymena pyriformis. A modeling set including 644 compounds was selected randomly from the original set and distributed to all groups that used their own QSAR tools for model development. The remaining 339 compounds in the original set (external set I) as well as 110 additional compounds (external set II) published recently by the same laboratory (after this computational study was already in progress) were used as two independent validation sets to assess the external predictive power of individual models. In total, our virtual collaboratory has developed 15 different types of QSAR models of aquatic toxicity for the training set. The internal prediction accuracy for the modeling set ranged from 0.76 to 0.93 as measured by the leave-one-out cross-validation correlation coefficient ( Q abs2). The prediction accuracy for the external validation sets I and II ranged from 0.71 to 0.85 (linear regression coefficient R absI2) and from 0.38 to 0.83 (linear regression coefficient R absII2), respectively. The use of an applicability domain threshold implemented in most models generally improved the external prediction accuracy but at the same time led to a decrease in chemical space coverage. Finally, several consensus models were developed by averaging the predicted aquatic toxicity for every compound using all 15 models, with or without taking into account their respective applicability domains. We find that consensus models afford higher prediction accuracy for the external validation data sets with the highest space coverage as compared to individual constituent models. Our studies prove the power of a collaborative and consensual approach to QSAR model development. The best validated models of aquatic toxicity developed by our collaboratory (both individual and consensus) can be used as reliable computational predictors of aquatic toxicity and are available from any of the participating laboratories.

Validated QSAR Prediction of OH Tropospheric Degradation of VOCs: Splitting into Training−Test Sets and Consensus Modeling

The rate constant for hydroxyl radical tropospheric degradation of 460 heterogeneous organic compounds is predicted by QSAR modeling. The applied Multiple Linear Regression is based on a variety of theoretical molecular descriptors, selected by the Genetic Algorithms-Variable Subset Selection (GA-VSS) procedure. The models were validated for predictivity by both internal and external validation. For the external validation two splitting approaches, D-optimal Experimental Design and Kohonen Artificial Neural Networks (K-ANN), were applied to the original data set to compare the two methodologies. We emphasize that external validation is the only way to establish a reliable QSAR model for predictive purposes. Predicted data by consensus modeling from different models are also proposed.

Statistically Validated QSARs, Based on Theoretical Descriptors, for Modeling Aquatic Toxicity of Organic Chemicals in Pimephales promelas (Fathead Minnow)

The use of Quantitative Structure-Activity Relationships in assessing the potential negative effects of chemicals plays an important role in ecotoxicology. (LC50)(96h) in Pimephales promelas (Duluth database) is widely modeled as an aquatic toxicity end-point. The object of this study was to compare different molecular descriptors in the development of new statistically validated QSAR models to predict the aquatic toxicity of chemicals classified according to their MOA and in a unique general model. The applied multiple linear regression approach (ordinary least squares) is based on theoretical molecular descriptor variety (1D, 2D, and 3D, from DRAGON package, and some calculated logP). The best combination of modeling descriptors was selected by the Genetic Algorithm-Variable Subset Selection procedure. The robustness and the predictive performance of the proposed models was verified using both internal (cross-validation by LOO, bootstrap, Y-scrambling) and external statistical validations (by splitting the original data set into training and validation sets by Kohonen-artificial neural networks (K-ANN)). The model applicability domain (AD) was checked by the leverage approach to verify prediction reliability.