How to Critically Evaluate AI Models and Outputs
Introduction
AI predictions, no matter how sophisticated, remain approximations: they are shaped by their training data, their architectures, and their assumptions. Please find a rigorous, step-by-step framework for scrutinizing AI models and their predictions.
We explore relevant metrics, common sources of error, practical tests for plausibility, and examples of both robust and misleading outcomes. Such evaluation is crucial to avoid overconfidence, misinterpretation, and unproductive reliance on flawed results.
Why Critical Evaluation is Necessary
AI systems, especially deep learning models, often produce predictions with high apparent confidence even when extrapolating outside of their valid domain. Their inner workings, particularly in the case of black-box neural networks, can obscure whether predictions are physically plausible or chemically meaningful.
A failure to evaluate critically can result in wasted resources, incorrect hypotheses, and flawed designs. Over-reliance on generative models has led to proposed molecules that violated fundamental rules of valency or stability, while overconfident ML scoring functions have misranked ligands, leading to failed experimental validation.
Understanding Key Outputs of AI Models
AI predictions in computational chemistry take diverse forms, such as energies, structures, binding scores, reaction pathways, each accompanied by specific metrics that quantify confidence and performance.
Performance Metrics
Regression tasks (e.g., property prediction, energy prediction)
Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) measure deviation from experimental or reference QM values. Lower is better, but always benchmarked against the expected noise in the reference data.
Coefficient of determination (R²) assesses correlation quality.
Classification tasks (e.g., virtual screening)
Area under the Receiver Operating Characteristic curve (ROC-AUC) measures the ability to discriminate actives from inactives.
Enrichment factor (EF) reflects how well top-ranked candidates are enriched in true positives compared to random selection.
Generative models (e.g., molecule design)
Evaluate not only the predicted property but also synthetic accessibility, chemical validity, and novelty.
Uncertainty Measures
Some models output uncertainty estimates, such as Bayesian neural networks, but many do not. Where absent, one should assume that predictions beyond the training domain carry significant risk.
Checklist: Evaluating AI Predictions
To make the evaluation actionable, we offer a checklist that computational chemists can apply systematically to AI predictions:
Domain Relevance
Is the prediction within the chemical space represented in the training data?
Does the model extrapolate beyond its validated regime?
Data Quality
Was the model trained on clean, representative, and appropriately partitioned data?
Were training and test sets truly independent (no data leakage)?
Plausibility of Output
Does the molecule respect fundamental chemical rules (valency, charge balance, stereochemistry)?
Are proposed reactions consistent with known mechanistic pathways?
Benchmark Against Known Results
Compare predictions to experimental data or trusted QM calculations.
Check whether error margins are within acceptable limits.
Uncertainty & Confidence
Is uncertainty quantified, and is it appropriately low for the intended use?
Are confidence intervals reported and meaningful?
Interpretability: Can you rationalize the prediction in terms of known structure–activity relationships or physical principles?
Common Errors and Pitfalls
Even well-trained models can fail in predictable ways. Being aware of these pitfalls helps avoid costly mistakes.
Overfitting and Data Leakage: A common error is inadvertently training and testing on overlapping data, which inflates reported accuracy but fails in real applications.
Misinterpreting Confidence: High-confidence predictions outside the training domain are often incorrect. AI does not “know what it does not know,” unless explicitly designed to estimate uncertainty.
Ignoring Physical Constraints: Generative models have produced molecules with impossible ring strains or unstable intermediates because they optimize a statistical objective without chemical reasoning (Polishchuk et al., 2013).
Assuming Static Structures Reflect Biological Reality: AI-predicted structures or docking poses are often treated as rigid and accurate, neglecting flexibility, solvation, and dynamics.
Examples of Rigorous and Flawed Use
Smith et al. (2017) introduced ANI-1x potentials and extensively benchmarked them against QM reference calculations, reporting MAE within ~1 kcal/mol on held-out test sets — suitable for many tasks but transparently reporting limitations for charged or exotic species.
Wallach et al. (2015) noted that early deep learning models for virtual screening produced inflated ROC-AUCs due to bias in test sets, which collapsed when confronted with truly independent targets.
Recommendations for Practice
Benchmark early and often: Always compare AI predictions to trusted reference methods or experimental data before deploying them.
Consult domain experts: Collaborate with synthetic chemists, structural biologists, or spectroscopists to validate predictions from different angles.
Test in diverse scenarios: Evaluate performance not only on average but also on edge cases and challenging examples.
Document and report: Clearly record the model, data, metrics, and known limitations when communicating results.
Conclusion
Critical evaluation is not merely a safeguard; it is a fundamental scientific responsibility when using AI in computational chemistry. While these models can augment human expertise and accelerate discovery, they are susceptible to biases, blind spots, and overfitting. Computational chemists who approach AI predictions with skepticism, rigor, and an insistence on validation will be best positioned to leverage these tools effectively.
We will now shift from evaluating predictions to exploring the tools and frameworks available for computational chemists looking to adopt AI methods responsibly.
References
Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. Proceedings of ICML, 1050–1059.
Polishchuk, P. G., Madzhidov, T. I., & Varnek, A. (2013). Estimation of the size of drug-like chemical space based on GDB-17 data. Journal of Computer-Aided Molecular Design, 27(8), 675–679. https://doi.org/10.1007/s10822-013-9672-4
Sieg, J., Flachsenberg, F., & Rarey, M. (2019). In need of bias control: Evaluating chemical data for machine learning in structure-based virtual screening. Journal of Chemical Information and Modeling, 59(3), 947–961. https://doi.org/10.1021/acs.jcim.8b00832
Sliwoski, G., et al. (2014). Computational methods in drug discovery. Pharmacological Reviews, 66(1), 334–395. https://doi.org/10.1124/pr.112.007336
Smith, J. S., et al. (2017). ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost. Chemical Science, 8(4), 3192–3203. https://doi.org/10.1039/C6SC05720A
von Lilienfeld, O. A., et al. (2020). Exploiting machine learning for chemical discovery and design. Nature Reviews Chemistry, 4(7), 347–358. https://doi.org/10.1038/s41570-020-0186-2
Wallach, I., Dzamba, M., & Heifets, A. (2015). AtomNet: A deep convolutional neural network for bioactivity prediction in structure-based drug discovery. arXiv preprint, arXiv:1510.02855.
Wallach, I., Dzamba, M., & Heifets, A. (2018). The limitations of deep learning in drug discovery. Nature Reviews Drug Discovery, 17(3), 155–156. https://doi.org/10.1038/nrd.2018.3
Zhavoronkov, A., et al. (2019). Deep learning enables rapid identification of potent DDR1 kinase inhibitors. Nature Biotechnology, 37(9), 1038–1040. https://doi.org/10.1038/s41587-019-0224-x
