Reproducibility, Transparency and Evaluation of Machine Learning in Health Applications

Abstract

This paper argues for the importance of detailed reporting of results of machine learning modeling applied in medical, healthcare and health applications. It describes ten criteria under which results of modeling should be reported. The ten proposed criteria are experimental design, statistical model evaluation, model calibration, top predictors, global sensitivity analysis, decision curve analysis, global model explanation, local prediction explanation, programming interface and source code. The criteria are discussed and illustrated in the context of existing models. The goal of the reporting is to ensure that results are reproducible, and models gain trust of end users. A brief checklist is provided to help facilitate model evaluation.