New method makes AI predictions more trustworthy by adding honest confidence scores
Researchers have developed a technique that assigns reliable probability estimates to AI predictions, addressing a critical gap in machine learning reliability. For companies deploying AI in high-stakes decisions—from autonomous vehicles to medical diagnostics—knowing when a model is actually confident versus guessing could significantly reduce costly errors and liability risks.
Originaltitel: Assigning Prediction Conditioned Well-Calibrated Probabilities to Set Predictions
This is the data and class-objects that are needed to reproduce results from the paper "Assigning Prediction Conditioned Well-Calibrated Probabilities to Set Predictions" The data consists of Python objects designed for thepaper which contain all models and data splits and cna be used to derive metrics with the code from github Training history for the models in those objects Simulated data whose distribution is described in the paper The MNIST dataset (LeCun, Y., Cortes, C., & Burges, C.J.C. (1998)), which can also be found at http://yann.lecun.com/exdb/mnist/ The CIFAR-10 dataset (Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.(2009)), which can also be found at https://www.cs.toronto.edu/~kriz/cifar.html Labels, hierarchies and prediction probabilties on the clean fitzpatrick dataset originally generated by Cortes‑Gomez, S., Patiño, C. M., Byun, Y., Wu, Z. S., Horvitz, E., & Wilder, B. (2025) for the paper Utility‑Driven Conformal Prediction: A Decision‑Aware Framework for Actionable Uncertainty Quantification.