BEGIN:VCALENDAR VERSION:2.0 PRODID:-//Talks.cam//talks.cam.ac.uk// X-WR-CALNAME:Talks.cam BEGIN:VEVENT SUMMARY:Beyond Calibration of Probabilistic Classifier Outputs - Agathe Fe rnandes Machado (Université du Québec à Montréal) DTSTART:20250603T110500Z DTEND:20250603T112500Z UID:TALK230800@talks.cam.ac.uk DESCRIPTION:The assessment of binary classifier performance traditionally centers on discriminative ability\, using metrics such as accuracy. Howeve r\, these metrics often disregard the model&rsquo\;s inherent uncertainty\ , an aspect that becomes particularly important in sensitive decision-maki ng domains like healthcare or insurance. For instance\, when estimating mo rtality rates or predicting accident occurrence in an insurance context\, since model-predicted scores are often interpreted as event probabilities\ , having a calibrated model allows for enhancing the reliability of these predictions. While calibration enhances the alignment between predicted sc ores and actual outcomes\, a common misconception is to interpret the resu lting scores as true posterior probabilities. Specifically\, tree-based cl assifiers may seem calibrated based on standard calibration metrics and ex hibit high predictive performance even if their score distributions do not align with true event distributions. In this study\, we analyze how the l ack of score heterogeneity in tree-based classifiers can lead to misinterp retation of low calibration errors. \;Furthermore\, since the notion o f calibration is often used to assess fairness by comparing errors across groups defined by sensitive attributes such as gender or race\, a lack of score heterogeneity within minority groups can conceal underlying disparit ies. Addressing this issue allows for a more accurate evaluation of fairne ss based on calibration-related metrics\, which is especially relevant in the insurance context\, as emphasized by Baumann et al. (2023).\nCo-Author s: Arthur Charpentier (Université\; du Qué\;bec à\; Mont ré\;al)\, Ewen Gallic (Aix Marseille Université\;)\, Emmanuel Flachaire (Aix Marseille Université\;)\, Franç\;ois Hu (Millim an France) LOCATION:Seminar Room 1\, Newton Institute END:VEVENT END:VCALENDAR