Cost-Sensitive Learning to Defer to Multiple Experts

Priberam Labs

Instituto Superior Técnico, Anfiteatro PA2

Abstract:Learning to defer (L2D) aims to improve human-AI collaboration systems by learning how to defer decisions to humans when they are more likely to be correct than an ML classifier. Existing research in L2D overlooks key real-world aspects that impede its practical adoption, namely: i) neglecting cost-sensitive scenarios, where type I and type II errors have different costs; ii) requiring concurrent human predictions for every instance of the training dataset; and iii) not dealing with human work-capacity constraints. To address these issues, we propose the deferral under cost and capacity constraints framework (DeCCaF). DeCCaF is a novel L2D approach, employing supervised learning to model the probability of human error under less restrictive data requirements (only one expert prediction per instance) and using constraint programming to globally minimize the error cost, subject to workload limitations. We test DeCCaF in a series of cost-sensitive fraud detection scenarios with different teams of 9 synthetic fraud analysts, with individual work-capacity constraints. The results demonstrate that our approach performs significantly better than the baselines in a wide array of scenarios, achieving an average 8.4% reduction in the misclassification cost. The code used for the experiments is available at <a href="https://github.com/feedzai/deccaf" title="https://github.com/feedzai/deccaf" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">https://github.com/feedzai/deccaf</a>. Bio:Jean Alves is a Senior Research Data Scientist at Feedzai, where she's been working for 2 years. Jean received a Bachelor's in Physics Engineering and a Master's in Data Science Engineering, both at IST. Jean has a special interest in Human AI collaboration applications, and has recently worked on the application of LLMs to this use case.<a href="https://www.priberam.com/" title="https://www.priberam.com/" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">www.priberam.com</a>

Our {communityGuidelinesLink} describe the sort of content we prohibit on Eventbrite. If you suspect an event may be in violation, you can report it to us so we can investigate.

To report other categories of prohibited or illegal content, submit {link}

Cost-Sensitive Learning to Defer to Multiple Experts

More events from Priberam Labs

Discover more events from Priberam Labs, from Science & Tech to other experiences you might love.

Still looking for the right event?

Explore all events in Lisboa and filter by date, category, and more to find the perfect fit.

Cost-Sensitive Learning to Defer to Multiple Experts

More events from Priberam Labs

Discover more events from Priberam Labs, from Science & Tech to other experiences you might love.

You might also like...

Browse more events with different dates, prices, and formats to find your next great experience.

Still looking for the right event?

Explore all events in Lisboa and filter by date, category, and more to find the perfect fit.