Navigating Privacy Risks in Language Models

Add to your list(s) Download to your calendar using vCal

• Peter Kairouz -- Google
• Tuesday 16 July 2024, 15:00-16:00
• West 2, West Hub (https://www.westcambridgehub.uk/visit).

If you have a question about this talk, please contact Nic Lane.

The emergence of large language models (LLMs) presents significant opportunities in content generation, question answering, and information retrieval. Nonetheless, training, fine-tuning, and deploying these models entails privacy risks. This talk will address these risks, outlining privacy principles inspired by known LLM vulnerabilities when handling user data. We demonstrate how techniques like federated learning and user-level differential privacy (DP) can systematically mitigate many of these risks at the cost of increased computation. In scenarios where only moderate-to-weak user-level DP is achievable, we propose a strong (task-and-model-agnostic) membership inference attack that allows us to quantify risk by estimating the actual privacy leakage (empirical epsilon) accurately in a single training run. The talk will conclude with a few projections and compelling research directions.

This talk is part of the Cambridge ML Systems Seminar Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Artificial Intelligence Research Group Talks (Computer Laboratory)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• Guy Emerson's list
• Hanchen DaDaDash
• Interested Talks
• School of Technology
• Security-related talks
• Speech Seminars
• Trust & Technology Initiative - interesting events
• West 2, West Hub (https://www.westcambridgehub.uk/visit)
• bld31
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list
• yk449

Note that ex-directory lists are not shown.