Scaling Multilingual Generation for Low-Resource Languages

Add to your list(s) Download to your calendar using vCal

• Priyanka Agrawal, Google Deepmind
• Friday 16 February 2024, 12:00-13:00
• Computer Lab, SS03.

If you have a question about this talk, please contact Eric Chamoun.

Abstract:

The availability of large, high-quality datasets has been one of the main drivers of recent progress in generation tasks like summarization, QA. Such annotated datasets however are difficult and costly to collect, and rarely exist in languages other than English, rendering the technology inaccessible to underrepresented languages. An alternative to building large monolingual training datasets is to leverage pre-trained language models (PLMs). The talk will first discuss an approach, QAmeleon, that tunes a PLM using parameter-efficient fine-tuning methods (PEFT) to synthesize QA data with only five examples per language. Using this data during training delivers accuracy superior to translation-based baselines, and bridges nearly 60% of the gap between an English-only baseline and a fully supervised upper bound trained on almost 50,000 hand-labeled examples. Next, the talk will discuss the cross-lingual transfer approach for a much stricter zero-shot setting to enable generation in unseen languages. Our method composes language and task PEFT modules via element-wise arithmetic operations to leverage unlabeled data and labeled data in other languages. The talk further studied the consistency for cross-lingual generation tasks i.e. the output is in a language different from the source. Here we propose MuPlan which uses intermediate plans resulting in more faithful generation in both fine-tuning and zero-shot setups.

Bio:

Priyanka Agrawal is a Senior Research Scientist at the Google Deepmind in London, formally part of Google Brain, and is focused on building responsible Generative AI models and scaling them to underrepresented languages. Prior to that she was a Senior Researcher and Lead at http://Booking.com and IBM Research Labs, where she was driving work in cross-domain transfer and representation learning. She is an alumni from the Computer Science Department at the Indian Institute of Science. Her work is published at top-tier ML and NLP conferences like NeurIPS, ACL and she holds 25+ US Patents. Priyanka also serves as Area Chair and PC member at these conferences and has been an invited panelist and speaker at various ML/NLP and diversity forums.

This talk is part of the NLIP Seminar Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Computer Lab, SS03
• Department of Computer Science and Technology talks and seminars
• Graduate-Seminars
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• NLIP Seminar Series
• PMRFPS's
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.