Revisiting Cross-Lingual Transfer Learning

Add to your list(s) Download to your calendar using vCal

• Mikel Artetxe, Meta
• Thursday 01 December 2022, 11:00-12:00
• GR04, English Faculty Building, 9 West Road, Sidgwick Site.

If you have a question about this talk, please contact Panagiotis Fytas.

Given downstream training data in one language (typically English), the goal of cross-lingual transfer learning is to perform the task in another language. Existing approaches have been broadly classified into 3 categories: zero-shot (fine-tune a multilingual language model in English and zero-shot transfer into the target language), translate-train (translate the training data into the target language through MT and fine-tune a multilingual language model), and translate-test (translate the evaluation data into English through MT and use an English model). Prior work mostly finds that translate-train performs best followed by zero-shot and translate-test, and focuses on improving multilingual models. In this 3-part talk, we will revisit some of the fundamentals of this problem, challenging the conventional wisdom in the area. First, we will see that a large part of the improvements from using parallel data can be attributed to explicitly modeling parallel interactions, and similar improvements can be obtained using synthetic data. Second, we will revisit the integration of MT into the pipeline, showing that the potential of translate-test has been largely underestimated. Finally, we will see how creating multilingual benchmarks through translation, as it is commonly done, can result in evaluation artifacts, which calls to reconsider some prior findings.

This talk is part of the Language Technology Lab Seminars series.

This talk is included in these lists:

• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• GR04, English Faculty Building, 9 West Road, Sidgwick Site
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• Language Technology Lab Seminars
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.