Abstract

Multilingual representation spaces, spanned by multilingual word embeddings or massively multilingual transformers, conceptually enable modeling of meaning across a wide range of languages and language transfer of task-specific NLP models from resource-rich to resource-lean languages. It is not yet clear, however, to which extent this conceptual promise holds in practice. Recent models, both cross-lingual word embedding models and multilingual transformers, have been praised for being able to induce multilingual representation spaces without any explicit supervision (i.e., without any word-level alignments or parallel corpora). In this talk, I will point to some prominent shortcomings and pitfalls of existing evaluations of multilingual representation spaces, which mask important limitations of state-of-the-art multilingual representation models. Remedying for some of these evaluation shortcomings, portrays meaning representation and language transfer capabilities of current state-of-the-art multilingual representation spaces in a less favorable light.