"Combining Static and Contextualised Multilingual Embeddings"

by Katharina Hämmerl, LMU Munich

Update: the event has now finished (Apr 26th 2022).

Abstract

I work on multilingual contextual (sentence) representations. During my Master’s thesis and for the titular paper, we explored how static embeddings might be combined with contextual models for well-aligned representations on both sides. We extracted high quality static embeddings from XLM-R and aligned them further. We also continued training the contextual model to improve its cross-lingual alignment, yielding especially large performance increases for sentence retrieval.

Speaker’s bio

My name is Katharina Hämmerl, I am a PhD student at CIS, LMU, co-supervised by Alex Fraser and Jindřich Libovický. My research focuses on multilingual pre-trained language models and sentence representations. Between autumn 2018 and spring 2021, I obtained my Master’s degree in Computational Linguistics from CIS. Before that, I studied English Philology and Computer Science.

CONTACT DETAILS


RGCL
University of Wolverhampton
Wulfruna Street
Wolverhampton, WV1 1LY
United Kingdom