12.12.2021

Speaker: Lyke Esselink, University of Amsterdam

Title: Text-to-sign translation: making information accessible

Abstract:

Communication between healthcare professionals and deaf patients is challenging, and the current COVID-19 pandemic makes this issue even more acute. Sign language interpreters can often not enter hospitals and face masks make lipreading impossible. To address this urgent problem, SignLab Amsterdam developed a system which allows healthcare professionals to translate sentences that are frequently used in the diagnosis and treatment of COVID-19 into Sign Language of the Netherlands (NGT). Translations are displayed by means of videos and avatar animations. The architecture of the system is such that it could be extended to other applications and other sign languages in a relatively straightforward way.

In the first part of this talk, I will present an overview of the system created by SignLab Amsterdam. I will provide a background on the problem at hand, explain the basics of sign languages and sign synthesis, and outline our system and the process behind its implementation. The second part of the talk will focus on an extensive evaluation study that we did, of which the results are not yet published. I will cover the methodology of this study, some important lessons that we learned from the process, and unveil some of the results.

Bio:

Lyke Esselink is a Master’s student in Artificial Intelligence at the Radboud University in Nijmegen, and completed her bachelor’s degree in AI at the University of Amsterdam. Since the start of 2020, she combined her education with her interest in sign language through research at SignLab Amsterdam, where she has investigated the translation of text to Sign Language of the Netherlands. Research interest areas include Machine Translation, Natural Language Processing and accessibility technologies.