"How Good is our MT? A Glimpse into MT Evaluation Challenges in Commercial Settings"
by Dr Anna Zaretskaya, TransPerfect
Abstract
In this presentation we will talk about how machine translation quality is typically evaluated in commercial settings. While there are a lot of good-quality MT systems available for casual and commercial use, translation industry started to pay a lot more attention to MT evaluation. TransPerfect’s customers, typically large organisations, often struggle to find the best way of choosing between different MT providers. In addition, they also want to be sure that the translation quality of the chosen provider is the best they can get, that it brings them maximum ROI, and that the MT quality is improving over time. These are only some of the typical cases that require well-defined MT evaluation methods. While in research such methods already exist (e.g. WMT shared tasks), they are not always directly applicable in commercial scenarios.
Speaker’s bio
Anna Zaretskaya currently works at TransPerfect as a Senior MT Implementation Manager. In this role she is responsible for designing and implementing MT solutions for TransPerfect clients. She joined the company in 2016 after finishing her PhD in translation technologies and user needs. She has a background in general linguistics (undergraduate studies) and computational linguistics (MS).