For our latest Research Seminar on the 7th March we welcomed Hanna Bechara, PhD student, over from Dublin. Her talk on her thesis was well attended and prompted interesting questions from the group.
TITLE: Semantic Textual Similarity and its Application in Evaluation
Semantic Textual Similarity measures the degree of semantic equivalence between two sentences or phrases. Similarity measures between sentences are required in a wide variety of NLP applications, such as information retrieval, automatic text summarisation. Our work investigates the applications of semantic textual similarity in evaluation. We focus specifically on the evaluation of machine translation and automatic text simplification. By using methods previously employed in Semantic Textual Similarity (STS) tasks, we use semantically similar sentences and their quality scores as features to estimate the quality of machine translated sentences. Our results show that this method can improve the prediction of machine translation quality for semantically similar sentences. We apply the semantic similarity methods to evaluate the output of automatically simplified text. We find that our features are strong indicators for quality. On the Shared Task on Quality Assessment for Text Simplification (QATS), our classification systems ranked second overall among all participating systems and consistently outperformed the baseline for all types of quality measures.
Our next seminar will be given by Reshmi Gopalakrishna Pillai on Detection of strength and casual agents of stress and relaxation in social media content. This will be from13:00 on the 19th March in MU402, all are welcome and we look forward to seeing you there.