Varvara Logacheva, ESR6

Location: Sheffield University, UK

Project Title: Learning from human feedback on the quality of the translations

Project Description: 

Machine translation (MT) is now widely used by professional translators as a draft for further post-editing. In this case post-editor produces high-quality edits of the automatically translated sentences. The analysis of such edits can give the information on the common pitfalls of machine translation, and their incorporation into an MT system can improve its quality and tune it to a specific domain or user. The aim of my research is to find ways of using this feedback in machine translation.
The available post-editions are too scarce to be used directly in an MT system. Therefore, I generalise the user feedback by using it as training data for a quality estimation (QE) system. QE system aims at defining the quality of an automatic translation without comparing it to a reference. It is trained on automatically translated sentences manually labelled with errors, and then used for estimating the quality of unseen data. By estimating the quality of new automatic translations I propagate the information on user preferences onto unseen texts.

I estimate the quality of automatic translations at different levels of granularity (word, phrase, and sentence) and use this information to select the data for an MT system training and to improve an MT system by incorporating quality scores as new features.

Research Interests: machine translation, quality estimation

 

Publication list

  1. F. Blain, V. Logacheva, and L. Specia. (2016) ``Phrase Level Segmentation and Labelling of Machine Translation Errors''. In Proceedings of LREC-2016

  2. O. Bojar, R. Chatterjee, C. Federmann, B. Haddow, C. Hokamp, M. Huck, V. Logacheva, P. Koehn, C. Monz, M. Negri, P. Pecina, M. Post, C. Scarton, L. Specia, and M. Turchi (2015) “Findings of the 2015 Workshop on Statistical Machine Translation,” In Proceedings of the 10th Workshop on Machine Translation (WMT-2015), Lisbon, Portugal, pp. 1–46

  3. O. Bojar, R. Chatterjee, C. Federmann, Y. Graham, B. Haddow, M. Huck, A.J. Yepes, P. Koehn, V. Logacheva, C. Monz, M. Negri, A. Neveol, M. Neves, M. Popel, M. Post, R. Rubino, C. Scarton, L. Specia, M. Turchi, K. Verspoor and M. Zampieri. (2016) ``Findings of the 2016 Conference on Machine Translation''. In Proceedings of the First Conference on Machine Translation (WMT-2016), Berlin, Germany, August 2016, pp. 131--198.

  4. Logacheva, V. and Specia, L. (2014). Confidence-based Active Learning Methods for Machine Translation. In EACL-2014: the 14th Conference of the European Chapter of the Association for Computational Linguistics, Workshop on Humans and Computer-assisted Translation, Gothenburg, Sweden, 26 April, 2014.

  5. Logacheva, V. and Specia, L. (2014). Quality-based active sample selection strategy for statistical machine translation. In Proceedings of the 9th edition of the Language Resources and Evaluation Conference (LREC2014), Reykjavik, Iceland, 26-31 May 2014

  6. V. Logacheva and L. Specia. (2015) “The role of artificially generated negative data for quality estimation of machine translation”. In Proceedings of the 18th annual conference of the European Association for Machine Translation (EAMT-2015), Antalya, Turkey, May 2015, pp. 51–58.

  7. V. Logacheva, C. Hokamp, and L. Specia, “Data enhancement and selection strategies for the word-level quality estimation”. In Proceedings of the 10th Workshop on Machine Translation (WMT-2015), Lisbon, Portugal, September 2015, pp. 311–316

  8. V. Logacheva and L. Specia. (2015) “Phrase-level Quality Estimation for Machine Translation”. In Proceedings of the 12th International Workshop on Spoken Language Translation (IWSLT 2015), Da Nang, Vietnam, December 2015.

  9. V. Logacheva, M. Lukasik and L. Specia. (2016) ``Metrics for Evaluation of Word-level Machine Translation Quality Estimation''. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL-2016), Berlin, Germany, August 2016.

  10. V. Logacheva, F. Blain, and L. Specia. (2016) ``USFD's Phrase-level Quality Estimation Systems''. In Proceedings of the First Conference on Machine Translation (WMT-2016), Berlin, Germany, August 2016, pp. 800 - 805.

  11. V. Logacheva, C. Hokamp, and L. Specia. (2016) ``MARMOT: A Toolkit for Translation Quality Estimation at the Word Level''. In Proceedings of LREC-2016.

  12. K. Shah, V. Logacheva, G. Paetzold, F. Blain, D. Beck, F. Bougares, and L. Specia (2015) “Shef-nn: Translation quality estimation with neural networks”. In Proceedings of the 10th Workshop on Machine Translation (WMT-2015), Lisbon, Portugal, September, pp. 342–347

Resources