Hanna Bechara, ESR12

Location: University of Wolverhampton, UK

Project Title: Evaluation for Machine Translation

Project Description: 

I started work on the Expert Project in January 2014 as an early stage researcher. My research focuses on the evaluation of machine translation quality, especially in cases where a reference translation is not available. 

In my first year I conducted an investigation into the relation between verb pattern matches in the Pattern Dictionary of English Verbs (PDEV) and translation quality through a qualitative analysis of human-ranked sentences from 5 different machine translation systems. The purpose of the analysis is to establish links between hypothesis sentences and the verbs in the reference translation. It attempts to answer the question of whether or not the semantic and syntactic information captured by Corpus Pattern Analysis (CPA) can indicate whether a sentence is a “good” translation. 

More recently, I have introduced a new semantic-based approach to quality estimation. This approach exploits state-of-the-art Semantic Textual Similarity methods to semantically compare the MT output to a similar sentence with an independent quality score or reference translation. 

I am currently working on an in-depth study into the impact quality estimation can have on post-editing efficiency in a real world setting, using data provided by Hermes, one of the commercial partners in the EXPERT project.

Research Interests: Machine Learning, SMT, Evaluation, Quality Estimation

Publication list

  1. Hanna Béchara, Sara Moze, Ismail El-Maarouf, Constantin Orasan, Patrick Hanks and Ruslan Mitkov. 2015. The Role of Corpus Pattern Analysis in Machine Translation Evaluation. In Proceedings of the The 7th International Conference of the Iberian Association of Translation and Interpreting Studies (AIETI), Malaga, Spain.

  2. Hanna Bechara, Hernani Costa, Shiva Taslimipoor, Rohit Gupta, Constantin Orasan, Gloria Corpas Pastor and Ruslan Mitkov. (2015). MiniExperts: A SVM approach for Measuring Semantic Textual Similarity. In Proceedings of the 2015 Conference of the North American Chapter of the 34 Association for Computational Linguistics: Human Language Technologies, Denver, Colorado

    http://alt.qcri.org/semeval2015/cdrom/pdf/SemEval017.pdf

  3. Hanna Bechara, Rohit Gupta, Liling Tan, Constantin Orasan, Ruslan Mitkov and Josef van Genabith. 2016. WOLVESAAR: Replicating the Success of Monolingual Word Alignment and Neural Embeddings for Semantic Textual Similarity. In Proceedings of Tenth International Workshop on Semantic Evaluation (SemEval 2016). San Diego, USA.

  4. Bechara, H., Parra Escartin, C. Orasan, C and Specia, L. (2016) Semantic Textual Similarity for Quality Estimation. In Proceedings of 19th Annual Conference of the European Association for Machine Translation, EAMT. Riga, Latvia. May 29-31.

  5. Rohit Gupta, Hanna Bechara, and Constantin Orasan. 2014. Intelligent Translation Memory Matching and Retrieval Metric Exploiting Linguistic Technology. In Proceedings of the thirty sixth Conference on Translating and Computer, London, UK.

    http://www.mt-archive.info/10/Asling-2014-Gupta.pdf

  6. Rohit Gupta, Hanna Bechara, Ismaïl El Maarouf and Constantin Orasan. (2014). UoW: NLP techniques developed at the University of Wolverhampton for Semantic Similarity and Textual Entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland. pp. 785-789

    http://aclweb.org/anthology/S14-2139

  7. Sanja Stajner, Hanna Bechara, Horacio Saggion (2015) A Deeper Exploration of the Standard PB-SMT Approach to Text Simplification and its Evaluation. In the Proceedings of the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Beijing, China

  8. Stajner, S, Popovic, M and Bechara, H. (2016). Quality Estimation for Text Simplification. In Proceedings of the 10th edition of the Language Resources and Evaluation Conference (LREC), Portoroz, Slovenia, 25 28 May