We present a framework for machine translation evaluation using neural networks in a pairwise setting, where the goal is to select the better translation from a pair of hypotheses, given the reference translation.
This paper presents the results of the WMT11 shared tasks, which included a translation task, a system combination task, and a task for machine translation evaluation metrics.
The transcultural adaptation process followed these steps: translation, back translation, evaluation of translations by a panel of judges, and testing the final version.
Edinburgh System Description for the 2005 IWSLT Speech Translation Evaluation, Philipp Koehn, Amittai Axelrod, Alexandra Birch Mayne, Chris Callison-Burch, Miles Osborne and David Talbot, International Workshop on Spoken Language Translation 2005, pdf.
A 1-gram, 2-gram, 3-gram, and 4-gram translation evaluation using BLEU [9] was performed on 100 sentences created from both the SBS and the WBW machine translators, using two human references.
Ten Years of WMT Evaluation Campaigns: Lessons Learnt, Ondrej Bojar, Christian Federmann, Barry Haddow, Philipp Koehn, Matt Post, and Lucia Specia, Proceedings of LREC 2016 Workshop Translation Evaluation: From Fragmented Tools and Data Sets to an Integrated Ecosystem, Portoroz, 2016, pdf, bib.
Reconciliation of the two adaptations as in Step 3, back-translation of adapted items and evaluation of issues, and revision as needed based on back-translation evaluation.
Bibliographic databases were searched for articles concerning the translation or evaluation of the measurement properties of a translated version of a neck-specific questionnaire.
A study was included if it was a full text original article (e.g. not an abstract, review or editorial), published in English, concerning the translation or evaluation of the measurement properties of a translated version of a neck-specific questionnaire.
Boolean queries, query translation, cost evaluation, information retrieval, heterogeneity, digital libraries, query subsumption, filtering.
We tested the system on lecture speeches with topics related to Technology, Entertainment and Design (TED) that were used in the international workshop on spoken language translation (IWSLT) evaluation campaign, and obtained 6.5%, 7.0%, 10.6%, and 8.4% word error rates for test sets in 2011, 2012, 2013, and 2014, which to our knowledge are the best results for these evaluation sets.
