WMT14

Workshop on Machine Translation

---

Location

• Maryland, United States of America

Links

• statmt.org/wmt14/program.html

Shared Tasks

• Translation
• Metrics
• Quality estimation
• Medical translation

Schedule

Day 1

9:00	Opening Remarks
9:10	Session 1: Shared Translation Tasks Efficient Elicitation of Annotations for Human Evaluation of Machine Translation Keisuke Sakaguchi, Matt Post, Benjamin Van Durme
9:30	Findings of the 2014 Workshop on Statistical Machine Translation Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, Aleš Tamchyna
10:00	Panel Discussion (slides)
10:30	☕️
11:00	Session 2: Poster Session Shared Task: Translation
	Parallel FDA5 for Fast Deployment of Accurate Statistical Machine Translation Systems Ergun Bicici, Qun Liu, Andy Way
	Yandex School of Data Analysis Russian-English Machine Translation System for WMT14 Alexey Borisov, Irina Galinskaya
	CimS – The CIS and IMS joint submission to WMT 2014 translating from English into German Fabienne Cap, Marion Weller, Anita Ramm, Alexander Fraser
	English-to-Hindi system description for WMT 2014: Deep Source-Context Features for Moses Marta R. Costa-jussà, Parth Gupta, Paolo Rosso, Rafael E. Banchs
	The KIT-LIMSI Translation System for WMT 2014 Quoc Khanh Do, Teresa Herrmann, Jan Niehues, Alexander Allauzen, François Yvon, Alex Waibel
	The IIT Bombay Hindi-English Translation System at WMT 2014 Piyush Dungarwal, Rajen Chatterjee, Abhijit Mishra, Anoop Kunchukuttan, Ritesh Shah, Pushpak Bhattacharyya
	Edinburgh's Phrase-based Machine Translation Systems for WMT-14 Nadir Durrani, Barry Haddow, Philipp Koehn, Kenneth Heafield
	EU-BRIDGE MT: Combined Machine Translation Markus Freitag, Stephan Peitz, Joern Wuebker, Hermann Ney, Matthias Huck, Rico Sennrich, Nadir Durrani, Maria Nadejde, Philip Williams, Philipp Koehn, Teresa Herrmann, Eunah Cho, Alex Waibel
	Phrasal: A Toolkit for New Directions in Statistical Machine Translation Spence Green, Daniel Cer, Christopher Manning
	Anaphora Models and Reordering for Phrase-Based SMT Christian Hardmeier, Sara Stymne, Jörg Tiedemann, Aaron Smith, Joakim Nivre
	The Karlsruhe Institute of Technology Translation Systems for the WMT 2014 Teresa Herrmann, Mohammed Mediani, Eunah Cho, Thanh-Le Ha, Jan Niehues, Isabel Slawik, Yuqi Zhang, Alex Waibel
	The DCU-ICTCAS MT system at WMT 2014 on German-English Translation Task Liangyou Li, Xiaofeng Wu, Santiago Cortes Vaillo, Jun Xie, Andy Way, Qun Liu
	The CMU Machine Translation Systems at WMT 2014 Austin Matthews, Waleed Ammar, Archna Bhatia, Weston Feely, Greg Hanneman, Eva Schlinger, Swabha Swayamdipta, Yulia Tsvetkov, Alon Lavie, Chris Dyer
	Stanford University's Submissions to the WMT 2014 Translation Task Julia Neidert, Sebastian Schuster, Spence Green, Kenneth Heafield, Christopher Manning
	The RWTH Aachen German-English Machine Translation System for WMT 2014 Stephan Peitz, Joern Wuebker, Markus Freitag, Hermann Ney
	Large-scale Exact Decoding: The IMS-TTT submission to WMT14 Daniel Quernheim, Fabienne Cap
	Abu-MaTran at WMT 2014 Translation Task: Two-step Data Selection and RBMT-Style Synthetic Rules Raphael Rubino, Antonio Toral, Víctor M. Sánchez-Cartagena, Jorge Ferrández-Tordera, Sergio Ortiz Rojas, Gema Ramírez-Sánchez, Felipe Sánchez-Martínez, Andy Way
	The UA-Prompsit hybrid machine translation system for the 2014 Workshop on Statistical Machine Translation Víctor M. Sánchez-Cartagena, Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez
	Machine Translation and Monolingual Postediting: The AFRL WMT-14 System Lane Schwartz, Timothy Anderson, Jeremy Gwinnup, Katherine Young
	CUNI in WMT14: Chimera Still Awaits Bellerophon Aleš Tamchyna, Martin Popel, Rudolf Rosa, Ondrej Bojar
	Manawi: Using Multi-Word Expressions and Named Entities to Improve Machine Translation Liling Tan, Santanu Pal
	Edinburgh's Syntax-Based Systems at WMT 2014 Philip Williams, Rico Sennrich, Maria Nadejde, Matthias Huck, Eva Hasler, Philipp Koehn
	DCU-Lingo24 Participation in WMT 2014 Hindi-English Translation task Xiaofeng Wu, Rejwanul Haque, Tsuyoshi Okita, Piyush Arora, Andy Way, Qun Liu
11:00	Shared Task: Medical Translation
	Machine Translation of Medical Texts in the Khresmoi Project Ondřej Dušek, Jan Hajič, Jaroslava Hlaváčová, Michal Novák, Pavel Pecina, Rudolf Rosa, Aleš Tamchyna, Zdeňka Urešová, Daniel Zeman
	Postech’s System Description for Medical Text Translation Task Jianri Li, Se-Jong Kim, Hwidong Na, Jong-Hyeok Lee
	Domain Adaptation for Medical Text Translation using Web Resources Yi Lu, Longyue Wang, Derek F. Wong, Lidia S. Chao, Yiming Wang
	DCU Terminology Translation System for Medical Query Subtask at WMT14 Tsuyoshi Okita, Ali Vahid, Andy Way, Qun Liu
	LIMSI @ WMT'14 Medical Translation Task Nicolas Pécheux, Li Gong, Quoc Khanh Do, Benjamin Marie, Yulia Ivanishcheva, Alexander Allauzen, Thomas Lavergne, Jan Niehues, Aurélien Max, François Yvon
	Combining Domain Adaptation Approaches for Medical Text Translation Longyue Wang, Yi Lu, Derek F. Wong, Lidia S. Chao, Yiming Wang, Francisco Oliveira
	Experiments in Medical Translation Shared Task at WMT 2014 Jian Zhang
12:30	🍴
14:00	Session 3: Invited Talk Machine Translation in Academia and in the Commercial World – a Contrastive Perspective Alon Lavie
15:30	☕️
16:00	Session 4: Evaluation Chair: Christian Federmann Randomized Significance Tests in Machine Translation Yvette Graham, Nitika Mathur, Timothy Baldwin
16:20	Estimating Word Alignment Quality for SMT Reordering Tasks Sara Stymne, Jörg Tiedemann, Joakim Nivre
16:40	Dependency-based Automatic Enumeration of Semantically Equivalent Word Orders for Evaluating Japanese Translations Hideki Isozaki, Natsume Kouchi, Tsutomu Hirao

Day 2

9:00	Session 5: Shared Evaluation Metrics and Quality Estimation Tasks Quality Estimation Shared Task
9:30	Results of the WMT14 Metrics Shared Task Matous Machacek, Ondrej Bojar
9:50	Panel Discussion
10:30	☕️
11:00	Session 6: Poster Session Shared Task: Quality Estimation
	Efforts on Machine Learning over Human-mediated Translation Edit Rate Eleftherios Avramidis
	SHEF-Lite 2.0: Sparse Multi-task Gaussian Processes for Translation Quality Estimation Daniel Beck, Kashif Shah, Lucia Specia
	Referential Translation Machines for Predicting Translation Quality Ergun Bicici, Andy Way
	FBK-UPV-UEdin participation in the WMT14 Quality Estimation shared-task José Guilherme Camargo de Souza, Jesús González-Rubio, Christian Buck, Marco Turchi, Matteo Negri
	Target-Centric Features for Translation Quality Estimation Chris Hokamp, Iacer Calixto, Joachim Wagner, Jian Zhang
	LIG System for Word Level QE task at WMT14 Ngoc Quang Luong, Laurent Besacier, Benjamin Lecouteux
	Exploring Consensus in Machine Translation for Quality Estimation Carolina Scarton, Lucia Specia
	LIMSI Submission for WMT’14 QE Task Guillaume Wisniewski, Nicolas Pécheux, Alexander Allauzen, François Yvon
11:00	Shared Task: Evaluation Metrics
	Parmesan: Meteor without Paraphrases with Paraphrased References Petra Barancikova
	A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU Boxing Chen, Colin Cherry
	VERTa participation in the WMT14 Metrics Task Elisabet Comelles, Jordi Atserias
	Meteor Universal: Language Specific Translation Evaluation for Any Target Language Michael Denkowski, Alon Lavie
	Application of Prize based on Sentence Length in Chunk-based Automatic Evaluation of Machine Translation Hiroshi Echizen’ya, Kenji Araki, Eduard Hovy
	LAYERED: Metric for Machine Translation Evaluation Shubham Gautam, Pushpak Bhattacharyya
	IPA and STOUT: Leveraging Linguistic and Source-based Features for Machine Translation Evaluation Meritxell Gonzàlez, Alberto Barrón-Cedeño, Lluís Màrquez
	DiscoTK: Using Discourse Structure for Machine Translation Evaluation Shafiq Joty, Francisco Guzmán, Lluís Màrquez, Preslav Nakov
	Tolerant BLEU: a Submission to the WMT14 Metrics Task Jindřich Libovický, Pavel Pecina
	BEER: BEtter Evaluation as Ranking Miloš Stanojević, Khalil Sima’an
	RED, The DCU-CASICT Submission of Metrics Tasks Xiaofeng Wu, Hui Yu, Qun Liu
12:30	🍴
14:00	Session 7: Data and Adaptation Chair: Alexander Fraser Crowdsourcing High-Quality Parallel Data Extraction from Twitter Wang Ling, Luis Marujo, Chris Dyer, Alan W Black, Isabel Trancoso
14:20	Using Comparable Corpora to Adapt MT Models to New Domains Ann Irvine, Chris Callison-Burch
14:40	Dynamic Topic Adaptation for SMT using Distributional Profiles Eva Hasler, Barry Haddow, Philipp Koehn
15:00	Unsupervised Adaptation for Statistical Machine Translation Saab Mansour, Hermann Ney
15:20	☕️
16:00	Session 8: Translation Models Chair: Jan Niehus An Empirical Comparison of Features and Tuning for Phrase-based Machine Translation Spence Green, Daniel Cer, Christopher Manning
16:20	Bayesian Reordering Model with Feature Selection Abdullah Alrajeh, Mahesan Niranjan
16:40	Augmenting String-to-Tree and Tree-to-String Translation with Non-Syntactic Phrases Matthias Huck, Hieu Hoang, Philipp Koehn
17:00	Linear Mixture Models for Robust Machine Translation Marine Carpuat, Cyril Goutte, George Foster

Results

General Task

Full results of the shared task: Findings of the 2014 Workshop on Statistical Machine Translation

News translation

The results were determined with a relative ranking through the TrueSkill model.

→ English

Language pair	System	Score
Czech →	ONLINE-B	0.591
Russian →	AFRL-PE	0.583
German →	ONLINE-B	0.451
French →	UEDIN-PHRASE	0.608
Hindi →	ONLINE-B	1.326

English →

Language pair	System	Score
→ Czech	CU-DEPFIX	0.371
→ Russian	PROMT-RULE	0.575
→ German	UEDIN-SYNTAX	0.264
→ French	ONLINE-B	0.327
→ Hindi	ONLINE-B	1.008