WMT18

Conference on Machine Translation

---

Location

• Brussels, Belgium

Links

• www.statmt.org/wmt18/program.html

Shared Tasks

• News
• Biomedical
• Automatic post-editing
• Metrics
• Quality estimation
• Multimodal
• Parallel corpus filtering

Schedule

Day 1

8:45	Opening Remarks
9:00	Session 1: Shared Tasks Overview Presentations I
9:00	Findings of the 2018 Conference on Machine Translation (WMT18) Ondřej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Christof Monz
9:30	Findings of the Third Shared Task on Multimodal Machine Translation Loïc Barrault, Fethi Bougares, Lucia Specia, Chiraag Lala, Desmond Elliott, Stella Frank
9:50	Findings of the WMT 2018 Biomedical Translation Shared Task: Evaluation on Medline test sets Mariana Neves, Antonio Jimeno Yepes, Aurelie Neveol, Cristian Grozea, Amy Siu, Madeleine Kittner, Karin Verspoor
	Boaster Session for Research Papers Presented as Posters (Session I)
10:16	Scaling Neural Machine Translation Myle Ott, Sergey Edunov, David Grangier, Michael Auli
10:18	Character-level Chinese-English Translation through ASCII Encoding Nikola Nikolov, Yuhuang Hu, Mi Xue Tan, Richard H.R. Hahnloser
10:20	Neural Machine Translation of Logographic Language Using Sub-character Level Information Longtu Zhang, Mamoru Komachi
10:22	An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation Gongbo Tang, Rico Sennrich, Joakim Nivre
10:24	Discourse-Related Language Contrasts in English-Croatian Human and Machine Translation Margita Šoštarić, Christian Hardmeier, Sara Stymne
10:26	Coreference and Coherence in Neural Machine Translation: A Study Using Oracle Experiments Dario Stojanovski, Alexander Fraser
10:28	A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation Mathias Müller, Annette Rios, Elena Voita, Rico Sennrich
10:30	☕️
11:00	Session 2: Poster Session I
11:00	Shared Task: News Translation
	An Empirical Study of Machine Translation for the Shared Task of WMT18 Chao Bei, Hao Zong, Yiming Wang, Baoyong Fan, Shiqi Li, Conghu Yuan
	Robust parfda Statistical Machine Translation Results Ergun Bicici
	The TALP-UPC Machine Translation Systems for WMT18 News Shared Translation Task Noe Casas, Carlos Escolano, Marta R. Costa-jussà, Jose A. R. Fonollosa
	Phrase-based Unsupervised Machine Translation with Compositional Phrase Embeddings Maksym Del, Andre Tättar, Mark Fishel
	Alibaba’s Neural Machine Translation Systems for WMT18 Yongchao Deng, Shanbo Cheng, Jun Lu, Kai Song, Jingang Wang, Shenglan Wu, Liang Yao, Guchun Zhang, Haibo Zhang, Pei Zhang, Changfeng Zhu, Boxing Chen
	The RWTH Aachen University English-German and German-English Unsupervised Neural Machine Translation Systems for WMT 2018 Miguel Graca, Yunsu Kim, Julian Schamper, Jiahui Geng, Hermann Ney
	Cognate-aware morphological segmentation for multilingual neural translation Stig-Arne Grönroos, Sami Virpioja, Mikko Kurimo
	The AFRL WMT18 Systems: Ensembling, Continuation and Combination Jeremy Gwinnup, Tim Anderson, Grant Erdmann, Katherine Young
	The University of Edinburgh's Submissions to the WMT18 News Translation Task Barry Haddow, Nikolay Bogoychev, Denis Emelin, Ulrich Germann, Roman Grundkiewicz, Kenneth Heafield, Antonio Valerio Miceli Barone, Rico Sennrich
	TencentFmRD Neural Machine Translation for WMT18 Bojie Hu, Ambyer Han, Shen Huang
	The MLLP-UPV German-English Machine Translation System for WMT18 Javier Iranzo-Sánchez, Pau Baquero-Arnal, Goncal V. Garces Díaz-Munío, Adrià Martínez-Villaronga, Jorge Civera, Alfons Juan
	Microsoft’s Submission to the WMT2018 News Translation Task: How I Learned to Stop Worrying and Love the Data Marcin Junczys-Dowmunt
	CUNI Submissions in WMT18 Tom Kocmi, Roman Sudarikov, Ondřej Bojar
	The JHU Machine Translation Systems for WMT 2018 Philipp Koehn, Kevin Duh, Brian Thompson
	JUCBNMT at WMT2018 News Translation Task: Character Based Neural Machine Translation of Finnish to English Sainik Kumar Mahata, Dipankar Das, Sivaji Bandyopadhyay
	NICT’s Neural and Statistical Machine Translation Systems for the WMT18 News Translation Task Benjamin Marie, Rui Wang, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita
	PROMT Systems for WMT 2018 Shared Translation Task Alexander Molchanov
	NTT’s Neural Machine Translation Systems for WMT 2018 Makoto Morishita, Jun Suzuki, Masaaki Nagata
	The Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2018 Ngoc-Quan Pham, Jan Niehues, Alexander Waibel
	Tilde’s Machine Translation Systems for WMT 2018 Marcis Pinnis, Matiss Rikters, Rihards Krislauks
	CUNI Transformer Neural MT System for WMT18 Martin Popel
	The University of Helsinki submissions to the WMT18 news task Alessandro Raganato, Yves Scherrer, Tommi Nieminen, Arvi Hurskainen, Jörg Tiedemann
	The RWTH Aachen University Supervised Machine Translation Systems for WMT 2018 Julian Schamper, Jan Rosendahl, Parnia Bahar, Yunsu Kim, Arne Nix, Hermann Ney
	The University of Cambridge's Machine Translation Systems for WMT18 Felix Stahlberg, Adrià de Gispert, Bill Byrne
	The LMU Munich Unsupervised Machine Translation Systems Dario Stojanovski, Viktor Hangya, Matthias Huck, Alexander Fraser
	Tencent Neural Machine Translation Systems for WMT18 Mingxuan Wang, Li Gong, Wenhuan Zhu, Jun Xie, Chao Bian
	The NiuTrans Machine Translation System for WMT18 Qiang Wang, Bei Li, Jiqiang Liu, Bojian Jiang, Zheyang Zhang, Yinqiao Li, Ye Lin, Tong Xiao, Jingbo Zhu
	The University of Maryland’s Chinese-English Neural Machine Translation Systems at WMT18 Weijia Xu, Marine Carpuat
11:00	Extra Test Suites
	EvalD Reference-Less Discourse Evaluation for WMT18 Ondřej Bojar, Jiří Mírovský, Kateřina Rysová, Magdalena Rysová
	The WMT’18 Morpheval test suites for English-Czech, English-German, English-Finnish and Turkish-English Franck Burlot, Yves Scherrer, Vinit Ravishankar, Ondřej Bojar, Stig-Arne Grönroos, Maarit Koponen, Tommi Nieminen, Francois Yvon
	Testsuite on Czech–English Grammatical Contrasts Silvie Cinkova, Ondřej Bojar
	A Pronoun Test Suite Evaluation of the English–German MT Systems at WMT 2018 Liane Guillou, Christian Hardmeier, Ekaterina Lapshinova-Koltunski, Sharid Loáiciga
	Fine-grained evaluation of German-English Machine Translation based on a Test Suite Vivien Macketanz, Eleftherios Avramidis, Aljoscha Burchardt, Hans Uszkoreit
	The Word Sense Disambiguation Test Suite at WMT18 Annette Rios, Mathias Müller, Rico Sennrich
11:00	Shared Task: Multimodal Translation
	LIUM-CVC Submissions for WMT18 Multimodal Translation Task Ozan Caglayan, Adrien Bardet, Fethi Bougares, Loïc Barrault, Kai Wang, Marc Masana, Luis Herranz, Joost van de Weijer
	The MeMAD Submission to the WMT18 Multimodal Translation Task Stig-Arne Grönroos, Benoit Huet, Mikko Kurimo, Jorma Laaksonen, Bernard Merialdo, Phu Pham, Mats Sjöberg, Umut Sulubacak, Jörg Tiedemann, Raphael Troncy, Raúl Vázquez
	The AFRL-Ohio State WMT18 Multimodal System: Combining Visual with Traditional Jeremy Gwinnup, Joshua Sandvick, Michael Hutt, Grant Erdmann, John Duselis, James Davis
	CUNI System for the WMT18 Multimodal Translation Task Jindřich Helcl, Jindřich Libovický, Dusan Varis
	Sheffield Submissions for WMT18 Multimodal Translation Shared Task Chiraag Lala, Pranava Swaroop Madhyastha, Carolina Scarton, Lucia Specia
	Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report Renjie Zheng, Yilin Yang, Mingbo Ma, Liang Huang
	UMONS Submission for WMT18 Multimodal Translation Task Jean-Benoit Delbrouck, Stephane Dupont
11:00	Shared Task: Biomedical Translation
	Translation of Biomedical Documents with Focus on Spanish-English Mirela-Stefania Duma, Wolfgang Menzel
	Ensemble of Translators with Automatic Selection of the Best Translation - the submission of FOKUS to the WMT 18 biomedical translation task Cristian Grozea
	LMU Munich's Neural Machine Translation Systems at WMT 2018 Matthias Huck, Dario Stojanovski, Viktor Hangya, Alexander Fraser
	Hunter NMT System for WMT18 Biomedical Translation Task: Transfer Learning in Neural Machine Translation Abdul Khan, Subhadarshi Panda, Jia Xu, Lampros Flokas
	UFRGS Participation on the WMT Biomedical Translation Shared Task Felipe Soares, Karin Becker
	Neural Machine Translation with the Transformer and Multi-Source Romance Languages for the Biomedical WMT 2018 task Brian Tubay, Marta R. Costa-jussà
12:30	🍴
14:00	Session 3: Research Papers on Embedding and Context
14:00	Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation Nikolaos Pappas, Lesly Miculicich, James Henderson
14:20	A neural interlingua for multilingual machine translation Yichao Lu, Phillip Keung, Faisal Ladhak, Vikas Bhardwaj, Shaonan Zhang, Jason Sun
14:40	Improving Neural Language Models with Weight Norm Initialization and Regularization Christian Herold, Yingbo Gao, Hermann Ney
15:00	Contextual Neural Model for Translating Bilingual Multi-Speaker Conversations Sameen Maruf, Andre F. T. Martins, Gholamreza Haffari
15:30	☕️
16:00	Session 4: Research Papers on Analysis and Data
16:00	Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation Antonio Toral, Sheila Castilho, Ke Hu, Andy Way
16:20	Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation Brian Thompson, Huda Khayrallah, Antonios Anastasopoulos, Arya D. McCarthy, Kevin Duh, Rebecca Marvin, Paul McNamee, Jeremy Gwinnup, Tim Anderson, Philipp Koehn
16:40	Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection Wei Wang, Taro Watanabe, Macduff Hughes, Tetsuji Nakagawa, Ciprian Chelba
17:00	Using Monolingual Data in Neural Machine Translation: a Systematic Study Franck Burlot, Francois Yvon

Day 2

9:00	Session 5: Shared Tasks Overview Presentations I
9:00	Results of the WMT18 Metrics Shared Task: Both characters and embeddings achieve good performance Qingsong Ma, Ondřej Bojar, Yvette Graham
9:20	Findings of the WMT 2018 Shared Task on Quality Estimation Lucia Specia, Frederic Blain, Varvara Logacheva, Ramón Astudillo, Andre F. T. Martins
9:40	Findings of the WMT 2018 Shared Task on Automatic Post-Editing Rajen Chatterjee, Matteo Negri, Raphael Rubino, Marco Turchi
9:55	Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering Philipp Koehn, Huda Khayrallah, Kenneth Heafield, Mikel L. Forcada
	Boaster Session for Research Papers Presented as Posters (Session II)
10:14	Neural Machine Translation into Language Varieties Surafel Melaku Lakew, Aliia Erofeeva, Marcello Federico
10:16	Effective Parallel Corpus Mining using Bilingual Sentence Embeddings Mandy Guo, Qinlan Shen, Yinfei Yang, Heming Ge, Daniel Cer, Gustavo Hernandez Abrego, Keith Stevens, Noah Constant, Yun-hsuan Sung, Brian Strope, Ray Kurzweil
10:18	On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation Tamer Alkhouli, Gabriel Bretschner, Hermann Ney
10:20	A Call for Clarity in Reporting BLEU Scores Matt Post
10:22	Exploring gap filling as a cheaper alternative to reading comprehension questionnaires when evaluating machine translation for gisting Mikel L. Forcada, Carolina Scarton, Lucia Specia, Barry Haddow, Alexandra Birch
10:24	Simple Fusion: Return of the Language Model Felix Stahlberg, James Cross, Veselin Stoyanov
10:26	Correcting Length Bias in Neural Machine Translation Kenton Murray, David Chiang
10:28	Extracting In-domain Training Corpora for Neural Machine Translation Using Data Selection Methods Catarina Cruz Silva, Chao-Hong Liu, Alberto Poncelas, Andy Way
10:30	☕️
11:00	Session 6: Poster Session II
11:00	Shared Task: Metrics
	Meteor++: Incorporating Copy Knowledge into Machine Translation Evaluation Yinuo Guo, Chong Ruan, Junfeng Hu
	ITER: Improving Translation Edit Rate through Optimizable Edit Costs Joybrata Panja, Sudip Kumar Naskar
	RUSE: Regressor Using Sentence Embeddings for Automatic Machine Translation Evaluation Hiroki Shimanaka, Tomoyuki Kajiwara, Mamoru Komachi
11:00	Shared Task: Quality Estimation
	Keep It or Not: Word Level Quality Estimation for Post-Editing Prasenjit Basu, Santanu Pal, Sudip Kumar Naskar
	RTM results for Predicting Translation Performance Ergun Bicici
	Neural Machine Translation for English-Tamil Himanshu Choudhary, Aditya Kumar Pathak, Rajiv Ratan Saha, Ponnurangam Kumaraguru
	The Benefit of Pseudo-Reference Translations in Quality Estimation of MT Output Melania Duma, Wolfgang Menzel
	Supervised and Unsupervised Minimalist Quality Estimators: Vicomtech’s Participation in the WMT 2018 Quality Estimation Task Thierry Etchegoyhen, Eva Martínez Garcia, Andoni Azpeitia
	Contextual Encoding for Translation Quality Estimation Junjie Hu, Wei-Cheng Chang, Yuexin Wu, Graham Neubig
	Sheffield Submissions for the WMT18 Quality Estimation Shared Task Julia Ive, Carolina Scarton, Frederic Blain, Lucia Specia
	UAlacant machine translation quality estimation at WMT 2018: a simple approach using phrase tables and feed-forward neural networks Felipe Sánchez-Martínez, Miquel Esplà-Gomis, Mikel L. Forcada
	Alibaba Submission for WMT18 Quality Estimation Task Jiayi Wang, Kai Fan, Bo Li, Fengming Zhou, Boxing Chen, Yangbin Shi, Luo Si
	Quality Estimation with Force-Decoded Attention and Cross-lingual Embeddings Elizaveta Yankovskaya, Andre Tättar, Mark Fishel
11:00	Shared Task: Automatic Post-Editing
	MS-UEdin Submission to the WMT2018 APE Shared Task: Dual-Source Transformer for Automatic Post-Editing Marcin Junczys-Dowmunt, Roman Grundkiewicz
	A Transformer-Based Multi-Source Automatic Post-Editing System Santanu Pal, Nico Herbig, Antonio Krüger, Josef van Genabith
	DFKI-MLT System Description for the WMT18 Automatic Post-editing Task Daria Pylypenko, Raphael Rubino
	Multi-encoder Transformer Network for Automatic Post-Editing Jaehun Shin, Jong-Hyeok Lee
	Multi-source transformer with combined losses for automatic post editing Amirhossein Tebbifakhr, Ruchit Agrawal, Rajen Chatterjee, Matteo Negri, Marco Turchi
11:00	Shared Task: Parallel Corpus Filtering
	The Speechmatics Parallel Corpus Filtering System for WMT18 Tom Ash, Remi Francis, Will Williams
	STACC, OOV Density and N-gram Saturation: Vicomtech’s Participation in the WMT 2018 Shared Task on Parallel Corpus Filtering Andoni Azpeitia, Thierry Etchegoyhen, Eva Martínez Garcia
	A hybrid pipeline of rules and machine learning to filter web-crawled parallel corpora Eduard Barbu, Verginica Barbu Mititelu
	Coverage and Cynicism: The AFRL Submission to the WMT 2018 Parallel Corpus Filtering Task Grant Erdmann, Jeremy Gwinnup
	MAJE Submission to the WMT2018 Shared Task on Parallel Corpus Filtering Marina Fomicheva, Jesús González-Rubio
	An Unsupervised System for Parallel Corpus Filtering Viktor Hangya, Alexander Fraser
	Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora Marcin Junczys-Dowmunt
	The JHU Parallel Corpus Filtering Systems for WMT 2018 Huda Khayrallah, Hainan Xu, Philipp Koehn
	Measuring sentence parallelism using Mahalanobis distances: The NRC unsupervised submissions to the WMT18 Parallel Corpus Filtering shared task Patrick Littell, Samuel Larkin, Darlene Stewart, Michel Simard, Cyril Goutte, Chi-kiu Lo
	Accurate semantic textual similarity for cleaning noisy parallel corpora using semantic machine translation evaluation metric: The NRC supervised submissions to the Parallel Corpus Filtering task Chi-kiu Lo, Michel Simard, Darlene Stewart, Samuel Larkin, Cyril Goutte, Patrick Littell
	Alibaba Submission to the WMT18 Parallel Corpus Filtering Task Jun Lu, Xiaoyu Lv, Yangbin Shi, Boxing Chen
	UTFPR at WMT 2018: Minimalistic Supervised Corpora Filtering for Machine Translation Gustavo Paetzold
	The ILSP/ARC submission to the WMT 2018 Parallel Corpus Filtering Shared Task Vassilis Papavassiliou, Sokratis Sofianopoulos, Prokopis Prokopidis, Stelios Piperidis
	SYSTRAN Participation to the WMT2018 Shared Task on Parallel Corpus Filtering Minh Quang Pham, Josep Crego, Jean Senellart
	Tilde’s Parallel Corpus Filtering Methods for WMT 2018 Marcis Pinnis
	The RWTH Aachen University Filtering System for the WMT 2018 Parallel Corpus Filtering Task Nick Rossenbach, Jan Rosendahl, Yunsu Kim, Miguel Graca, Aman Gokrani, Hermann Ney
	Prompsit’s submission to WMT 2018 Parallel Corpus Filtering shared task Víctor M. Sánchez-Cartagena, Marta Bañón, Sergio Ortiz Rojas, Gema Ramírez
	NICT’s Corpus Filtering Systems for the WMT18 Parallel Corpus Filtering Task Rui Wang, Benjamin Marie, Masao Utiyama, Eiichiro Sumita
12:30	🍴
14:00	Session 3: Invited Talk
14:00	Translation and the Media Alexandra Birch, University of Edinburgh
15:30	☕️
16:00	Session 8: Research Papers on Multilingual Translation
16:00	Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation Zhong Zhou, Matthias Sperber, Alexander Waibel
16:20	Trivial Transfer Learning for Low-Resource Neural Machine Translation Tom Kocmi, Ondřej Bojar
16:40	Input Combination Strategies for Multi-Source Transformer Decoder Jindřich Libovický, Jindřich Helcl, David Mareček
17:00	Parameter Sharing Methods for Multilingual Self-Attentional Translation Models Devendra Sachan and Graham Neubig

Results

General Task

Full results of the shared task: Findings of the 2018 Conference on Machine Translation (WMT18)

The winner systems were listed according to their average score and average z-score.

English →

Language pair	System	Average score	Average z-score
→ Chinese	TENCENT-ENSEMBLE	80.7	0.219
→ Czech	CUNI-TRANSFORMER	67.2	0.594
→ German	FACEBOOK-FAIR	85.5	0.653
→ Estonian	TILDE-NC-NMT	64.9	0.549
→ Finnish	NICT	64.7	0.521
→ Russian	ALIBABA-ENS	72.0	0.352
→ Turkish	ONLINE-B	66.3	0.277

→ English

Language pair	System	Average score	Average z-score
Chinese →	NIUTRANS	78.8	0.140
Czech →	CUNI-TRANSFORMER	71.8	0.298
German →	RWTH	79.9	0.413
Estonian →	TILDE-NC-NMT	73.3	0.326
Finnish →	NICT	75.2	0.153
Russian →	ALIBABA	81.0	0.215
Turkish →	ONLINE-G	70.2	0.101