WMT18

Conference on Machine Translation


The Third Conference on Machine Translation (WMT18) took place from 31 October to 1 November 2018 in Brussels, Belgium. It was organised by WMT.

statmt.org/wmt18/program.html

Table of contents
  1. Tasks
  2. Schedule
    1. Day 1
    2. Day 2
  3. Results
    1. News translation
      1. → English
      2. English →

Tasks

Schedule

Day 1

   
08:45 – 09:00 Opening Remarks
09:00 – 10:30 Session 1: Shared Tasks Overview Presentations I
09:00 – 09:30 Findings of the 2018 Conference on Machine Translation (WMT18)
Ondřej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Christof Monz
09:30 – 9:50 Findings of the Third Shared Task on Multimodal Machine Translation
Loïc Barrault, Fethi Bougares, Lucia Specia, Chiraag Lala, Desmond Elliott, Stella Frank
09:50 – 10:10 Findings of the WMT 2018 Biomedical Translation Shared Task: Evaluation on Medline test sets
Mariana Neves, Antonio Jimeno Yepes, Aurelie Neveol, Cristian Grozea, Amy Siu, Madeleine Kittner, Karin Verspoor
  Boaster Session for Research Papers Presented as Posters (Session I)
10:16 – 10:18 Scaling Neural Machine Translation
Myle Ott, Sergey Edunov, David Grangier, Michael Auli
10:18 – 10:20 Character-level Chinese-English Translation through ASCII Encoding
Nikola Nikolov, Yuhuang Hu, Mi Xue Tan, Richard H.R. Hahnloser
10:20 – 10:22 Neural Machine Translation of Logographic Language Using Sub-character Level Information
Longtu Zhang, Mamoru Komachi
10:22 – 10:24 An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation
Gongbo Tang, Rico Sennrich, Joakim Nivre
10:24 – 10:26 Discourse-Related Language Contrasts in English-Croatian Human and Machine Translation
Margita Šoštarić, Christian Hardmeier, Sara Stymne
10:26 – 10:28 Coreference and Coherence in Neural Machine Translation: A Study Using Oracle Experiments
Dario Stojanovski, Alexander Fraser
10:28 – 10:30 A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation
Mathias Müller, Annette Rios, Elena Voita, Rico Sennrich
10:30 - 11:00 ☕️
11:00 – 12:30 Session 2: Poster Session I
11:00 – 12:30 Shared Task: News Translation
  An Empirical Study of Machine Translation for the Shared Task of WMT18
Chao Bei, Hao Zong, Yiming Wang, Baoyong Fan, Shiqi Li, Conghu Yuan
  Robust parfda Statistical Machine Translation Results
Ergun Bicici
  The TALP-UPC Machine Translation Systems for WMT18 News Shared Translation Task
Noe Casas, Carlos Escolano, Marta R. Costa-jussà, Jose A. R. Fonollosa
  Phrase-based Unsupervised Machine Translation with Compositional Phrase Embeddings
Maksym Del, Andre Tättar, Mark Fishel
  Alibaba’s Neural Machine Translation Systems for WMT18
Yongchao Deng, Shanbo Cheng, Jun Lu, Kai Song, Jingang Wang, Shenglan Wu, Liang Yao, Guchun Zhang, Haibo Zhang, Pei Zhang, Changfeng Zhu, Boxing Chen
  The RWTH Aachen University English-German and German-English Unsupervised Neural Machine Translation Systems for WMT 2018
Miguel Graca, Yunsu Kim, Julian Schamper, Jiahui Geng, Hermann Ney
  Cognate-aware morphological segmentation for multilingual neural translation
Stig-Arne Grönroos, Sami Virpioja, Mikko Kurimo
  The AFRL WMT18 Systems: Ensembling, Continuation and Combination
Jeremy Gwinnup, Tim Anderson, Grant Erdmann, Katherine Young
  The University of Edinburgh’s Submissions to the WMT18 News Translation Task
Barry Haddow, Nikolay Bogoychev, Denis Emelin, Ulrich Germann, Roman Grundkiewicz, Kenneth Heafield, Antonio Valerio Miceli Barone, Rico Sennrich
  TencentFmRD Neural Machine Translation for WMT18
Bojie Hu, Ambyer Han, Shen Huang
  The MLLP-UPV German-English Machine Translation System for WMT18
Javier Iranzo-Sánchez, Pau Baquero-Arnal, Goncal V. Garces Díaz-Munío, Adrià Martínez-Villaronga, Jorge Civera, Alfons Juan
  Microsoft’s Submission to the WMT2018 News Translation Task: How I Learned to Stop Worrying and Love the Data
Marcin Junczys-Dowmunt
  CUNI Submissions in WMT18
Tom Kocmi, Roman Sudarikov, Ondřej Bojar
  The JHU Machine Translation Systems for WMT 2018
Philipp Koehn, Kevin Duh, Brian Thompson
  JUCBNMT at WMT2018 News Translation Task: Character Based Neural Machine Translation of Finnish to English
Sainik Kumar Mahata, Dipankar Das, Sivaji Bandyopadhyay
  NICT’s Neural and Statistical Machine Translation Systems for the WMT18 News Translation Task
Benjamin Marie, Rui Wang, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita
  PROMT Systems for WMT 2018 Shared Translation Task
Alexander Molchanov
  NTT’s Neural Machine Translation Systems for WMT 2018
Makoto Morishita, Jun Suzuki, Masaaki Nagata
  The Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2018
Ngoc-Quan Pham, Jan Niehues, Alexander Waibel
  Tilde’s Machine Translation Systems for WMT 2018
Marcis Pinnis, Matiss Rikters, Rihards Krislauks
  CUNI Transformer Neural MT System for WMT18
Martin Popel
  The University of Helsinki submissions to the WMT18 news task
Alessandro Raganato, Yves Scherrer, Tommi Nieminen, Arvi Hurskainen, Jörg Tiedemann
  The RWTH Aachen University Supervised Machine Translation Systems for WMT 2018
Julian Schamper, Jan Rosendahl, Parnia Bahar, Yunsu Kim, Arne Nix, Hermann Ney
  The University of Cambridge’s Machine Translation Systems for WMT18
Felix Stahlberg, Adrià de Gispert, Bill Byrne
  The LMU Munich Unsupervised Machine Translation Systems
Dario Stojanovski, Viktor Hangya, Matthias Huck, Alexander Fraser
  Tencent Neural Machine Translation Systems for WMT18
Mingxuan Wang, Li Gong, Wenhuan Zhu, Jun Xie, Chao Bian
  The NiuTrans Machine Translation System for WMT18
Qiang Wang, Bei Li, Jiqiang Liu, Bojian Jiang, Zheyang Zhang, Yinqiao Li, Ye Lin, Tong Xiao, Jingbo Zhu
  The University of Maryland’s Chinese-English Neural Machine Translation Systems at WMT18
Weijia Xu, Marine Carpuat
11:00 – 12:30 Extra Test Suites
  EvalD Reference-Less Discourse Evaluation for WMT18
Ondřej Bojar, Jiří Mírovský, Kateřina Rysová, Magdalena Rysová
  The WMT’18 Morpheval test suites for English-Czech, English-German, English-Finnish and Turkish-English
Franck Burlot, Yves Scherrer, Vinit Ravishankar, Ondřej Bojar, Stig-Arne Grönroos, Maarit Koponen, Tommi Nieminen, Francois Yvon
  Testsuite on Czech–English Grammatical Contrasts
Silvie Cinkova, Ondřej Bojar
  A Pronoun Test Suite Evaluation of the English–German MT Systems at WMT 2018
Liane Guillou, Christian Hardmeier, Ekaterina Lapshinova-Koltunski, Sharid Loáiciga
  Fine-grained evaluation of German-English Machine Translation based on a Test Suite
Vivien Macketanz, Eleftherios Avramidis, Aljoscha Burchardt, Hans Uszkoreit
  The Word Sense Disambiguation Test Suite at WMT18
Annette Rios, Mathias Müller, Rico Sennrich
11:00 – 12:30 Shared Task: Multimodal Translation
  LIUM-CVC Submissions for WMT18 Multimodal Translation Task
Ozan Caglayan, Adrien Bardet, Fethi Bougares, Loïc Barrault, Kai Wang, Marc Masana, Luis Herranz, Joost van de Weijer
  The MeMAD Submission to the WMT18 Multimodal Translation Task
Stig-Arne Grönroos, Benoit Huet, Mikko Kurimo, Jorma Laaksonen, Bernard Merialdo, Phu Pham, Mats Sjöberg, Umut Sulubacak, Jörg Tiedemann, Raphael Troncy, Raúl Vázquez
  The AFRL-Ohio State WMT18 Multimodal System: Combining Visual with Traditional
Jeremy Gwinnup, Joshua Sandvick, Michael Hutt, Grant Erdmann, John Duselis, James Davis
  CUNI System for the WMT18 Multimodal Translation Task
Jindřich Helcl, Jindřich Libovický, Dusan Varis
  Sheffield Submissions for WMT18 Multimodal Translation Shared Task
Chiraag Lala, Pranava Swaroop Madhyastha, Carolina Scarton, Lucia Specia
  Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report
Renjie Zheng, Yilin Yang, Mingbo Ma, Liang Huang
  UMONS Submission for WMT18 Multimodal Translation Task
Jean-Benoit Delbrouck, Stephane Dupont
11:00 – 12:30 Shared Task: Biomedical Translation
  Translation of Biomedical Documents with Focus on Spanish-English
Mirela-Stefania Duma, Wolfgang Menzel
  Ensemble of Translators with Automatic Selection of the Best Translation - the submission of FOKUS to the WMT 18 biomedical translation task
Cristian Grozea
  LMU Munich’s Neural Machine Translation Systems at WMT 2018
Matthias Huck, Dario Stojanovski, Viktor Hangya, Alexander Fraser
  Hunter NMT System for WMT18 Biomedical Translation Task: Transfer Learning in Neural Machine Translation
Abdul Khan, Subhadarshi Panda, Jia Xu, Lampros Flokas
  UFRGS Participation on the WMT Biomedical Translation Shared Task
Felipe Soares, Karin Becker
  Neural Machine Translation with the Transformer and Multi-Source Romance Languages for the Biomedical WMT 2018 task
Brian Tubay, Marta R. Costa-jussà
12:30 – 14:00 🍴
14:00 – 15:20 Session 3: Research Papers on Embedding and Context
14:00 – 14:20 Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation
Nikolaos Pappas, Lesly Miculicich, James Henderson
14:20 – 14:40 A neural interlingua for multilingual machine translation
Yichao Lu, Phillip Keung, Faisal Ladhak, Vikas Bhardwaj, Shaonan Zhang, Jason Sun
14:40 – 15:00 Improving Neural Language Models with Weight Norm Initialization and Regularization
Christian Herold, Yingbo Gao, Hermann Ney
15:00 – 15:20 Contextual Neural Model for Translating Bilingual Multi-Speaker Conversations
Sameen Maruf, Andre F. T. Martins, Gholamreza Haffari
15:30 – 16:00 ☕️
16:00 – 17:20 Session 4: Research Papers on Analysis and Data
16:00 – 16:20 Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation
Antonio Toral, Sheila Castilho, Ke Hu, Andy Way
16:20 – 16:40 Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation
Brian Thompson, Huda Khayrallah, Antonios Anastasopoulos, Arya D. McCarthy, Kevin Duh, Rebecca Marvin, Paul McNamee, Jeremy Gwinnup, Tim Anderson, Philipp Koehn
16:40 – 17:00 Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection
Wei Wang, Taro Watanabe, Macduff Hughes, Tetsuji Nakagawa, Ciprian Chelba
17:00 – 17:20 Using Monolingual Data in Neural Machine Translation: a Systematic Study
Franck Burlot, Francois Yvon

Day 2

   
09:00 – 10:30 Session 5: Shared Tasks Overview Presentations I
09:00 – 09:20 Results of the WMT18 Metrics Shared Task: Both characters and embeddings achieve good performance
Qingsong Ma, Ondřej Bojar, Yvette Graham
09:20 – 09:40 Findings of the WMT 2018 Shared Task on Quality Estimation
Lucia Specia, Frederic Blain, Varvara Logacheva, Ramón Astudillo, Andre F. T. Martins
09:40 – 09:55 Findings of the WMT 2018 Shared Task on Automatic Post-Editing
Rajen Chatterjee, Matteo Negri, Raphael Rubino, Marco Turchi
09:55 – 10:10 Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering
Philipp Koehn, Huda Khayrallah, Kenneth Heafield, Mikel L. Forcada
  Boaster Session for Research Papers Presented as Posters (Session II)
10:14 – 10:16 Neural Machine Translation into Language Varieties
Surafel Melaku Lakew, Aliia Erofeeva, Marcello Federico
10:16 – 10:18 Effective Parallel Corpus Mining using Bilingual Sentence Embeddings
Mandy Guo, Qinlan Shen, Yinfei Yang, Heming Ge, Daniel Cer, Gustavo Hernandez Abrego, Keith Stevens, Noah Constant, Yun-hsuan Sung, Brian Strope, Ray Kurzweil
10:18 – 10:20 On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation
Tamer Alkhouli, Gabriel Bretschner, Hermann Ney
10:20 – 10:22 A Call for Clarity in Reporting BLEU Scores
Matt Post
10:22 – 10:24 Exploring gap filling as a cheaper alternative to reading comprehension questionnaires when evaluating machine translation for gisting
Mikel L. Forcada, Carolina Scarton, Lucia Specia, Barry Haddow, Alexandra Birch
10:24 – 10:26 Simple Fusion: Return of the Language Model
Felix Stahlberg, James Cross, Veselin Stoyanov
10:26 – 10:28 Correcting Length Bias in Neural Machine Translation
Kenton Murray, David Chiang
10:28 – 10:30 Extracting In-domain Training Corpora for Neural Machine Translation Using Data Selection Methods
Catarina Cruz Silva, Chao-Hong Liu, Alberto Poncelas, Andy Way
10:30 - 11:00 ☕️
11:00 – 12:30 Session 6: Poster Session II
11:00 – 12:30 Shared Task: Metrics
  Meteor++: Incorporating Copy Knowledge into Machine Translation Evaluation
Yinuo Guo, Chong Ruan, Junfeng Hu
  ITER: Improving Translation Edit Rate through Optimizable Edit Costs
Joybrata Panja, Sudip Kumar Naskar
  RUSE: Regressor Using Sentence Embeddings for Automatic Machine Translation Evaluation
Hiroki Shimanaka, Tomoyuki Kajiwara, Mamoru Komachi
11:00 – 12:30 Shared Task: Quality Estimation
  Keep It or Not: Word Level Quality Estimation for Post-Editing
Prasenjit Basu, Santanu Pal, Sudip Kumar Naskar
  RTM results for Predicting Translation Performance
Ergun Bicici
  Neural Machine Translation for English-Tamil
Himanshu Choudhary, Aditya Kumar Pathak, Rajiv Ratan Saha, Ponnurangam Kumaraguru
  The Benefit of Pseudo-Reference Translations in Quality Estimation of MT Output
Melania Duma, Wolfgang Menzel
  Supervised and Unsupervised Minimalist Quality Estimators: Vicomtech’s Participation in the WMT 2018 Quality Estimation Task
Thierry Etchegoyhen, Eva Martínez Garcia, Andoni Azpeitia
  Contextual Encoding for Translation Quality Estimation
Junjie Hu, Wei-Cheng Chang, Yuexin Wu, Graham Neubig
  Sheffield Submissions for the WMT18 Quality Estimation Shared Task
Julia Ive, Carolina Scarton, Frederic Blain, Lucia Specia
  UAlacant machine translation quality estimation at WMT 2018: a simple approach using phrase tables and feed-forward neural networks
Felipe Sánchez-Martínez, Miquel Esplà-Gomis, Mikel L. Forcada
  Alibaba Submission for WMT18 Quality Estimation Task
Jiayi Wang, Kai Fan, Bo Li, Fengming Zhou, Boxing Chen, Yangbin Shi, Luo Si
  Quality Estimation with Force-Decoded Attention and Cross-lingual Embeddings
Elizaveta Yankovskaya, Andre Tättar, Mark Fishel
11:00 – 12:30 Shared Task: Automatic Post-Editing
  MS-UEdin Submission to the WMT2018 APE Shared Task: Dual-Source Transformer for Automatic Post-Editing
Marcin Junczys-Dowmunt, Roman Grundkiewicz
  A Transformer-Based Multi-Source Automatic Post-Editing System
Santanu Pal, Nico Herbig, Antonio Krüger, Josef van Genabith
  DFKI-MLT System Description for the WMT18 Automatic Post-editing Task
Daria Pylypenko, Raphael Rubino
  Multi-encoder Transformer Network for Automatic Post-Editing
Jaehun Shin, Jong-Hyeok Lee
  Multi-source transformer with combined losses for automatic post editing
Amirhossein Tebbifakhr, Ruchit Agrawal, Rajen Chatterjee, Matteo Negri, Marco Turchi
11:00 – 12:30 Shared Task: Parallel Corpus Filtering
  The Speechmatics Parallel Corpus Filtering System for WMT18
Tom Ash, Remi Francis, Will Williams
  STACC, OOV Density and N-gram Saturation: Vicomtech’s Participation in the WMT 2018 Shared Task on Parallel Corpus Filtering
Andoni Azpeitia, Thierry Etchegoyhen, Eva Martínez Garcia
  A hybrid pipeline of rules and machine learning to filter web-crawled parallel corpora
Eduard Barbu, Verginica Barbu Mititelu
  Coverage and Cynicism: The AFRL Submission to the WMT 2018 Parallel Corpus Filtering Task
Grant Erdmann, Jeremy Gwinnup
  MAJE Submission to the WMT2018 Shared Task on Parallel Corpus Filtering
Marina Fomicheva, Jesús González-Rubio
  An Unsupervised System for Parallel Corpus Filtering
Viktor Hangya, Alexander Fraser
  Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora
Marcin Junczys-Dowmunt
  The JHU Parallel Corpus Filtering Systems for WMT 2018
Huda Khayrallah, Hainan Xu, Philipp Koehn
  Measuring sentence parallelism using Mahalanobis distances: The NRC unsupervised submissions to the WMT18 Parallel Corpus Filtering shared task
Patrick Littell, Samuel Larkin, Darlene Stewart, Michel Simard, Cyril Goutte, Chi-kiu Lo
  Accurate semantic textual similarity for cleaning noisy parallel corpora using semantic machine translation evaluation metric: The NRC supervised submissions to the Parallel Corpus Filtering task
Chi-kiu Lo, Michel Simard, Darlene Stewart, Samuel Larkin, Cyril Goutte, Patrick Littell
  Alibaba Submission to the WMT18 Parallel Corpus Filtering Task
Jun Lu, Xiaoyu Lv, Yangbin Shi, Boxing Chen
  UTFPR at WMT 2018: Minimalistic Supervised Corpora Filtering for Machine Translation
Gustavo Paetzold
  The ILSP/ARC submission to the WMT 2018 Parallel Corpus Filtering Shared Task
Vassilis Papavassiliou, Sokratis Sofianopoulos, Prokopis Prokopidis, Stelios Piperidis
  SYSTRAN Participation to the WMT2018 Shared Task on Parallel Corpus Filtering
Minh Quang Pham, Josep Crego, Jean Senellart
  Tilde’s Parallel Corpus Filtering Methods for WMT 2018
Marcis Pinnis
  The RWTH Aachen University Filtering System for the WMT 2018 Parallel Corpus Filtering Task
Nick Rossenbach, Jan Rosendahl, Yunsu Kim, Miguel Graca, Aman Gokrani, Hermann Ney
  Prompsit’s submission to WMT 2018 Parallel Corpus Filtering shared task
Víctor M. Sánchez-Cartagena, Marta Bañón, Sergio Ortiz Rojas, Gema Ramírez
  NICT’s Corpus Filtering Systems for the WMT18 Parallel Corpus Filtering Task
Rui Wang, Benjamin Marie, Masao Utiyama, Eiichiro Sumita
12:30 – 14:00 🍴
14:00 – 15:30 Session 3: Invited Talk
14:00 – 15:30 Translation and the Media
Alexandra Birch, University of Edinburgh
15:30 – 16:00 ☕️
16:00 – 17:20 Session 8: Research Papers on Multilingual Translation
16:00 – 16:20 Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation
Zhong Zhou, Matthias Sperber, Alexander Waibel
16:20 – 16:40 Trivial Transfer Learning for Low-Resource Neural Machine Translation
Tom Kocmi, Ondřej Bojar
16:40 – 17:00 Input Combination Strategies for Multi-Source Transformer Decoder
Jindřich Libovický, Jindřich Helcl, David Mareček
17:00 – 17:20 Parameter Sharing Methods for Multilingual Self-Attentional Translation Models
Devendra Sachan and Graham Neubig

Results

News translation

Full results of the shared task: Findings of the 2018 Conference on Machine Translation (WMT18)

The results were determined with a monolingual direct assessment, the average score and average z-score.

→ English

Language pair System Average score Average z-score
Chinese → NIUTRANS 78.8 0.140
Czech → CUNI-TRANSFORMER 71.8 0.298
German → RWTH 79.9 0.413
Estonian → TILDE-NC-NMT 73.3 0.326
Finnish → NICT 75.2 0.153
Russian → ALIBABA 81.0 0.215
Turkish → ONLINE-G 70.2 0.101

English →

Language pair System Average score Average z-score
→ Chinese TENCENT-ENSEMBLE 80.7 0.219
→ Czech CUNI-TRANSFORMER 67.2 0.594
→ German FACEBOOK-FAIR 85.5 0.653
→ Estonian TILDE-NC-NMT 64.9 0.549
→ Finnish NICT 64.7 0.521
→ Russian ALIBABA-ENS 72.0 0.352
→ Turkish ONLINE-B 66.3 0.277

Want to learn more about WMT18?


Edit this article →

Machine Translate is created and edited by contributors like you!

Learn more about contributing →

Licensed under CC-BY-SA-4.0.

Cite this article →