WMT17
Conference on Machine Translation
The Second Conference on Machine Translation (WMT17) took place from 7 September to 8 September, 2017, in Copenhagen, Denmark. It was organised by WMT.
Tasks
- News
- Biomedical
- Automatic post-editing
- Metrics
- Quality estimation
- Multimodal
- NMT training
- Bandit learning
Schedule
Day 1
| 08:45 – 09:00 | Opening Remarks |
| 09:00 – 10:30 | Session 1: Shared Tasks Overview Presentations I |
| 09:00 – 09:40 | Shared Task: News Translation |
| Findings of the 2017 Conference on Machine Translation (WMT17) Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, Marco Turchi | |
| 09:40 – 10:10 | Shared Task: Multimodal Translation |
| Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description Desmond Elliott, Stella Frank, Loïc Barrault, Fethi Bougares, Lucia Specia | |
| 10:10 – 10:30 | Biomedical Translation |
| Findings of the WMT 2017 Biomedical Translation Shared Task Antonio Jimeno Yepes, Aurelie Neveol, Mariana Neves, Karin Verspoor, Ondrej Bojar, Arthur Boyer, Cristian Grozea, Barry Haddow, Madeleine Kittner, Yvonne Lichtblau, Pavel Pecina, Roland Roller, Rudolf Rosa, Amy Siu, Philippe Thomas, Saskia Trescher | |
| 10:30 - 11:00 | ☕️ |
| 11:00 – 12:30 | Session 2: Shared Tasks Poster Session I |
| 11:00 – 12:30 | Shared Task: News Translation |
| CUNI submission in WMT17: Chimera goes neural Roman Sudarikov, David Mareček, Tom Kocmi, Dusan Varis, Ondřej Bojar | |
| [email protected]’17 Franck Burlot, Pooyan Safari, Matthieu Labeau, Alexandre Allauzen, François Yvon | |
| SYSTRAN Purely Neural MT Engines for WMT2017 Yongchao Deng, Jungi Kim, Guillaume Klein, Catherine KOBUS, Natalia Segal, Christophe Servan, Bo Wang, Dakun Zhang, Josep Crego, Jean Senellart | |
| FBK’s Participation to the English-to-German News Translation Task of WMT 2017 Mattia Antonino Di Gangi, Nicola Bertoldi, Marcello Federico | |
| The JHU Machine Translation Systems for WMT 2017 Shuoyang Ding, Huda Khayrallah, Philipp Koehn, Matt Post, Gaurav Kumar, Kevin Duh | |
| The TALP-UPC Neural Machine Translation System for German/Finnish-English Using the Inverse Direction Model in Rescoring Carlos Escolano, Marta R. Costa-jussà, José A. R. Fonollosa | |
| LIUM Machine Translation Systems for WMT17 News Translation Task Mercedes García-Martínez, Ozan Caglayan, Walid Aransa, Adrien Bardet, Fethi Bougares, Loïc Barrault | |
| Extending hybrid word-character neural machine translation with multi-task learning of morphological analysis Stig-Arne Grönroos, Sami Virpioja, Mikko Kurimo | |
| The AFRL-MITLL WMT17 Systems: Old, New, Borrowed, BLEU Jeremy Gwinnup, Timothy Anderson, Grant Erdmann, Katherine Young, Michaeel Kazi, Elizabeth Salesky, Brian Thompson, Jonathan Taylor | |
| University of Rochester WMT 2017 NMT System Submission Chester Holtz, Chuyang Ke, Daniel Gildea | |
| LMU Munich’s Neural Machine Translation Systems for News Articles and Health Information Texts Matthias Huck, Fabienne Braune, Alexander Fraser | |
| Rule-based Machine translation from English to Finnish Arvi Hurskainen, Jörg Tiedemann | |
| NRC Machine Translation System for WMT 2017 Chi-kiu Lo, Boxing Chen, Colin Cherry, George Foster, Samuel Larkin, Darlene Stewart, Roland Kuhn | |
| The Helsinki Neural Machine Translation System Robert Östling, Yves Scherrer, Jörg Tiedemann, Gongbo Tang, Tommi Nieminen | |
| The QT21 Combined Machine Translation System for English to Latvian Jan-Thorsten Peter, Hermann Ney, Ondřej Bojar, Ngoc-Quan Pham, Jan Niehues, Alex Waibel, Franck Burlot, François Yvon, Mārcis Pinnis, Valters Sics, Joost Bastings, Miguel Rios, Wilker Aziz, Philip Williams, Frédéric Blain, Lucia Specia | |
| The RWTH Aachen University English-German and German-English Machine Translation System for WMT 2017 Jan-Thorsten Peter, Andreas Guta, Tamer Alkhouli, Parnia Bahar, Jan Rosendahl, Nick Rossenbach, Miguel Graça, Hermann Ney | |
| The Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2017 Ngoc-Quan Pham, Jan Niehues, Thanh-Le Ha, Eunah Cho, Matthias Sperber, Alexander Waibel | |
| Tilde’s Machine Translation Systems for WMT 2017 Mārcis Pinnis, Rihards Krišlauks, Toms Miks, Daiga Deksne, Valters Šics | |
| C-3MA: Tartu-Riga-Zurich Translation Systems for WMT17 Matīss Rikters, Chantal Amrhein, Maksym Del, Mark Fishel | |
| The University of Edinburgh’s Neural MT Systems for WMT17 Rico Sennrich, Alexandra Birch, Anna Currey, Ulrich Germann, Barry Haddow, Kenneth Heafield, Antonio Valerio Miceli Barone, Philip Williams | |
| XMU Neural Machine Translation Systems for WMT 17 Zhixing Tan, Boli Wang, Jinming Hu, Yidong Chen, Xiaodong Shi | |
| The JAIST Machine Translation Systems for WMT 17 Long Trieu, Trung-Tin Pham, Le-Minh Nguyen | |
| Sogou Neural Machine Translation Systems for WMT17 Yuguang Wang, Shanbo Cheng, Liyang Jiang, Jiajun Yang, Wei Chen, Muze Li, Lin Shi, Yanfeng Wang, Hongtao Yang | |
| PJIIT’s systems for WMT 2017 Conference Krzysztof Wolk, Krzysztof Marasek | |
| Hunter MT: A Course for Young Researchers in WMT17 Jia Xu, Yi Zong Kuang, Shondell Baijoo, Jacob Hyun Lee, Uman Shahzad, Mir Ahmed, Meredith Lancaster, Chris Carlan | |
| CASICT-DCU Neural Machine Translation Systems for WMT17 Jinchao Zhang, Peerachet Porkaew, Jiawei Hu, Qiuye Zhao, Qun Liu | |
| 11:00 – 12:30 | Shared Task: Multimodal Translation |
| LIUM-CVC Submissions for WMT17 Multimodal Translation Task Ozan Caglayan, Walid Aransa, Adrien Bardet, Mercedes García-Martínez, Fethi Bougares, Loïc Barrault, Marc Masana, Luis Herranz, Joost van de Weijer | |
| DCU System Report on the WMT 2017 Multi-modal Machine Translation Task Iacer Calixto, Koel Dutta Chowdhury, Qun Liu | |
| The AFRL-OSU WMT17 Multimodal Translation System: An Image Processing Approach John Duselis, Michael Hutt, Jeremy Gwinnup, James Davis, Joshua Sandvick | |
| CUNI System for the WMT17 Multimodal Translation Task Jindřich Helcl, Jindřich Libovický | |
| Generating Image Descriptions using Multilingual Data Alan Jaffe | |
| OSU Multimodal Machine Translation System Report Mingbo Ma, Dapeng Li, Kai Zhao, Liang Huang | |
| Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation Pranava Swaroop Madhyastha, Josiah Wang, Lucia Specia | |
| NICT-NAIST System for WMT17 Multimodal Translation Task Jingyi Zhang, Masao Utiyama, Eiichro Sumita, Graham Neubig, Satoshi Nakamura | |
| 11:00 – 12:30 | Shared Task: Biomedical Translation |
| Automatic Threshold Detection for Data Selection in Machine Translation Mirela-Stefania Duma, Wolfgang Menzel | |
| 12:30 – 14:00 | 🍴 |
| 14:00 – 15:30 | Session 3: Invited Talk Holger Schwenk, Facebook |
| 15:30 – 16:00 | ☕️ |
| 16:00 – 17:30 | Session 4: Research Papers on Lexicon and Morphology |
| 16:00 – 16:15 | Sense-Aware Statistical Machine Translation using Adaptive Context-Dependent Clustering Xiao Pu, Nikolaos Pappas, Andrei Popescu-Belis |
| 16:15 – 16:30 | Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings Annette Rios Gonzales, Laura Mascarell, Rico Sennrich |
| 16:30 – 16:45 | Word Representations in Factored Neural Machine Translation Franck Burlot, Mercedes García-Martínez, Loïc Barrault, Fethi Bougares, François Yvon |
| 16:45 – 17:00 | Modeling Target-Side Inflection in Neural Machine Translation Aleš Tamchyna, Marion Weller-Di Marco, Alexander Fraser |
| 17:00 – 17:15 | Evaluating the morphological competence of Machine Translation Systems Franck Burlot, François Yvon |
| 17:15 – 17:30 | Target-side Word Segmentation Strategies for Neural Machine Translation Matthias Huck, Simon Riess, Alexander Fraser |
Day 2
| 09:00 – 09:20 | Shared Task: Quality Estimation |
| 09:20 – 09:40 | Shared Task: Metrics |
| Results of the WMT17 Metrics Shared Task Ondřej Bojar, Yvette Graham, Amir Kamran | |
| 09:40 – 10:00 | Shared Task: Automatic Post-Editing |
| 10:00 – 10:15 | Shared Task: Bandit Learning |
| A Shared Task on Bandit Learning for Machine Translation Artem Sokolov, Julia Kreutzer, Kellen Sunderland, Pavel Danchenko, Witold Szymaniak, Hagen Fürstenau, Stefan Riezler | |
| 10:15 – 10:30 | Shared Task: Neural Training |
| Results of the WMT17 Neural MT Training Task Ondřej Bojar, Jindřich Helcl, Tom Kocmi, Jindřich Libovický, Tomáš Musil | |
| 10:30 - 11:00 | ☕️ |
| 11:00 – 12:30 | Session 6: Shared Tasks Poster Session II |
| 11:00 – 12:30 | Shared Task: Quality Estimation |
| Sentence-level quality estimation by predicting HTER as a multi-component metric Eleftherios Avramidis | |
| Predicting Translation Performance with Referential Translation Machines Ergun Biçici | |
| Bilexical Embeddings for Quality Estimation Frédéric Blain, Carolina Scarton, Lucia Specia | |
| Improving Machine Translation Quality Estimation with Neural Network Features Zhiming Chen, Yiming Tan, Chenlin Zhang, Qingyu Xiang, Lilin Zhang, Maoxi Li, Mingwen Wang | |
| UHH Submission to the WMT17 Quality Estimation Shared Task Melania Duma, Wolfgang Menzel | |
| Predictor-Estimator using Multilevel Task Learning with Stack Propagation for Neural Quality Estimation Hyun Kim, Jong-Hyeok Lee, Seung-Hoon Na | |
| Unbabel’s Participation in the WMT17 Translation Quality Estimation Shared Task André F. T. Martins, Fabio Kepler, Jose Monteiro | |
| Feature-Enriched Character-Level Convolutions for Text Regression Gustavo Paetzold, Lucia Specia | |
| 11:00 – 12:30 | Shared Task: Metrics |
| UHH Submission to the WMT17 Metrics Shared Task Melania Duma, Wolfgang Menzel | |
| MEANT 2.0: Accurate semantic MT evaluation for any output language Chi-kiu Lo | |
| Blend: a Novel Combined MT Metric Based on Direct Assessment - CASICT-DCU submission to WMT17 Metrics Task Qingsong Ma, Yvette Graham, Shugen Wang, Qun Liu | |
| CUNI Experiments for WMT17 Metrics Task David Mareček, Ondřej Bojar, Ondřej Hübsch, Rudolf Rosa, Dusan Varis | |
| chrF++: words helping character n-grams Maja Popović | |
| bleu2vec: the Painfully Familiar Metric on Continuous Vector Space Steroids Andre Tättar, Mark Fishel | |
| 11:00 – 12:30 | Shared Task: Automatic Post-Editing |
| LIG-CRIStAL Submission for the WMT 2017 Automatic Post-Editing Task Alexandre Berard, Laurent Besacier, Olivier Pietquin | |
| Multi-source Neural Automatic Post-Editing: FBK’s participation in the WMT 2017 APE shared task Rajen Chatterjee, M. Amin Farajian, Matteo Negri, Marco Turchi, Ankit Srivastava, Santanu Pal | |
| The AMU-UEdin Submission to the WMT 2017 Shared Task on Automatic Post-Editing Marcin Junczys-Dowmunt, Marcin Junczys-Dowmunt | |
| Ensembling Factored Neural Machine Translation Models for Automatic Post-Editing and Quality Estimation Chris Hokamp | |
| Neural Post-Editing Based on Quality Estimation Yiming Tan, Zhiming Chen, Liu Huang, Lilin Zhang, Maoxi Li, Mingwen Wang | |
| CUNI System for WMT17 Automatic Post-Editing Task Dusan Varis, Ondřej Bojar | |
| 11:00 – 12:30 | Shared Task: Bandit Learning |
| The UMD Neural Machine Translation Systems at WMT17 Bandit Learning Task Amr Sharaf, Shi Feng, Khanh Nguyen, Kiante Brantley, Hal Daumé III | |
| LIMSI Submission for WMT’17 Shared Task on Bandit Learning Guillaume Wisniewski | |
| 11:00 – 12:30 | Shared Task: Neural Training |
| Variable Mini-Batch Sizing and Pre-Trained Embeddings Mostafa Abdou, Vladan Gloncak, Ondřej Bojar | |
| The AFRL WMT17 Neural Machine Translation Training Task Submission Jeremy Gwinnup, Grant Erdmann, Katherine Young | |
| 12:30 – 14:00 | 🍴 |
| 14:00 – 15:15 | Session 7: Research Papers on Syntax and Deep Models |
| 14:00 – 14:15 | Predicting Target Language CCG Supertags Improves Neural Machine Translation Maria Nadejde, Siva Reddy, Rico Sennrich, Tomasz Dwojak, Marcin Junczys-Dowmunt, Philipp Koehn, Alexandra Birch |
| 14:15 – 14:30 | Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning Jan Niehues, Eunah Cho |
| 14:30 – 14:45 | Tree as a Pivot: Syntactic Matching Methods in Pivot Translation Akiva Miura, Graham Neubig, Katsuhito Sudoh, Satoshi Nakamura |
| 14:45 – 15:00 | Deep architectures for Neural Machine Translation Antonio Valerio Miceli Barone, Jindřich Helcl, Rico Sennrich, Barry Haddow, Alexandra Birch |
| 15:00 – 15:15 | Biasing Attention-Based Recurrent Neural Networks Using External Alignment Information Tamer Alkhouli, Hermann Ney |
| 15:15 – 16:00 | ☕️ |
| 16:00 – 17:15 | Session 8: Research Papers on Domain Adaptation and External Data |
| 16:00 – 16:15 | Effective Domain Mixing for Neural Machine Translation Denny Britz, Quoc Le, Reid Pryzant |
| 16:15 – 16:30 | Multi-Domain Neural Machine Translation through Unsupervised Adaptation M. Amin Farajian, Marco Turchi, Matteo Negri, Marcello Federico |
| 16:30 – 16:45 | Adapting Neural Machine Translation with Parallel Synthetic Data Mara Chinea-Rios, Álvaro Peris, Francisco Casacuberta |
| 16:45 – 17:00 | Copied Monolingual Data Improves Low-Resource Neural Machine Translation Anna Currey, Antonio Valerio Miceli Barone, Kenneth Heafield |
| 17:00 – 17:15 | Guiding Neural Machine Translation Decoding with External Knowledge Rajen Chatterjee, Matteo Negri, Marco Turchi, Marcello Federico, Lucia Specia, Frédéric Blain |
Results
Full results of the shared tasks: Findings of the 2017 Conference on Machine Translation (WMT17)
News translation
The results were determined with a monolingual direct assessment, the average score and average z-score.
→ English
| Language pair | System | Average score | Average z-score |
|---|---|---|---|
| Chinese → | SogouKnowing-nmt | 73.2 | 0.209 |
| Czech → | uedin-nmt | 74.6 | 0.181 |
| German → | online-B | 78.2 | 0.213 |
| Finnish → | online-B | 73.8 | 0.407 |
| Latvian → | online-B | 76.2 | 0.266 |
| Russian → | online-B | 82.0 | 0.271 |
| Turkish → | online-B | 68.8 | 0.294 |
English →
| Language pair | System | Average score | Average z-score |
|---|---|---|---|
| → Chinese | SogouKnowing-nmt | 73.2 | 0.208 |
| → Czech | uedin-nmt | 62.0 | 0.308 |
| → German | LMU-nmt-reranked | 72.9 | 0.257 |
| → Finnish | online-B | 59.6 | 0.378 |
| → Latvian | tilde-nc-nmt-smt | 54.4 | 0.196 |
| → Russian | online-B | 75.4 | 0.402 |
| → Turkish | online-B | 53.4 | 0.513 |