WMT15
Workshop on Machine Translation
The Tenth Workshop on Machine Translation (WMT15) took place from 17 September to 18 September, 2015, in Lisbon, Portugal. It was organised by WMT.
Table of contents
Tasks
Tuning task
The tuning task is a continuation of the WMT11 tunable metrics task.
Schedule
Day 1
| 09:00 – 09:05 | Opening Remarks |
| 09:05 – 09:50 | Session 1: Shared Tasks |
| 09:05 – 09:50 | Findings of the 2015 Workshop on Statistical Machine Translation (slides) Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, Marco Turchi |
| 09:50 – 10:30 | Session 2: Data Selection |
| 09:50 – 10:10 | Statistical Machine Translation with Automatic Identification of Translationese (slides) Naama Twitto, Noam Ordan, Shuly Wintner |
| 10:10 – 10:30 | Data Selection With Fewer Words (slides) Amittai Axelrod, Philip Resnik, Xiaodong He, Mari Ostendorf |
| 10:30 – 11:00 | ☕️ |
| 11:00 – 12:30 | Session 3A: Poster Session - Shared Task: Translation DFKI’s experimental hybrid MT system for WMT 2015 Eleftherios Avramidis, Maja Popović, Aljoscha Burchardt |
| ParFDA for Fast Deployment of Accurate Statistical Machine Translation Systems, Benchmarks, and Statistics Ergun Bicici, Qun Liu, Andy Way | |
| CUNI in WMT15: Chimera Strikes Again Ondřej Bojar, Aleš Tamchyna | |
| CimS - The CIS and IMS Joint Submission to WMT 2015 addressing morphological and syntactic differences in English to German SMT Fabienne Cap, Marion Weller, Anita Ramm, Alexander Fraser | |
| The Karlsruhe Institute of Technology Translation Systems for the WMT 2015 Eunah Cho, Thanh-Le Ha, Jan Niehues, Teresa Herrmann, Mohammed Mediani, Yuqi Zhang, Alex Waibel | |
| New Language Pairs in TectoMT Ondřej Dušek, Luís Gomes, Michal Novák, Martin Popel, Rudolf Rosa | |
| Tuning Phrase-Based Segmented Translation for a Morphologically Complex Target Language Stig-Arne Grönroos, Sami Virpioja, Mikko Kurimo | |
| The AFRL-MITLL WMT15 System: There’s More than One Way to Decode It! Jeremy Gwinnup, Tim Anderson, Grant Erdmann, Katherine Young, Christina May, Michaeel Kazi, Elizabeth Salesky, Brian Thompson | |
| The KIT-LIMSI Translation System for WMT 2015 Thanh-Le Ha, Quoc-Khanh Do, Eunah Cho, Jan Niehues, Alexandre Allauzen, François Yvon, Alex Waibel | |
| The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015 Barry Haddow, Matthias Huck, Alexandra Birch, Nikolay Bogoychev, Philipp Koehn | |
| Montreal Neural Machine Translation Systems for WMT’15 Sébastien Jean, Orhan Firat, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio | |
| GF Wide-coverage English-Finnish MT system for WMT 2015 Prasanth Kolachina, Aarne Ranta | |
| [email protected]’15: Translation Task Benjamin Marie, Alexandre Allauzen, Franck Burlot, Quoc-Khanh Do, Julia Ive, elena knyazeva, Matthieu Labeau, Thomas Lavergne, Kevin Löser, Nicolas Pécheux, François Yvon | |
| UdS-Sant: English–German Hybrid Machine Translation System Santanu Pal, Sudip Naskar, Josef van Genabith | |
| The RWTH Aachen German-English Machine Translation System for WMT 2015 Jan-Thorsten Peter, Farzad Toutounchi, Joern Wuebker, Hermann Ney | |
| Exact Decoding with Multi Bottom-Up Tree Transducers Daniel Quernheim | |
| Sheffield Systems for the Finnish-English WMT Translation Task David Steele, Karin Sim Smith, Lucia Specia | |
| Morphological Segmentation and OPUS for Finnish-English Machine Translation Jörg Tiedemann, Filip Ginter, Jenna Kanerva | |
| Abu-MaTran at WMT 2015 Translation Task: Morphological Segmentation and Web Crawling Raphael Rubino, Tommi Pirinen, Miquel Esplà-Gomis, Nikola Ljubešić, Sergio Ortiz Rojas, Vassilis Papavassiliou, Prokopis Prokopidis, Antonio Toral | |
| The University of Illinois submission to the WMT 2015 Shared Translation Task Lane Schwartz, Bill Bryce, Chase Geigle, Sean Massung, Yisi Liu, Haoruo Peng, Vignesh Raja, Subhro Roy, Shyam Upadhyay | |
| Edinburgh’s Syntax-Based Systems at WMT 2015 Philip Williams, Rico Sennrich, Maria Nadejde, Matthias Huck, Philipp Koehn | |
| 11:00 – 12:30 | Session 3B: Poster Session - Shared Task: Automatic Post-Editing |
| The FBK Participation in the WMT15 Automatic Post-editing Shared Task Rajen Chatterjee, Marco Turchi, Matteo Negri | |
| USAAR-SAPE: An English–Spanish Statistical Automatic Post-Editing System Santanu Pal, Mihaela Vela, Sudip Kumar Naskar, Josef van Genabith | |
| Why Predicting Post-Edition is so Hard? Failure Analysis of LIMSI Submission to the APE Shared Task Guillaume Wisniewski, Nicolas Pécheux, François Yvon | |
| 12:30 – 14:00 | 🍴 |
| 14:00 – 15:30 | Session 4: Invited Talk A Practical Guide to Real-Time Neural Translation Jacob Devlin |
| 15:30 – 16:00 | ☕️ |
| 16:00 – 17:00 | Session 5: Syntax-Based Translation and Rescoring |
| 16:00 – 16:20 | Hierarchical Machine Translation With Discontinuous Phrases (slides) Miriam Kaeshammer |
| 16:20 – 16:40 | Discontinuous Statistical Machine Translation with Target-Side Dependency Syntax (slides) Nina Seemann, Andreas Maletti |
| 16:40 – 17:00 | ListNet-based MT Rescoring (slides) Jan Niehues, Quoc-Khanh Do, Alexandre Allauzen, Alex Waibel |
Day 2
| 09:00 – 09:50 | Session 6: Shared Tasks |
| 09:00 – 09:20 | Overview of the Quality Estimation Task (slides) Multiple Speakers Results of the WMT15 Metrics Shared Task Miloš Stanojević, Amir Kamran, Philipp Koehn, Ondřej Bojar Results of the WMT15 Tuning Shared Task Miloš Stanojević, Amir Kamran, Ondřej Bojar |
| 09:50 – 10:30 | Session 7: Translation Modeling |
| 09:50 – 10:10 | Extended Translation Models in Phrase-based Decoding (slides) Andreas Guta, Joern Wuebker, Miguel Graca, Yunsu Kim, Hermann Ney |
| 10:10 – 10:30 | Investigations on Phrase-based Decoding with Recurrent Neural Network Language and Translation Models (slides) Tamer Alkhouli, Felix Rietig, Hermann Ney |
| 10:30 – 11:00 | ☕️ |
| 11:00 – 12:30 | Session 8A: Poster Session - Shared Task: Quality Estimation Referential Translation Machines for Predicting Translation Quality and Related Statistics Ergun Bicici, Qun Liu, Andy Way |
| UAlacant word-level machine translation quality estimation system at WMT 2015 Miquel Esplà-Gomis, Felipe Sánchez-Martínez, Mikel Forcada | |
| QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation Julia Kreutzer, Shigehiko Schamoni, Stefan Riezler | |
| LORIA System for the WMT15 Quality Estimation Shared Task David Langlois | |
| Data enhancement and selection strategies for the word-level Quality Estimation Varvara Logacheva, Chris Hokamp, Lucia Specia | |
| USHEF and USAAR-USHEF participation in the WMT15 QE shared task Carolina Scarton, Liling Tan, Lucia Specia | |
| SHEF-NN: Translation Quality Estimation with Neural Networks Kashif Shah, Varvara Logacheva, Gustavo Paetzold, Frédéric Blain, Daniel Beck, Fethi Bougares, Lucia Specia | |
| Strategy-Based Technology for Estimating MT Quality Liugang Shang, Dongfeng Cai, Duo Ji | |
| UGENT-LT3 SCATE System for Machine Translation Quality Estimation Arda Tezcan, Veronique Hoste, Bart Desmet, Lieve Macken | |
| 11:00 – 12:30 | Session 8B: Poster Session - Shared Task: Metrics |
| Multi-level Evaluation for Machine Translation Boxing Chen, Hongyu Guo, Roland Kuhn | |
| VERTa: a Linguistically-motivated Metric at the WMT15 Metrics Task Elisabet Comelles, Jordi Atserias | |
| UPF-Cobalt Submission to WMT15 Metrics Task Marina Fomicheva, Núria Bel, Iria da Cunha, Anton Malinovskiy | |
| Machine Translation Evaluation using Recurrent Neural Networks Rohit Gupta, Constantin Orasan, Josef van Genabith | |
| Alignment-based sense selection in METEOR and the RATATOUILLE recipe Benjamin Marie, Marianna Apidianaki | |
| chrF: character n-gram F-score for automatic MT evaluation Maja Popović | |
| BEER 1.1: ILLC UvA submission to metrics and tuning task Miloš Stanojević, Khalil Sima’an | |
| Predicting Machine Translation Adequacy with Document Embeddings Mihaela Vela, Liling Tan | |
| LeBLEU: N-gram-based Translation Evaluation Score for Morphologically Complex Languages Sami Virpioja, Stig-Arne Grönroos | |
| CASICT-DCU Participation in WMT2015 Metrics Task Hui Yu, Qingsong Ma, Xiaofeng Wu, Qun Liu | |
| 11:00 – 12:30 | Session 8C: Poster Session - Shared Task: Tuning |
| Drem: The AFRL Submission to the WMT15 Tuning Task Grant Erdmann, Jeremy Gwinnup | |
| MT Tuning on RED: A Dependency-Based Evaluation Metric Liangyou Li, Hui Yu, Qun Liu | |
| Improving evaluation and optimization of MT systems against MEANT Chi-kiu Lo, Philipp Dowling, Dekai Wu | |
| 12:30 – 14:00 | 🍴 |
| 14:00 – 15:20 | Session 9: Evaluation and System Combination |
| An Investigation of Machine Translation Evaluation Metrics in Cross-lingual Question Answering Kyoshiro Sugiyama, Masahiro Mizukami, Graham Neubig, Koichiro Yoshino, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura | |
| Dependency Analysis of Scrambled References for Better Evaluation of Japanese Translation (slides) Hideki Isozaki, Natsume Kouchi | |
| How do Humans Evaluate Machine Translation Francisco Guzmán, Ahmed Abdelali, Irina Temnikova, Hassan Sajjad, Stephan Vogel | |
| Local System Voting Feature for Machine Translation System Combination (slides) Markus Freitag, Jan-Thorsten Peter, Stephan Peitz, Minwei Feng, Hermann Ney | |
| 15:20 – 16:00 | ☕️ |
| 16:00 – 17:00 | Session 10: Closing and Open Discussion |
Results
Full results of the shared tasks: Findings of the 2015 Workshop on Statistical Machine Translation
News translation
The results were determined with a relative ranking through the TrueSkill model.
→ English
| Language pair | System | Score |
|---|---|---|
| Czech → | ONLINE-B | 0.619 |
| French → | ONLINE-B | 0.498 |
| German → | ONLINE-B | 0.567 |
| Russian → | ONLINE-G | 0.494 |
| Finnish → | ONLINE-B | 0.675 |
English →
| Language pair | System | Score |
|---|---|---|
| → Czech | CU-CHIMERA | 0.686 |
| → French | LIMSI-CNRS | 0.540 |
| → German | UEDIN-SYNTAX | 0.359 |
| → Russian | PROMT-RULE | 1.015 |
| → Finnish | ONLINE-B | 1.069 |