WMT15

Workshop on Machine Translation


The Tenth Workshop on Machine Translation (WMT15) took place from 17 September to 18 September, 2015, in Lisbon, Portugal. It was organised by WMT.

statmt.org/wmt15/program.html

Table of contents
  1. Tasks
    1. Tuning task
  2. Schedule
    1. Day 1
    2. Day 2
  3. Results
    1. News translation
      1. → English
      2. English →

Tasks

Tuning task

The tuning task is a continuation of the WMT11 tunable metrics task.

Schedule

Day 1

   
09:00 – 09:05 Opening Remarks
09:05 – 09:50 Session 1: Shared Tasks
09:05 – 09:50 Findings of the 2015 Workshop on Statistical Machine Translation (slides)
Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, Marco Turchi
09:50 – 10:30 Session 2: Data Selection
09:50 – 10:10 Statistical Machine Translation with Automatic Identification of Translationese (slides)
Naama Twitto, Noam Ordan, Shuly Wintner
10:10 – 10:30 Data Selection With Fewer Words (slides)
Amittai Axelrod, Philip Resnik, Xiaodong He, Mari Ostendorf
10:30 – 11:00 ☕️
11:00 – 12:30 Session 3A: Poster Session - Shared Task: Translation
DFKI’s experimental hybrid MT system for WMT 2015
Eleftherios Avramidis, Maja Popović, Aljoscha Burchardt
  ParFDA for Fast Deployment of Accurate Statistical Machine Translation Systems, Benchmarks, and Statistics
Ergun Bicici, Qun Liu, Andy Way
  CUNI in WMT15: Chimera Strikes Again
Ondřej Bojar, Aleš Tamchyna
  CimS - The CIS and IMS Joint Submission to WMT 2015 addressing morphological and syntactic differences in English to German SMT
Fabienne Cap, Marion Weller, Anita Ramm, Alexander Fraser
  The Karlsruhe Institute of Technology Translation Systems for the WMT 2015
Eunah Cho, Thanh-Le Ha, Jan Niehues, Teresa Herrmann, Mohammed Mediani, Yuqi Zhang, Alex Waibel
  New Language Pairs in TectoMT
Ondřej Dušek, Luís Gomes, Michal Novák, Martin Popel, Rudolf Rosa
  Tuning Phrase-Based Segmented Translation for a Morphologically Complex Target Language
Stig-Arne Grönroos, Sami Virpioja, Mikko Kurimo
  The AFRL-MITLL WMT15 System: There’s More than One Way to Decode It!
Jeremy Gwinnup, Tim Anderson, Grant Erdmann, Katherine Young, Christina May, Michaeel Kazi, Elizabeth Salesky, Brian Thompson
  The KIT-LIMSI Translation System for WMT 2015
Thanh-Le Ha, Quoc-Khanh Do, Eunah Cho, Jan Niehues, Alexandre Allauzen, François Yvon, Alex Waibel
  The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015
Barry Haddow, Matthias Huck, Alexandra Birch, Nikolay Bogoychev, Philipp Koehn
  Montreal Neural Machine Translation Systems for WMT’15
Sébastien Jean, Orhan Firat, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio
  GF Wide-coverage English-Finnish MT system for WMT 2015
Prasanth Kolachina, Aarne Ranta
  LIMSI@WMT’15: Translation Task
Benjamin Marie, Alexandre Allauzen, Franck Burlot, Quoc-Khanh Do, Julia Ive, elena knyazeva, Matthieu Labeau, Thomas Lavergne, Kevin Löser, Nicolas Pécheux, François Yvon
  UdS-Sant: English–German Hybrid Machine Translation System
Santanu Pal, Sudip Naskar, Josef van Genabith
  The RWTH Aachen German-English Machine Translation System for WMT 2015
Jan-Thorsten Peter, Farzad Toutounchi, Joern Wuebker, Hermann Ney
  Exact Decoding with Multi Bottom-Up Tree Transducers
Daniel Quernheim
  Sheffield Systems for the Finnish-English WMT Translation Task
David Steele, Karin Sim Smith, Lucia Specia
  Morphological Segmentation and OPUS for Finnish-English Machine Translation
Jörg Tiedemann, Filip Ginter, Jenna Kanerva
  Abu-MaTran at WMT 2015 Translation Task: Morphological Segmentation and Web Crawling
Raphael Rubino, Tommi Pirinen, Miquel Esplà-Gomis, Nikola Ljubešić, Sergio Ortiz Rojas, Vassilis Papavassiliou, Prokopis Prokopidis, Antonio Toral
  The University of Illinois submission to the WMT 2015 Shared Translation Task
Lane Schwartz, Bill Bryce, Chase Geigle, Sean Massung, Yisi Liu, Haoruo Peng, Vignesh Raja, Subhro Roy, Shyam Upadhyay
  Edinburgh’s Syntax-Based Systems at WMT 2015
Philip Williams, Rico Sennrich, Maria Nadejde, Matthias Huck, Philipp Koehn
11:00 – 12:30 Session 3B: Poster Session - Shared Task: Automatic Post-Editing
  The FBK Participation in the WMT15 Automatic Post-editing Shared Task
Rajen Chatterjee, Marco Turchi, Matteo Negri
  USAAR-SAPE: An English–Spanish Statistical Automatic Post-Editing System
Santanu Pal, Mihaela Vela, Sudip Kumar Naskar, Josef van Genabith
  Why Predicting Post-Edition is so Hard? Failure Analysis of LIMSI Submission to the APE Shared Task
Guillaume Wisniewski, Nicolas Pécheux, François Yvon
12:30 – 14:00 🍴
14:00 – 15:30 Session 4: Invited Talk
A Practical Guide to Real-Time Neural Translation
Jacob Devlin
15:30 – 16:00 ☕️
16:00 – 17:00 Session 5: Syntax-Based Translation and Rescoring
16:00 – 16:20 Hierarchical Machine Translation With Discontinuous Phrases (slides)
Miriam Kaeshammer
16:20 – 16:40 Discontinuous Statistical Machine Translation with Target-Side Dependency Syntax (slides)
Nina Seemann, Andreas Maletti
16:40 – 17:00 ListNet-based MT Rescoring (slides)
Jan Niehues, Quoc-Khanh Do, Alexandre Allauzen, Alex Waibel

Day 2

   
09:00 – 09:50 Session 6: Shared Tasks
09:00 – 09:20 Overview of the Quality Estimation Task (slides)
Multiple Speakers
Results of the WMT15 Metrics Shared Task
Miloš Stanojević, Amir Kamran, Philipp Koehn, Ondřej Bojar
Results of the WMT15 Tuning Shared Task
Miloš Stanojević, Amir Kamran, Ondřej Bojar
09:50 – 10:30 Session 7: Translation Modeling
09:50 – 10:10 Extended Translation Models in Phrase-based Decoding (slides)
Andreas Guta, Joern Wuebker, Miguel Graca, Yunsu Kim, Hermann Ney
10:10 – 10:30 Investigations on Phrase-based Decoding with Recurrent Neural Network Language and Translation Models (slides)
Tamer Alkhouli, Felix Rietig, Hermann Ney
10:30 – 11:00 ☕️
11:00 – 12:30 Session 8A: Poster Session - Shared Task: Quality Estimation
Referential Translation Machines for Predicting Translation Quality and Related Statistics
Ergun Bicici, Qun Liu, Andy Way
  UAlacant word-level machine translation quality estimation system at WMT 2015
Miquel Esplà-Gomis, Felipe Sánchez-Martínez, Mikel Forcada
  QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation
Julia Kreutzer, Shigehiko Schamoni, Stefan Riezler
  LORIA System for the WMT15 Quality Estimation Shared Task
David Langlois
  Data enhancement and selection strategies for the word-level Quality Estimation
Varvara Logacheva, Chris Hokamp, Lucia Specia
  USHEF and USAAR-USHEF participation in the WMT15 QE shared task
Carolina Scarton, Liling Tan, Lucia Specia
  SHEF-NN: Translation Quality Estimation with Neural Networks
Kashif Shah, Varvara Logacheva, Gustavo Paetzold, Frédéric Blain, Daniel Beck, Fethi Bougares, Lucia Specia
  Strategy-Based Technology for Estimating MT Quality
Liugang Shang, Dongfeng Cai, Duo Ji
  UGENT-LT3 SCATE System for Machine Translation Quality Estimation
Arda Tezcan, Veronique Hoste, Bart Desmet, Lieve Macken
11:00 – 12:30 Session 8B: Poster Session - Shared Task: Metrics
  Multi-level Evaluation for Machine Translation
Boxing Chen, Hongyu Guo, Roland Kuhn
  VERTa: a Linguistically-motivated Metric at the WMT15 Metrics Task
Elisabet Comelles, Jordi Atserias
  UPF-Cobalt Submission to WMT15 Metrics Task
Marina Fomicheva, Núria Bel, Iria da Cunha, Anton Malinovskiy
  Machine Translation Evaluation using Recurrent Neural Networks
Rohit Gupta, Constantin Orasan, Josef van Genabith
  Alignment-based sense selection in METEOR and the RATATOUILLE recipe
Benjamin Marie, Marianna Apidianaki
  chrF: character n-gram F-score for automatic MT evaluation
Maja Popović
  BEER 1.1: ILLC UvA submission to metrics and tuning task
Miloš Stanojević, Khalil Sima’an
  Predicting Machine Translation Adequacy with Document Embeddings
Mihaela Vela, Liling Tan
  LeBLEU: N-gram-based Translation Evaluation Score for Morphologically Complex Languages
Sami Virpioja, Stig-Arne Grönroos
  CASICT-DCU Participation in WMT2015 Metrics Task
Hui Yu, Qingsong Ma, Xiaofeng Wu, Qun Liu
11:00 – 12:30 Session 8C: Poster Session - Shared Task: Tuning
  Drem: The AFRL Submission to the WMT15 Tuning Task
Grant Erdmann, Jeremy Gwinnup
  MT Tuning on RED: A Dependency-Based Evaluation Metric
Liangyou Li, Hui Yu, Qun Liu
  Improving evaluation and optimization of MT systems against MEANT
Chi-kiu Lo, Philipp Dowling, Dekai Wu
12:30 – 14:00 🍴
14:00 – 15:20 Session 9: Evaluation and System Combination
  An Investigation of Machine Translation Evaluation Metrics in Cross-lingual Question Answering
Kyoshiro Sugiyama, Masahiro Mizukami, Graham Neubig, Koichiro Yoshino, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura
  Dependency Analysis of Scrambled References for Better Evaluation of Japanese Translation (slides)
Hideki Isozaki, Natsume Kouchi
  How do Humans Evaluate Machine Translation
Francisco Guzmán, Ahmed Abdelali, Irina Temnikova, Hassan Sajjad, Stephan Vogel
  Local System Voting Feature for Machine Translation System Combination (slides)
Markus Freitag, Jan-Thorsten Peter, Stephan Peitz, Minwei Feng, Hermann Ney
15:20 – 16:00 ☕️
16:00 – 17:00 Session 10: Closing and Open Discussion

Results

Full results of the shared tasks: Findings of the 2015 Workshop on Statistical Machine Translation

News translation

The results were determined with a relative ranking through the TrueSkill model.

→ English

Language pair System Score
Czech → ONLINE-B 0.619
French → ONLINE-B 0.498
German → ONLINE-B 0.567
Russian → ONLINE-G 0.494
Finnish → ONLINE-B 0.675

English →

Language pair System Score
→ Czech CU-CHIMERA 0.686
→ French LIMSI-CNRS 0.540
→ German UEDIN-SYNTAX 0.359
→ Russian PROMT-RULE 1.015
→ Finnish ONLINE-B 1.069

Want to learn more about WMT15?


Edit this article →

Machine Translate is created and edited by contributors like you!

Learn more about contributing →

Licensed under CC-BY-SA-4.0.

Cite this article →