WMT11

Workshop on Machine Translation


Location

  • Edinburgh, United Kingdom

Links

Shared Tasks

Schedule

Day 1

   
9:00 Session 1: Shared Tasks and Evaluation
Chair: Chris Callison-Burch
A Grain of Salt for the WMT Manual Evaluation
Ondrej Bojar, Miloš Ercegovčević, Martin Popel, Omar Zaidan
9:25 A Lightweight Evaluation Framework for Machine Translation Reordering
David Talbot, Hideto Kazawa, Hiroshi Ichikawa, Jason Katz-Brown, Masakazu Seno, Franz Och
9:50 Findings of the 2011 Workshop on Statistical Machine Translation
Chris Callison-Burch, Philipp Koehn, Christof Monz, Omar Zaidan
10:30 ☕️
11:00 Session 2: Shared Metrics and System Combination Tasks
Poster Session Metrics
Evaluate with Confidence Estimation: Machine ranking of translation outputs using grammatical features
Eleftherios Avramidis, Maja Popović, David Vilar, Aljoscha Burchardt
AMBER: A Modified BLEU, Enhanced Ranking Metric
Boxing Chen, Roland Kuhn
TESLA at WMT 2011: Translation Evaluation and Tunable Metric
Daniel Dahlmeier, Chang Liu, Hwee Tou Ng
Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems
Michael Denkowski, Alon Lavie
Approximating a Deep-Syntactic Metric for MT Evaluation and Tuning
Matouš Macháček, Ondrej Bojar
Evaluation without references: IBM1 scores as evaluation metrics
Maja Popović, David Vilar, Eleftherios Avramidis, Aljoscha Burchardt
Morphemes and POS tags for n-gram based evaluation metrics
Maja Popović
E-rating Machine Translation
Kristen Parton, Joel Tetreault, Nitin Madnani, Martin Chodorow
TINE: A Metric to Assess MT Adequacy
Miguel Rios, Wilker Aziz, Lucia Specia
Regression and Ranking based Optimisation for Sentence Level MT Evaluation
Xingyi Song, Trevor Cohn
MAISE: A Flexible, Configurable, Extensible Open Source Package for Mass AI System Evaluation
Omar Zaidan
Poster Session System Combination
MANY improvements for WMT11
Loïc Barrault
The UPV-PRHLT combination system for WMT 2011
Jesús González-Rubio, Francisco Casacuberta
CMU System Combination in WMT 2011
Kenneth Heafield, Alon Lavie
The RWTH System Combination System for WMT 2011
Gregor Leusch, Markus Freitag, Hermann Ney
Expected BLEU Training for Graphs: BBN System Description for WMT11 System Combination Task
Antti-Veikko Rosti, Bing Zhang, Spyros Matsoukas, Richard Schwartz
The UZH System Combination System for WMT 2011
Rico Sennrich
Description of the JHU System Combination Scheme for WMT 2011
Daguang Xu, Yuan Cao, Damianos Karakos
12:40 🍴
14:00 Session 3: Language Models and Document Alignment
Chair: David Talbot
Multiple-stream Language Models for Statistical Machine Translation
Abby Levenberg, Miles Osborne, David Matthews
14:25 KenLM: Faster and Smaller Language Model Queries
Kenneth Heafield
14:50 Wider Context by Using Bilingual Language Models in Machine Translation
Jan Niehues, Teresa Herrmann, Stephan Vogel, Alex Waibel
15:15 A Minimally Supervised Approach for Detecting and Ranking Document Translation Pairs
Kriste Krstovski, David A. Smith
15:40 ☕️
16:10 Session 4: Syntax and Semantics
Chair: Michel Galley
Agreement Constraints for Statistical Machine Translation into German
Philip Williams, Philipp Koehn
16:35 Fuzzy Syntactic Reordering for Phrase-based Statistical Machine Translation
Jacob Andreas, Nizar Habash, Owen Rambow
17:00 Filtering Antonymous, Trend-Contrasting, and Polarity-Dissimilar Distributional Paraphrases for Improving Statistical Machine Translation
Yuval Marton, Ahmed El Kholy, Nizar Habash
17:25 Productive Generation of Compound Words in Statistical Machine Translation
Sara Stymne, Nicola Cancedda

Day 2

   
9:00 Session 5: Machine Learning and Adaptation
Chair: Phil Blunsom
SampleRank Training for Phrase-Based Machine Translation
Barry Haddow, Abhishek Arun, Philipp Koehn
9:25 Instance Selection for Machine Translation using Feature Decay Algorithms
Ergun Bicici, Deniz Yuret
9:50 Investigations on Translation Model Adaptation Using Monolingual Data
Patrik Lambert, Holger Schwenk, Christophe Servan, Sadaf Abdul-Rauf
10:15 Topic Adaptation for Lecture Translation through Bilingual Latent Semantic Models
Nick Ruiz, Marcello Federico
10:40 ☕️
11:00 Session 6: Shared Translation Task
Poster Presentations
Personal Translator at WMT2011
Vera Aleksic, Gregor Thurmair
LIMSI @ WMT11
Alexandre Allauzen, Hélène Bonneau-Maynard, Hai-Son Le, Aurélien Max, Guillaume Wisniewski, François Yvon, Gilles Adda, Josep Maria Crego, Adrien Lardilleux, Thomas Lavergne, Artem Sokolov
Shallow Semantic Trees for SMT
Wilker Aziz, Miguel Rios, Lucia Specia
RegMT System for Machine Translation, System Combination, and Evaluation
Ergun Bicici, Deniz Yuret
Improving Translation Model by Monolingual Data
Ondrej Bojar, Aleš Tamchyna
The CMU-ARK German-English Translation System
Chris Dyer, Kevin Gimpel, Jonathan H. Clark, Noah A. Smith
Noisy SMS Machine Translation in Low-Density Languages
Vladimir Eidelman, Kristy Hollingshead, Philip Resnik
Stochastic Parse Tree Selection for an Existing RBMT System
Christian Federmann, Sabine Hunsicker
Joint WMT Submission of the QUAERO Project
Markus Freitag, Gregor Leusch, Joern Wuebker, Stephan Peitz, Hermann Ney, Teresa Herrmann, Jan Niehues, Alex Waibel, Alexandre Allauzen, Gilles Adda, Josep Maria Crego, Bianka Buschbeck, Tonio Wandmacher, Jean Senellart
CMU Syntax-Based Machine Translation at WMT 2011
Greg Hanneman, Alon Lavie
The Uppsala-FBK systems at WMT 2011
Christian Hardmeier, Jörg Tiedemann, Markus Saers, Marcello Federico, Mathur Prashant
The Karlsruhe Institute of Technology Translation Systems for the WMT 2011
Teresa Herrmann, Mohammed Mediani, Jan Niehues, Alex Waibel
CMU Haitian Creole-English Translation System for WMT 2011
Sanjika Hewavitharana, Nguyen Bach, Qin Gao, Vamshi Ambati, Stephan Vogel
Experiments with word alignment, normalization and clause reordering for SMT between English and German
Maria Holmqvist, Sara Stymne, Lars Ahrenberg
The Value of Monolingual Crowdsourcing in a Real-World Translation Scenario: Simulation using Haitian Creole Emergency SMS Messages
Chang Hu, Philip Resnik, Yakov Kronrod, Vladimir Eidelman, Olivia Buzek, Benjamin B. Bederson
The RWTH Aachen Machine Translation System for WMT 2011
Matthias Huck, Joern Wuebker, Christoph Schmidt, Markus Freitag, Stephan Peitz, Daniel Stein, Arnaud Dagnelies, Saab Mansour, Gregor Leusch, Hermann Ney
ILLC-UvA translation system for EMNLP-WMT 2011
Maxim Khalilov, Khalil Sima’an
UPM system for the translation task
Verónica López-Ludeña, Rubén San-Segundo
Two-step translation with grammatical post-processing
David Mareček, Rudolf Rosa, Petra Galuščáková, Ondrej Bojar
Influence of Parser Choice on Dependency-Based MT
Martin Popel, David Mareček, Nathan Green, Zdenek Zabokrtsky
The LIGA (LIG/LIA) Machine Translation System for WMT 2011
Marion Potet, Raphaël Rubino, Benjamin Lecouteux, Stéphane Huet, Laurent Besacier, Hervé Blanchon, Fabrice Lefèvre
Factored Translation with Unsupervised Word Clusters
Christian Rishj, Anders Sgaard
The BM-I2R Haitian-Créole-to-English translation system description for the WMT 2011 evaluation campaign
Marta R. Costa-jussà, Rafael E. Banchs
The Universitat d'Alacant hybrid machine translation system for WMT 2011
Víctor M. Sánchez-Cartagena, Felipe Sánchez-Martínez, Juan Antonio Pérez-Ortiz
LIUM’s SMT Machine Translation Systems for WMT 2011
Holger Schwenk, Patrik Lambert, Loïc Barrault, Christophe Servan, Sadaf Abdul-Rauf, Haithem Afli, Kashif Shah
Spell Checking Techniques for Replacement of Unknown Words and Data Cleaning for Haitian Creole SMS Translation
Sara Stymne
Joshua 3.0: Syntax-based Machine Translation with the Thrax Grammar Extractor
Jonathan Weese, Juri Ganitkevitch, Chris Callison-Burch, Matt Post, Adam Lopez
DFKI Hybrid Machine Translation System for WMT 2011 - On the Integration of SMT and RBMT
Jia Xu, Hans Uszkoreit, Casey Kennington, David Vilar, Xiaojun Zhang
CEU-UPV English-Spanish system for WMT11
Francisco Zamora-Martinez, Maria Jose Castro-Bleda
Hierarchical Phrase-Based MT at the Charles University for the WMT 2011 Shared Task
Daniel Zeman
12:40 🍴
14:00 Invited Talk
Crisis MT: Developing A Cookbook for MT in Crisis Situations
William Lewis, Robert Munro, Stephan Vogel
15:40 ☕️
16:10 Session 7: Translation Models
Chair: Chris Dyer
Generative Models of Monolingual and Bilingual Gappy Patterns
Kevin Gimpel, Noah A. Smith
16:35 Extraction Programs: A Unified Approach to Translation Rule Extraction
Mark Hopkins, Greg Langmead, Tai Vo
17:00 Bayesian Extraction of Minimal SCFG Rules for Hierarchical Phrase-based Translation
Baskaran Sankaran, Gholamreza Haffari, Anoop Sarkar
17:25 From n-gram-based to CRF-based Translation Models
Thomas Lavergne, Alexandre Allauzen, Josep Maria Crego, François Yvon

Results

General Task

Full results of the shared task: Findings of the 2011 Workshop on Statistical Machine Translation

Tunable metrics task

The tunable metrics shared task was invitation-only.

News translation

The results were determined with a relative ranking, the ≥ others (“greater than or equal to others”) score.

It measures how often a system was judged to be better than or equal to any other system.

→ English

Language pair System ≥ others
Czech → UEDIN
ONLINE-B
0.69
0.68
Spanish → ONLINE-B
ONLINE-A
KOC
SYSTRAN
ALACANT
0.72
0.72
0.67
0.66
0.66
French → ONLINE-A
LIMSI
ONLINE-B
KIT
CMU-DENKOWSKI
RBMT-1
0.66
0.66
0.66
0.64
0.62
0.58
German → ONLINE-B
CMU-DYER
ONLINE-A
RWTH-WUEBKER
0.72
0.66
0.66
0.59

English →

Language pair System ≥ others
→ Czech ONLINE-B
CU-MARECEK
UEDIN
CU-POPEL
0.65
0.63
0.59
0.58
→ Spanish ONLINE-B
ONLINE-A
RBMT-3
PROMT
CEU-UPV
UEDIN
UPPSALA
0.74
0.72
0.71
0.70
0.65
0.64
0.61
→ French LIMSI
ONLINE-B
KIT
0.73
0.70
0.69
→ German RBMT-3
ONLINE-B
RBMT-1
DFKI-FEDERMANN
RBMT-4
RBMT-2
ONLINE-A
LIMSI
KIT
COPENHAGEN
0.73
0.73
0.70
0.68
0.66
0.66
0.65
0.65
0.64
0.56

Want to learn more about WMT11?


Edit this article →

Machine Translate is created and edited by contributors like you!

Learn more about contributing →

Licensed under CC-BY-SA-4.0.

Cite this article →