WMT21

Conference on Machine Translation


The Sixth Conference on Machine Translation (WMT21) took place from 10 November to 11 November, 2021, at EMNLP 2021 in Punta Cana in the Dominican Republic and online. It was organised by WMT.

statmt.org/wmt21/program.html

Table of contents
  1. Tasks
    1. Shared tasks
    2. Evaluation tasks
    3. Other tasks
  2. Schedule
    1. Day 1
    2. Day 2
  3. Results
    1. News translation
      1. → English
      2. English →
      3. French ↔︎ German
      4. Hindi ↔︎ Bengali
      5. Zulu ↔︎ Xhosa

Tasks

Shared tasks

Evaluation tasks

Other tasks

Schedule

Day 1

   
09:00 – 09:10 Opening Remarks
09:10 – 10:15 Session 1: Shared Task Overview Papers I
09:10 – 10:15 Findings of the 2021 Conference on Machine Translation (WMT21)
Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondřej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta R. Costa-jussà, Cristina España-Bonet, Angela Fan, Christian Federmann, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christopher Homan, Matthias Huck, Kwabena Amponsah-Kaakyire, Jungo Kasai, Daniel Khashabi, Kevin Knight, Tom Kocmi, Philipp Koehn, Nicholas Lourie, Christof Monz, Makoto Morishita, Masaaki Nagata, Ajay Nagesh, Toshiaki Nakazawa, Matteo Negri, Santanu Pal, Allahsera Auguste Tapo, Marco Turchi, Valentin Vydrin, Marcos Zampieri
09:00 – 09:25 News Translation Task
09:25 – 09:35 Similar Languages Translation Task
09:35 – 09:45 Automatic Postediting Task
09:45 – 09:55 Triangular Translation Task
09:55 – 10:05 Indo-European Multilingual Translation Task
10:05 – 10:15 Findings on the WMT 2021 Shared Task on Large-Scale Multilingual Machine Translation
Guillaume Wenzek, Vishrav Chaudhary, Angela Fan, Sahir Gomez, Naman Goyal, Somya Jain, Douwe Kiela, Tristan Thrush, Francisco Guzmán
10:15 – 10:30 ☕️
10:30 – 12:00 News Translation Task
10:30 – 12:00 GTCOM Neural Machine Translation Systems for WMT21
Chao Bei, Hao Zong
10:30 – 12:00 The University of Edinburgh’s English-German and English-Hausa Submissions to the WMT21 News Translation Task
Pinzhen Chen, Jindřich Helcl, Ulrich Germann, Laurie Burchell, Nikolay Bogoychev, Antonio Valerio Miceli Barone, Jonas Waldendorf, Alexandra Birch, Kenneth Heafield
10:30 – 12:00 Tune in: The AFRL WMT21 News-Translation Systems
Grant Erdmann, Jeremy Gwinnup, Tim Anderson
10:30 – 12:00 The TALP-UPC Participation in WMT21 News Translation Task: an mBART-based NMT Approach
Carlos Escolano, Ioannis Tsiamas, Christine Basta, Javier Ferrando, Marta R. Costa-jussà, José A. R. Fonollosa
10:30 – 12:00 CUNI Systems in WMT21: Revisiting Backtranslation Techniques for English-Czech NMT
Petr Gebauer, Ondřej Bojar, Vojtěch Švandelík, Martin Popel
10:30 – 12:00 Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language Pairs
Amr Hendy, Esraa A. Gad, Mohamed Abdelghaffar, Jailan S. ElMosalami, Mohamed Afify, Ahmed Y. Tawfik, Hany Hassan Awadalla
10:30 – 12:00 Miðeind’s WMT 2021 Submission
Haukur Jónsson, Haukur Barri Símonarson, Vésteinn Snæbjarnarson, Pétur Orri Ragnarson, Vilhjálmur Þorsteinsson
10:30 – 12:00 Allegro.eu Submission to WMT21 News Translation Task
Mikołaj Koszowski, Karol Grzegorczyk, Tsimur Hadeliya
10:30 – 12:00 Illinois Japanese $$ English News Translation for WMT 2021
Giang Le, Shinka Mori, Lane Schwartz
10:30 – 12:00 MiSS@WMT21: Contrastive Learning-reinforced Domain Adaptation in Neural Machine Translation
Zuchao Li, Masao Utiyama, Eiichiro Sumita, Hai Zhao
10:30 – 12:00 The Fujitsu DMATH Submissions for WMT21 News Translation and Biomedical Translation Tasks
Ander Martinez
10:30 – 12:00 Adam Mickiewicz University’s English-Hausa Submissions to the WMT 2021 News Translation Task
Artur Nowakowski, Tomasz Dwojak
10:30 – 12:00 eTranslation’s Submissions to the WMT 2021 News Translation Task
Csaba Oravecz, Katina Bontcheva, David Kolovratník, Bhavani Bhaskar, Michael Jellinghaus, Andreas Eisele
10:30 – 12:00 The University of Edinburgh’s Bengali-Hindi Submissions to the WMT21 News Translation Task
Proyag Pal, Alham Fikri Aji, Pinzhen Chen, Sukanta Sen
10:30 – 12:00 The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21
Lihua Qian, Yi Zhou, Zaixiang Zheng, Yaoming ZHU, Zehui Lin, Jiangtao Feng, Shanbo Cheng, Lei Li, Mingxuan Wang, Hao Zhou
10:30 – 12:00 NVIDIA NeMo’s Neural Machine Translation Systems for English-German and English-Russian News and Biomedical Tasks at WMT21
Sandeep Subramanian, Oleksii Hrinchuk, Virginia Adams, Oleksii Kuchaiev
10:30 – 12:00 Facebook AI’s WMT21 News Translation Task Submission
Chau Tran, Shruti Bhosale, James Cross, Philipp Koehn, Sergey Edunov, Angela Fan
10:30 – 12:00 Tencent Translation System for the WMT21 News Translation Task
Longyue Wang, Mu Li, Fangxu Liu, Shuming Shi, Zhaopeng Tu, Xing Wang, Shuangzhi Wu, Jiali Zeng, Wen Zhang
10:30 – 12:00 HW-TSC’s Participation in the WMT 2021 News Translation Shared Task
Daimeng Wei, Zongyao Li, Zhanglin Wu, Zhengzhe Yu, Xiaoyu Chen, Hengchao Shang, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang, Ying Qin
10:30 – 12:00 LISN @ WMT 2021
Jitao Xu, Minh Quang Pham, Sadaf Abdul Rauf, François Yvon
10:30 – 12:00 WeChat Neural Machine Translation Systems for WMT21
Xianfeng Zeng, Yijin Liu, Ernan Li, Qiu Ran, Fandong Meng, Peng Li, Jinan Xu, Jie Zhou
10:30 – 12:00 Small Model and In-Domain Data Are All You Need
Hui Zeng
10:30 – 12:00 The Mininglamp Machine Translation System for WMT21
Shiyu Zhao, Xiaopu Li, Minghui Wu, Jie Hao
10:30 – 12:00 The NiuTrans Machine Translation Systems for WMT21
Shuhan Zhou, Tao Zhou, Binghao Wei, Yingfeng Luo, Yongyu Mu, Zefan Zhou, Chenglong Wang, Xuanjun Zhou, Chuanhao Lv, Yi Jing, Laohu Wang, Jingnan Zhang, Canan Huang, Zhongxiang Yan, Chi Hu, Bei Li, Tong Xiao, Jingbo Zhu
10:30 – 12:00 Similar Languages Translation Task
10:30 – 12:00 Improving Similar Language Translation With Transfer Learning
Ife Adebara, Muhammad Abdul-Mageed
10:30 – 12:00 T4T Solution: WMT21 Similar Language Task for the Spanish-Catalan and Spanish-Portuguese Language Pair
Miguel Canals, Marc Raventós Tato
10:30 – 12:00 Neural Machine Translation for Tamil–Telugu Pair
Sahinur Rahman Laskar, Bishwaraj Paul, Prottay Kumar Adhikary, Partha Pakray, Sivaji Bandyopadhyay
10:30 – 12:00 Low Resource Similar Language Neural Machine Translation for Tamil-Telugu
Vandan Mujadia, Dipti Sharma
10:30 – 12:00 Similar Language Translation for Catalan, Portuguese and Spanish Using Marian NMT
Reinhard Rapp
10:30 – 12:00 NITK-UoH: Tamil-Telugu Machine Translation Systems for the WMT21 Similar Language Translation Task
Richard Saldanha, Ananthanarayana V. S, Anand Kumar M, Parameswari Krishnamurthy
10:30 – 12:00 A3-108 Machine Translation System for Similar Language Translation Shared Task 2021
Saumitra Yadav, Manish Shrivastava
10:30 – 12:00 Automatic Post-Editing Task
10:30 – 12:00 Netmarble AI Center’s WMT21 Automatic Post-Editing Shared Task Submission
Shinhyeok Oh, Sion Jang, Hu Xu, Shounan An, Insoo Oh
10:30 – 12:00 Adapting Neural Machine Translation for Automatic Post-Editing
Abhishek Sharma, Prabhakar Gupta, Anil Nelakanti
10:30 – 12:00 Triangular Translation Task
10:30 – 12:00 ISTIC’s Triangular Machine Translation System for WMT2021
Hangcheng Guo, Wenbin Liu, Yanqing He, Tian Lan, Hongjiao Xu, Zhenfeng Wu, You Pan
10:30 – 12:00 HW-TSC’s Participation in the WMT 2021 Triangular MT Shared Task
Zongyao Li, Daimeng Wei, Hengchao Shang, Xiaoyu Chen, Zhanglin Wu, Zhengzhe Yu, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang, Ying Qin
10:30 – 12:00 DUTNLP Machine Translation System for WMT21 Triangular Translation Task
Huan Liu, Junpeng Liu, Kaiyu Huang, Degen Huang
10:30 – 12:00 Pivot Based Transfer Learning for Neural Machine Translation: CFILT IITB @ WMT 2021 Triangular MT
Shivam Mhaskar, Pushpak Bhattacharyya
10:30 – 12:00 Papago’s Submissions to the WMT21 Triangular Translation Task
Jeonghyeok Park, Hyunjoong Kim, Hyunchang Cho
10:30 – 12:00 Indo-European Multilingual Translation Task
10:30 – 12:00 Machine Translation of Low-Resource Indo-European Languages
Wei-Rui Chen, Muhammad Abdul-Mageed
10:30 – 12:00 CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task
Josef Jon, Michal Novák, João Paulo Aires, Dusan Varis, Ondřej Bojar
10:30 – 12:00 Transfer Learning with Shallow Decoders: BSC at WMT2021’s Multilingual Low-Resource Translation for Indo-European Languages Shared Task
Ksenia Kharitonova, Ona de Gibert Bonet, Jordi Armengol-Estapé, Mar Rodriguez i Alvarez, Maite Melero
10:30 – 12:00 EdinSaar@WMT21: North-Germanic Low-Resource Multilingual NMT
Svetlana Tchistiakova, Jesujoba Alabi, Koel Dutta Chowdhury, Sourav Dutta, Dana Ruiter
10:30 – 12:00 TenTrans Multilingual Low-Resource Translation System for WMT21 Indo-European Languages Task
Han Yang, Bojie Hu, Wanying Xie, ambyera han, Pan Liu, Jinan Xu, Qi Ju
10:30 – 12:00 Large-Scale Multilingual Translation Task
10:30 – 12:00 The University of Maryland, College Park Submission to Large-Scale Multilingual Shared Task at WMT 2021
Saptarashmi Bandyopadhyay, Tasnim Kabir, Zizhen Lian, Marine Carpuat
10:30 – 12:00 To Optimize, or Not to Optimize, That Is the Question: TelU-KU Models for WMT21 Large-Scale Multilingual Machine Translation
Sari Dewi Budiwati, Tirana Fatyanosa, Mahendra Data, Dedy Rahman Wijaya, Patrick Adolf Telnoni, Arie Ardiyanti Suryani, Agus Pratondo, Masayoshi Aritsugi
10:30 – 12:00 MMTAfrica: Multilingual Machine Translation for African Languages
Chris Chinenye Emezue, Bonaventure F. P. Dossou
10:30 – 12:00 The LMU Munich System for the WMT 2021 Large-Scale Multilingual Machine Translation Shared Task
Wen Lai, Jindřich Libovický, Alexander Fraser
10:30 – 12:00 Back-translation for Large-Scale Multilingual Machine Translation
Baohao Liao, Shahram Khadivi, Sanjika Hewavitharana
10:30 – 12:00 Maastricht University’s Large-Scale Multilingual Machine Translation System for WMT 2021
Danni Liu, Jan Niehues
10:30 – 12:00 Data Processing Matters: SRPH-Konvergen AI’s Machine Translation System for WMT’21
Lintang Sutawika, Jan Christian Blaise Cruz
10:30 – 12:00 TenTrans Large-Scale Multilingual Machine Translation System for WMT21
Wanying Xie, Bojie Hu, Han Yang, Dong Yu, Qi Ju
10:30 – 12:00 Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task
Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan, Xia Song, Furu Wei
10:30 – 12:00 HW-TSC’s Participation in the WMT 2021 Large-Scale Multilingual Translation Task
Zhengzhe Yu, Daimeng Wei, Zongyao Li, Hengchao Shang, Xiaoyu Chen, Zhanglin Wu, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang, Ying Qin
12:00 – 13:00 🍴
13:00 – 14:15 Session 3: Panel Discussion on Evaluation
Nitika Mathur (Univ. Melbourne), Benjamin Marie (NICT), Ricardo Rei (Unbabel), Tom Kocmi (Microsoft), moderated by Markus Freitag (Google)
14:15 – 14:45 🕑
14:45 – 16:15 Session 4: Research Papers on Evaluation
14:45 – 16:15 On the Stability of System Rankings at WMT
Rebecca Knowles
14:45 – 16:15 To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation
Tom Kocmi, Christian Federmann, Roman Grundkiewicz, Marcin Junczys-Dowmunt, Hitokazu Matsushita, Arul Menezes
14:45 – 16:15 Just Ask! Evaluating Machine Translation by Asking and Answering Questions
Mateusz Krubiński, Erfan Ghadery, Marie-Francine Moens, Pavel Pecina
14:45 – 16:15 A Fine-Grained Analysis of BERTScore
Michael Hanna, Ondřej Bojar
14:45 – 16:15 Evaluating Multiway Multilingual NMT in the Turkic Languages
Jamshidbek Mirzakhalov, Anoop Babu, Aigiz Kunafin, Ahsan Wahab, Bekhzodbek Moydinboyev, Sardana Ivanova, Mokhiyakhon Uzokova, Shaxnoza Pulatova, Duygu Ataman, Julia Kreutzer, Francis Tyers, Orhan Firat, John Licato, Sriram Chellappan
14:45 – 16:15 Extending Challenge Sets to Uncover Gender Bias in Machine Translation: Impact of Stereotypical Verbs and Adjectives
Jonas-Dario Troles, Ute Schmid
16:15 - 16:45 ☕️
16:45 – 18:15 Session 5: Research Papers on Data
16:45 – 18:15 Continual Learning in Multilingual NMT via Language-Specific Embeddings
Alexandre Berard
16:45 – 18:15 DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues
Sheila Castilho, João Lucas Cavalheiro Camargo, Miguel Menezes, Andy Way
16:45 – 18:15 Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters
Asa Cooper Stickland, Alexandre Berard, Vassilina Nikoulina
16:45 – 18:15 Translation Transformers Rediscover Inherent Data Domains
Maksym Del, Elizaveta Korotkova, Mark Fishel
16:45 – 18:15 Improving Machine Translation of Rare and Unseen Word Senses
Viktor Hangya, Qianchu Liu, Dario Stojanovski, Alexander Fraser, Anna Korhonen
16:45 – 18:15 Pushing the Right Buttons: Adversarial Evaluation of Quality Estimation
Diptesh Kanojia, Marina Fomicheva, Tharindu Ranasinghe, Frédéric Blain, Constantin Orăsan, Lucia Specia

Day 2

   
09:00 – 10:15 Session 6: Shared Task Overview Papers I
09:00 – 09:12 Findings of the WMT 2021 Shared Task on Efficient Translation
Kenneth Heafield, Qianqian Zhu, Roman Grundkiewicz
09:12 – 09:24 Findings of the WMT Shared Task on Machine Translation Using Terminologies
Md Mahfuz Ibn Alam, Ivana Kvapilíková, Antonios Anastasopoulos, Laurent Besacier, Georgiana Dinu, Marcello Federico, Matthias Gallé, Kweonwoo Jung, Philipp Koehn, Vassilina Nikoulina
09:24 – 09:36 Findings of the WMT 2021 Biomedical Translation Shared Task: Summaries of Animal Experiments as New Test Set
Lana Yeganova, Dina Wiemann, Mariana Neves, Federica Vezzani, Amy Siu, Inigo Jauregi Unanue, Maite Oronoz, Nancy Mah, Aurélie Névéol, David Martinez, Rachel Bawden, Giorgio Maria Di Nunzio, Roland Roller, Philippe Thomas, Cristian Grozea, Olatz Perez-de-Viñaspre, Maika Vicente Navarro, Antonio Jimeno Yepes
09:36 – 09:48 Findings of the WMT 2021 Shared Task on Quality Estimation
Lucia Specia, Frédéric Blain, Marina Fomicheva, Chrysoula Zerva, Zhenhao Li, Vishrav Chaudhary, André F. T. Martins
09:48 – 10:00 Findings of the WMT 2021 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT
Jindřich Libovický, Alexander Fraser
10:00 – 10:15 Results of the WMT21 Metric Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain
Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie, Ondřej Bojar
10:15 – 10:30 ☕️
10:30 – 12:00 Efficient Translation Task
10:30 – 12:00 Efficient Machine Translation with Model Pruning and Quantization
Maximiliana Behnke, Nikolay Bogoychev, Alham Fikri Aji, Kenneth Heafield, Graeme Nail, Qianqian Zhu, Svetlana Tchistiakova, Jelmer van der Linde, Pinzhen Chen, Sidharth Kashyap, Roman Grundkiewicz
10:30 – 12:00 HW-TSC’s Participation in the WMT 2021 Efficiency Shared Task
Hengchao Shang, Ting Hu, Daimeng Wei, Zongyao Li, Jianfei Feng, ZhengZhe Yu, Jiaxin Guo, Shaojun Li, Lizhi Lei, ShiMin Tao, Hao Yang, Jun Yao, Ying Qin
10:30 – 12:00 The NiuTrans System for the WMT 2021 Efficiency Task
Chenglong Wang, Chi Hu, Yongyu Mu, Zhongxiang Yan, Siming Wu, Yimin Hu, Hang Cao, Bei Li, Ye Lin, Tong Xiao, Jingbo Zhu
10:30 – 12:00 TenTrans High-Performance Inference Toolkit for WMT2021 Efficiency Task
Kaixin WU, Bojie Hu, Qi Ju
10:30 – 12:00 Terminology Translation Task
10:30 – 12:00 Lingua Custodia’s Participation at the WMT 2021 Machine Translation Using Terminologies Shared Task
Melissa Ailem, Jingshu Liu, Raheel Qader
10:30 – 12:00 Kakao Enterprise’s WMT21 Machine Translation Using Terminologies Task Submission
Yunju Bak, Jimin Sun, Jay Kim, Sungwon Lyu, Changmin Lee
10:30 – 12:00 The SPECTRANS System Description for the WMT21 Terminology Task
Nicolas Ballier, Dahn Cho, Bilal Faye, Zong-You Ke, Hanna Martikainen, Mojca Pecman, Guillaume Wisniewski, Jean-Baptiste Yunès, Lichao Zhu, Maria Zimina-Poirot
10:30 – 12:00 Dynamic Terminology Integration for COVID-19 and Other Emerging Domains
Toms Bergmanis, Mārcis Pinnis
10:30 – 12:00 CUNI Systems for WMT21: Terminology Translation Shared Task
Josef Jon, Michal Novák, João Paulo Aires, Dusan Varis, Ondřej Bojar
10:30 – 12:00 PROMT Systems for WMT21 Terminology Translation Task
Alexander Molchanov, Vladislav Kovalenko, Fedor Bykov
10:30 – 12:00 SYSTRAN @ WMT 2021: Terminology Task
Minh Quang Pham, Josep Crego, Antoine Senellart, Dan Berrebbi, Jean Senellart
10:30 – 12:00 TermMind: Alibaba’s WMT21 Machine Translation Using Terminologies Task Submission
Ke Wang, Shuqin Gu, Boxing Chen, Yu Zhao, Weihua Luo, Yuqi Zhang
10:30 – 12:00 Biomedical Translation Task
10:30 – 12:00 FJWU Participation for the WMT21 Biomedical Translation Task
Sumbal Naz, Sadaf Abdul Rauf, Sami Ul Haq
10:30 – 12:00 High Frequent In-domain Words Segmentation and Forward Translation for the WMT21 Biomedical Task
Bardia Rafieian, Marta Ruiz Costa Jussa
10:30 – 12:00 Huawei AARC’s Submissions to the WMT21 Biomedical Translation Task: Domain Adaption from a Practical Perspective
Weixuan Wang, Wei Peng, Xupeng Meng, Qun Liu
  Tencent AI Lab Machine Translation Systems for the WMT21 Biomedical Translation Task
Xing Wang, Zhaopeng Tu, Shuming Shi
10:30 – 12:00 HW-TSC’s Submissions to the WMT21 Biomedical Translation Task
Hao Yang, Zhanglin Wu, Zhengzhe Yu, Xiaoyu Chen, Daimeng Wei, Zongyao Li, Hengchao Shang, Minghan Wang, Jiaxin Guo, Lizhi Lei, chuanfei xu, Min Zhang, Ying Qin
10:30 - 12:00 LISN @ WMT 2021
Jitao Xu, Minh Quang Pham, Sadaf Abdul Rauf, François Yvon
10:30 – 12:00 Quality Estimation Task
10:30 – 12:00 RTM Super Learner Results at Quality Estimation Task
Ergun Biçici
10:30 – 12:00 HW-TSC’s Participation at WMT 2021 Quality Estimation Shared Task
Yimeng Chen, Chang Su, Yingtao Zhang, Yuxia Wang, Xiang Geng, Hao Yang, Shimin Tao, Guo Jiaxin, Wang Minghan, Min Zhang, Yujia Liu, Shujian Huang
10:30 – 12:00 Ensemble Fine-tuned mBERT for Translation Quality Estimation
Shaika Chowdhury, Naouel Baili, Brian Vannah
10:30 – 12:00 The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task
Shuoyang Ding, Marcin Junczys-Dowmunt, Matt Post, Christian Federmann, Philipp Koehn
10:30 – 12:00 TUDa at WMT21: Sentence-Level Direct Assessment with Adapters
Gregor Geigle, Jonas Stadtmüller, Wei Zhao, Jonas Pfeiffer, Steffen Eger
10:30 – 12:00 Quality Estimation Using Dual Encoders with Transfer Learning
Dam Heo, WonKee Lee, Baikjin Jung, Jong-Hyeok Lee
10:30 – 12:00 ICL’s Submission to the WMT21 Critical Error Detection Shared Task
Genze Jiang, Zhenhao Li, Lucia Specia
10:30 – 12:00 Papago’s Submission for the WMT21 Quality Estimation Shared Task
Seunghyun Lim, Hantae Kim, Hyunjoong Kim
10:30 – 12:00 NICT Kyoto Submission for the WMT’21 Quality Estimation Task: Multimetric Multilingual Pretraining for Critical Error Detection
Raphael Rubino, Atsushi Fujita, Benjamin Marie
10:30 – 12:00 QEMind: Alibaba’s Submission to the WMT21 Quality Estimation Shared Task
Jiayi Wang, Ke Wang, Boxing Chen, Yu Zhao, Weihua Luo, Yuqi Zhang
10:30 – 12:00 Direct Exploitation of Attention Weights for Translation Quality Estimation
Lisa Yankovskaya, Mark Fishel
10:30 – 12:00 IST-Unbabel 2021 Submission for the Quality Estimation Shared Task
Chrysoula Zerva, Daan van Stigt, Ricardo Rei, Ana C Farinha, Pedro Ramos, José G. C. de Souza, Taisiya Glushkova, Miguel Vera, Fabio Kepler, André F. T. Martins
10:30 – 12:00 Unsupervised and Very Low Resource Translation Task
10:30 – 12:00 The IICT-Yverdon System for the WMT 2021 Unsupervised MT and Very Low Resource Supervised MT Task
Àlex R. Atrio, Gabriel Luthier, Axel Fahy, Giorgos Vernikos, Andrei Popescu-Belis, Ljiljana Dolamic
10:30 – 12:00 Unsupervised Translation of German–Lower Sorbian: Exploring Training and Novel Transfer Methods on a Low-Resource Language
Lukas Edman, Ahmet Üstün, Antonio Toral, Gertjan van Noord
10:30 – 12:00 The LMU Munich Systems for the WMT21 Unsupervised and Very Low-Resource Translation Task
Jindřich Libovický, Alexander Fraser
10:30 – 12:00 Language Model Pretraining and Transfer Learning for Very Low Resource Languages
Jyotsana Khatri, Rudra Murthy, Pushpak Bhattacharyya
10:30 – 12:00 NRC-CNRC Systems for Upper Sorbian-German and Lower Sorbian-German Machine Translation 2021
Rebecca Knowles, Samuel Larkin
10:30 – 12:00 NoahNMT at WMT 2021: Dual Transfer for Very Low Resource Supervised Machine Translation
Meng Zhang, Minghao Wu, Pengfei Li, Liangyou Li, Qun Liu
10:30 – 12:00 Metrics Task
10:30 – 12:00 cushLEPOR: Customised hLEPOR Metric Using LABSE Distilled Knowledge Model to Improve Agreement with Human Judgements
Lifeng Han, Irina Sorokina, Gleb Erofeev, Serge Gladkoff
10:30 – 12:00 MTEQA at WMT21 Metrics Shared Task
Mateusz Krubiński, Erfan Ghadery, Marie-Francine Moens, Pavel Pecina
10:30 – 12:00 Are References Really Needed? Unbabel-IST 2021 Submission for the Metrics Shared Task
Ricardo Rei, Ana C Farinha, Chrysoula Zerva, Daan van Stigt, Craig Stewart, Pedro Ramos, Taisiya Glushkova, André F. T. Martins, Alon Lavie
10:30 – 12:00 Regressive Ensemble for Machine Translation Quality Evaluation
Michal Stefanik, Vít Novotný, Petr Sojka
10:30 – 12:00 Multilingual Machine Translation Evaluation Metrics Fine-tuned on Pseudo-Negative Examples for WMT 2021 Metrics Task
Kosuke Takahashi, Yoichi Ishibashi, Katsuhito Sudoh, Satoshi Nakamura
10:30 – 12:00 RoBLEURT Submission for WMT2021 Metrics Task
Yu Wan, Dayiheng Liu, Baosong Yang, Tianchi Bi, Haibo Zhang, Boxing Chen, Weihua Luo, Derek F. Wong, Lidia S. Chao
10:30 – 12:00 Test Suites
10:30 – 12:00 Linguistic Evaluation for the 2021 State-of-the-art Machine Translation Systems for German to English and English to German
Vivien Macketanz, Eleftherios Avramidis, Shushen Manakhimova, Sebastian Möller
12:00 – 13:00 🍴
13:00 – 14:15 Session 8: Research Papers on Training and Modelling
13:00 – 14:15 Pruning Neural Machine Translation for Speed Using Group Lasso
Maximiliana Behnke, Kenneth Heafield
13:00 – 14:15 Phrase-level Active Learning for Neural Machine Translation
Junjie Hu, Graham Neubig
13:00 – 14:15 Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora
Gaurav Kumar, Philipp Koehn, Sanjeev Khudanpur
13:00 – 14:15 Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement
HyoJung Han, Seokchan Ahn, Yoonjung Choi, Insoo Chung, Sangha Kim, Kyunghyun Cho
13:00 – 14:15 Simultaneous Neural Machine Translation with Constituent Label Prediction
Yasumasa Kano, Katsuhito Sudoh, Satoshi Nakamura
13:00 – 14:15 Contrastive Learning for Context-aware Neural Machine Translation Using Coreference Information
Yongkeun Hwang, Hyeongu Yun, Kyomin Jung
14:15–14:45 🕑
14:45 – 15:45 Session 9: Machine Translation Papers from the Findings of the EMNLP
14:45–15:00 The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
Orevaoghene Ahia, Julia Kreutzer, Sara Hooker
15:00 – 15:15 Subword Mapping and Anchoring across Languages
Giorgos Vernikos, Andrei Popescu-Belis
15:15 – 15:30 Uncertainty-Aware Machine Translation Evaluation
Taisiya Glushkova, Chrysoula Zerva, Ricardo Rei, André F. T. Martins
15:30 – 15:45 Sometimes We Want Ungrammatical Translations
Prasanna Parasarathi

Results

News translation

Full results of the shared task: Findings of the 2021 Conference on Machine Translation (WMT21)

The winner systems were listed according to their average score and average z-score.

→ English

The results were determined with a monolingual direct assessment with document context.

Language pair System Average score Average z-score
Czech → Facebook-AI 77.8 0.111
German → Borderline 71.9 0.126
Hausa → Facebook-AI 74.4 0.248
Icelandic → Facebook-AI 74.5 0.293
Japanese → HW-TSC 73.8 0.141
Russian → NVIDIA-NeMo 77.5 0.137
Chinese → NiuTrans 75.0 0.042

English →

The results were determined with a bilingual direct assessment in a full document.

Language pair System Average score Average z-score
→ Czech Facebook-AI 87.6 0.263
→ German Online-B 83.3 0.266
→ Hausa Facebook-AI 82.7 0.264
→ Icelandic Facebook-AI 84.5 0.594
→ Japanese Facebook-AI 86.4 0.430
→ Russian Online-W 83.3 0.277
→ Chinese HappyNewYear 74.9 0.284

French ↔︎ German

The results were determined with a bilingual direct assessment in a full document.

Language pair System Average score Average z-score
French → German Online-W 87.7 0.088
German → French Online-B 87.9 0.160

Hindi ↔︎ Bengali

The results were determined with a direct assessment without document context.

Language pair System Average score Average z-score
Hindi → Bengali HW-TSC 95.0 0.245
Bengali → Hindi GTCOM 82.1 0.202

Zulu ↔︎ Xhosa

The results were determined with a direct assessment without document context.

Language pair System Average score Average z-score
Zulu → Xhosa TRANSSION 80.7 0.502
Xhosa → Zulu HW-TSC 68.4 0.331

Want to learn more about WMT21?


Edit this article →

Machine Translate is created and edited by contributors like you!

Learn more about contributing →

Licensed under CC-BY-SA-4.0.

Cite this article →