Skip to main content Link Search Menu Expand Document (external link)

WMT21

Conference on Machine Translation


The Sixth Conference on Machine Translation (WMT21) took place from 10 November to 11 November, 2021, at EMNLP 2021 in Punta Cana in the Dominican Republic and online.

statmt.org/wmt21/program.html

Table of contents
  1. Tasks
    1. Shared tasks
    2. Evaluation tasks
    3. Other tasks
  2. Schedule
    1. Day 1
    2. Day 2
  3. Results
    1. News translation
      1. → English
      2. English →
      3. French ↔︎ German
      4. Hindi ↔︎ Bengali
      5. Zulu ↔︎ Xhosa

Tasks

Shared tasks

Evaluation tasks

Other tasks

Schedule

Day 1

   
09:00 – 09:10 Opening Remarks
09:10 – 10:15 Session 1: Shared Task Overview Papers I
09:10 – 10:15 Findings of the 2021 Conference on Machine Translation (WMT21)
Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondřej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta R. Costa-jussà, Cristina España-Bonet, Angela Fan, Christian Federmann, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christopher Homan, Matthias Huck, Kwabena Amponsah-Kaakyire, Jungo Kasai, Daniel Khashabi, Kevin Knight, Tom Kocmi, Philipp Koehn, Nicholas Lourie, Christof Monz, Makoto Morishita, Masaaki Nagata, Ajay Nagesh, Toshiaki Nakazawa, Matteo Negri, Santanu Pal, Allahsera Auguste Tapo, Marco Turchi, Valentin Vydrin and Marcos Zampieri
09:00 – 09:25 News Translation Task
09:25 – 09:35 Similar Languages Translation Task
09:35 – 09:45 Automatic Postediting Task
09:45 – 09:55 Triangular Translation Task
09:55 – 10:05 Indo-European Multilingual Translation Task
10:05 – 10:15 Findings on the WMT 2021 Shared Task on Large-Scale Multilingual Machine Translation
Guillaume Wenzek, Vishrav Chaudhary, Angela Fan, Sahir Gomez, Naman Goyal, Somya Jain, Douwe Kiela, Tristan Thrush and Francisco Guzmán
10:15 – 10:30 ☕️
10:30 – 12:00 News Translation Task
10:30 – 12:00 GTCOM Neural Machine Translation Systems for WMT21
Chao Bei and Hao Zong
10:30 – 12:00 The University of Edinburgh’s English-German and English-Hausa Submissions to the WMT21 News Translation Task
Pinzhen Chen, Jindřich Helcl, Ulrich Germann, Laurie Burchell, Nikolay Bogoychev, Antonio Valerio Miceli Barone, Jonas Waldendorf, Alexandra Birch and Kenneth Heafield
10:30 – 12:00 Tune in: The AFRL WMT21 News-Translation Systems
Grant Erdmann, Jeremy Gwinnup and Tim Anderson
10:30 – 12:00 The TALP-UPC Participation in WMT21 News Translation Task: an mBART-based NMT Approach
Carlos Escolano, Ioannis Tsiamas, Christine Basta, Javier Ferrando, Marta R. Costa-jussà and José A. R. Fonollosa
10:30 – 12:00 CUNI Systems in WMT21: Revisiting Backtranslation Techniques for English-Czech NMT
Petr Gebauer, Ondřej Bojar, Vojtěch Švandelík and Martin Popel
10:30 – 12:00 Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language Pairs
Amr Hendy, Esraa A. Gad, Mohamed Abdelghaffar, Jailan S. ElMosalami, Mohamed Afify, Ahmed Y. Tawfik and Hany Hassan Awadalla
10:30 – 12:00 Miðeind’s WMT 2021 Submission
Haukur Jónsson, Haukur Barri Símonarson, Vésteinn Snæbjarnarson, Pétur Orri Ragnarson and Vilhjálmur Þorsteinsson
10:30 – 12:00 Allegro.eu Submission to WMT21 News Translation Task
Mikołaj Koszowski, Karol Grzegorczyk and Tsimur Hadeliya
10:30 – 12:00 Illinois Japanese $$ English News Translation for WMT 2021
Giang Le, Shinka Mori and Lane Schwartz
10:30 – 12:00 [email protected]: Contrastive Learning-reinforced Domain Adaptation in Neural Machine Translation
Zuchao Li, Masao Utiyama, Eiichiro Sumita and Hai Zhao
10:30 – 12:00 The Fujitsu DMATH Submissions for WMT21 News Translation and Biomedical Translation Tasks
Ander Martinez
10:30 – 12:00 Adam Mickiewicz University’s English-Hausa Submissions to the WMT 2021 News Translation Task
Artur Nowakowski and Tomasz Dwojak
10:30 – 12:00 eTranslation’s Submissions to the WMT 2021 News Translation Task
Csaba Oravecz, Katina Bontcheva, David Kolovratník, Bhavani Bhaskar, Michael Jellinghaus and Andreas Eisele
10:30 – 12:00 The University of Edinburgh’s Bengali-Hindi Submissions to the WMT21 News Translation Task
Proyag Pal, Alham Fikri Aji, Pinzhen Chen and Sukanta Sen
10:30 – 12:00 The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21
Lihua Qian, Yi Zhou, Zaixiang Zheng, Yaoming ZHU, Zehui Lin, Jiangtao Feng, Shanbo Cheng, Lei Li, Mingxuan Wang and Hao Zhou
10:30 – 12:00 NVIDIA NeMo’s Neural Machine Translation Systems for English-German and English-Russian News and Biomedical Tasks at WMT21
Sandeep Subramanian, Oleksii Hrinchuk, Virginia Adams and Oleksii Kuchaiev
10:30 – 12:00 Facebook AI’s WMT21 News Translation Task Submission
Chau Tran, Shruti Bhosale, James Cross, Philipp Koehn, Sergey Edunov and Angela Fan
10:30 – 12:00 Tencent Translation System for the WMT21 News Translation Task
Longyue Wang, Mu Li, Fangxu Liu, Shuming Shi, Zhaopeng Tu, Xing Wang, Shuangzhi Wu, Jiali Zeng and Wen Zhang
10:30 – 12:00 HW-TSC’s Participation in the WMT 2021 News Translation Shared Task
Daimeng Wei, Zongyao Li, Zhanglin Wu, Zhengzhe Yu, Xiaoyu Chen, Hengchao Shang, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang and Ying Qin
10:30 – 12:00 LISN @ WMT 2021
Jitao Xu, Minh Quang Pham, Sadaf Abdul Rauf and François Yvon
10:30 – 12:00 WeChat Neural Machine Translation Systems for WMT21
Xianfeng Zeng, Yijin Liu, Ernan Li, Qiu Ran, Fandong Meng, Peng Li, Jinan Xu and Jie Zhou
10:30 – 12:00 Small Model and In-Domain Data Are All You Need
Hui Zeng
10:30 – 12:00 The Mininglamp Machine Translation System for WMT21
Shiyu Zhao, Xiaopu Li, Minghui Wu and Jie Hao
10:30 – 12:00 The NiuTrans Machine Translation Systems for WMT21
Shuhan Zhou, Tao Zhou, Binghao Wei, Yingfeng Luo, Yongyu Mu, Zefan Zhou, Chenglong Wang, Xuanjun Zhou, Chuanhao Lv, Yi Jing, Laohu Wang, Jingnan Zhang, Canan Huang, Zhongxiang Yan, Chi Hu, Bei Li, Tong Xiao and Jingbo Zhu
10:30 – 12:00 Similar Languages Translation Task
10:30 – 12:00 Improving Similar Language Translation With Transfer Learning
Ife Adebara and Muhammad Abdul-Mageed
10:30 – 12:00 T4T Solution: WMT21 Similar Language Task for the Spanish-Catalan and Spanish-Portuguese Language Pair
Miguel Canals and Marc Raventós Tato
10:30 – 12:00 Neural Machine Translation for Tamil–Telugu Pair
Sahinur Rahman Laskar, Bishwaraj Paul, Prottay Kumar Adhikary, Partha Pakray and Sivaji Bandyopadhyay
10:30 – 12:00 Low Resource Similar Language Neural Machine Translation for Tamil-Telugu
Vandan Mujadia and Dipti Sharma
10:30 – 12:00 Similar Language Translation for Catalan, Portuguese and Spanish Using Marian NMT
Reinhard Rapp
10:30 – 12:00 NITK-UoH: Tamil-Telugu Machine Translation Systems for the WMT21 Similar Language Translation Task
Richard Saldanha, Ananthanarayana V. S, Anand Kumar M and Parameswari Krishnamurthy
10:30 – 12:00 A3-108 Machine Translation System for Similar Language Translation Shared Task 2021
Saumitra Yadav and Manish Shrivastava
10:30 – 12:00 Automatic Post-Editing Task
10:30 – 12:00 Netmarble AI Center’s WMT21 Automatic Post-Editing Shared Task Submission
Shinhyeok Oh, Sion Jang, Hu Xu, Shounan An and Insoo Oh
10:30 – 12:00 Adapting Neural Machine Translation for Automatic Post-Editing
Abhishek Sharma, Prabhakar Gupta and Anil Nelakanti
10:30 – 12:00 Triangular Translation Task
10:30 – 12:00 ISTIC’s Triangular Machine Translation System for WMT2021
Hangcheng Guo, Wenbin Liu, Yanqing He, Tian Lan, Hongjiao Xu, Zhenfeng Wu and You Pan
10:30 – 12:00 HW-TSC’s Participation in the WMT 2021 Triangular MT Shared Task
Zongyao Li, Daimeng Wei, Hengchao Shang, Xiaoyu Chen, Zhanglin Wu, Zhengzhe Yu, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang and Ying Qin
10:30 – 12:00 DUTNLP Machine Translation System for WMT21 Triangular Translation Task
Huan Liu, Junpeng Liu, Kaiyu Huang and Degen Huang
10:30 – 12:00 Pivot Based Transfer Learning for Neural Machine Translation: CFILT IITB @ WMT 2021 Triangular MT
Shivam Mhaskar and Pushpak Bhattacharyya
10:30 – 12:00 Papago’s Submissions to the WMT21 Triangular Translation Task
Jeonghyeok Park, Hyunjoong Kim and Hyunchang Cho
10:30 – 12:00 Indo-European Multilingual Translation Task
10:30 – 12:00 Machine Translation of Low-Resource Indo-European Languages
Wei-Rui Chen and Muhammad Abdul-Mageed
10:30 – 12:00 CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task
Josef Jon, Michal Novák, João Paulo Aires, Dusan Varis and Ondřej Bojar
10:30 – 12:00 Transfer Learning with Shallow Decoders: BSC at WMT2021’s Multilingual Low-Resource Translation for Indo-European Languages Shared Task
Ksenia Kharitonova, Ona de Gibert Bonet, Jordi Armengol-Estapé, Mar Rodriguez i Alvarez and Maite Melero
10:30 – 12:00 [email protected]: North-Germanic Low-Resource Multilingual NMT
Svetlana Tchistiakova, Jesujoba Alabi, Koel Dutta Chowdhury, Sourav Dutta and Dana Ruiter
10:30 – 12:00 TenTrans Multilingual Low-Resource Translation System for WMT21 Indo-European Languages Task
Han Yang, Bojie Hu, Wanying Xie, ambyera han, Pan Liu, Jinan Xu and Qi Ju
10:30 – 12:00 Large-Scale Multilingual Translation Task
10:30 – 12:00 The University of Maryland, College Park Submission to Large-Scale Multilingual Shared Task at WMT 2021
Saptarashmi Bandyopadhyay, Tasnim Kabir, Zizhen Lian and Marine Carpuat
10:30 – 12:00 To Optimize, or Not to Optimize, That Is the Question: TelU-KU Models for WMT21 Large-Scale Multilingual Machine Translation
Sari Dewi Budiwati, Tirana Fatyanosa, Mahendra Data, Dedy Rahman Wijaya, Patrick Adolf Telnoni, Arie Ardiyanti Suryani, Agus Pratondo and Masayoshi Aritsugi
10:30 – 12:00 MMTAfrica: Multilingual Machine Translation for African Languages
Chris Chinenye Emezue and Bonaventure F. P. Dossou
10:30 – 12:00 The LMU Munich System for the WMT 2021 Large-Scale Multilingual Machine Translation Shared Task
Wen Lai, Jindřich Libovický and Alexander Fraser
10:30 – 12:00 Back-translation for Large-Scale Multilingual Machine Translation
Baohao Liao, Shahram Khadivi and Sanjika Hewavitharana
10:30 – 12:00 Maastricht University’s Large-Scale Multilingual Machine Translation System for WMT 2021
Danni Liu and Jan Niehues
10:30 – 12:00 Data Processing Matters: SRPH-Konvergen AI’s Machine Translation System for WMT’21
Lintang Sutawika and Jan Christian Blaise Cruz
10:30 – 12:00 TenTrans Large-Scale Multilingual Machine Translation System for WMT21
Wanying Xie, Bojie Hu, Han Yang, Dong Yu and Qi Ju
10:30 – 12:00 Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task
Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan, Xia Song and Furu Wei
10:30 – 12:00 HW-TSC’s Participation in the WMT 2021 Large-Scale Multilingual Translation Task
Zhengzhe Yu, Daimeng Wei, Zongyao Li, Hengchao Shang, Xiaoyu Chen, Zhanglin Wu, Jiaxin Guo, Minghan Wang, Lizhi Lei, Min Zhang, Hao Yang and Ying Qin
12:00 – 13:00 🍴
13:00 – 14:15 Session 3: Panel Discussion on Evaluation
Nitika Mathur (Univ. Melbourne), Benjamin Marie (NICT), Ricardo Rei (Unbabel), Tom Kocmi (Microsoft) and moderated by Markus Freitag (Google)
14:15 – 14:45 🕑
14:45 – 16:15 Session 4: Research Papers on Evaluation
14:45 – 16:15 On the Stability of System Rankings at WMT
Rebecca Knowles
14:45 – 16:15 To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation
Tom Kocmi, Christian Federmann, Roman Grundkiewicz, Marcin Junczys-Dowmunt, Hitokazu Matsushita and Arul Menezes
14:45 – 16:15 Just Ask! Evaluating Machine Translation by Asking and Answering Questions
Mateusz Krubiński, Erfan Ghadery, Marie-Francine Moens and Pavel Pecina
14:45 – 16:15 A Fine-Grained Analysis of BERTScore
Michael Hanna and Ondřej Bojar
14:45 – 16:15 Evaluating Multiway Multilingual NMT in the Turkic Languages
Jamshidbek Mirzakhalov, Anoop Babu, Aigiz Kunafin, Ahsan Wahab, Bekhzodbek Moydinboyev, Sardana Ivanova, Mokhiyakhon Uzokova, Shaxnoza Pulatova, Duygu Ataman, Julia Kreutzer, Francis Tyers, Orhan Firat, John Licato and Sriram Chellappan
14:45 – 16:15 Extending Challenge Sets to Uncover Gender Bias in Machine Translation: Impact of Stereotypical Verbs and Adjectives
Jonas-Dario Troles and Ute Schmid
16:15 - 16:45 ☕️
16:45 – 18:15 Session 5: Research Papers on Data
16:45 – 18:15 Continual Learning in Multilingual NMT via Language-Specific Embeddings
Alexandre Berard
16:45 – 18:15 DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues
Sheila Castilho, João Lucas Cavalheiro Camargo, Miguel Menezes and Andy Way
16:45 – 18:15 Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters
Asa Cooper Stickland, Alexandre Berard and Vassilina Nikoulina
16:45 – 18:15 Translation Transformers Rediscover Inherent Data Domains
Maksym Del, Elizaveta Korotkova and Mark Fishel
16:45 – 18:15 Improving Machine Translation of Rare and Unseen Word Senses
Viktor Hangya, Qianchu Liu, Dario Stojanovski, Alexander Fraser and Anna Korhonen
16:45 – 18:15 Pushing the Right Buttons: Adversarial Evaluation of Quality Estimation
Diptesh Kanojia, Marina Fomicheva, Tharindu Ranasinghe, Frédéric Blain, Constantin Orăsan and Lucia Specia

Day 2

   
09:00 – 10:15 Session 6: Shared Task Overview Papers I
09:00 – 09:12 Findings of the WMT 2021 Shared Task on Efficient Translation
Kenneth Heafield, Qianqian Zhu and Roman Grundkiewicz
09:12 – 09:24 Findings of the WMT Shared Task on Machine Translation Using Terminologies
Md Mahfuz Ibn Alam, Ivana Kvapilíková, Antonios Anastasopoulos, Laurent Besacier, Georgiana Dinu, Marcello Federico, Matthias Gallé, Kweonwoo Jung, Philipp Koehn and Vassilina Nikoulina
09:24 – 09:36 Findings of the WMT 2021 Biomedical Translation Shared Task: Summaries of Animal Experiments as New Test Set
Lana Yeganova, Dina Wiemann, Mariana Neves, Federica Vezzani, Amy Siu, Inigo Jauregi Unanue, Maite Oronoz, Nancy Mah, Aurélie Névéol, David Martinez, Rachel Bawden, Giorgio Maria Di Nunzio, Roland Roller, Philippe Thomas, Cristian Grozea, Olatz Perez-de-Viñaspre, Maika Vicente Navarro and Antonio Jimeno Yepes
09:36 – 09:48 Findings of the WMT 2021 Shared Task on Quality Estimation
Lucia Specia, Frédéric Blain, Marina Fomicheva, Chrysoula Zerva, Zhenhao Li, Vishrav Chaudhary and André F. T. Martins
09:48 – 10:00 Findings of the WMT 2021 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT
Jindřich Libovický and Alexander Fraser
10:00 – 10:15 Results of the WMT21 Metric Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain
Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie and Ondřej Bojar
10:15 – 10:30 ☕️
10:30 – 12:00 Efficient Translation Task
10:30 – 12:00 Efficient Machine Translation with Model Pruning and Quantization
Maximiliana Behnke, Nikolay Bogoychev, Alham Fikri Aji, Kenneth Heafield, Graeme Nail, Qianqian Zhu, Svetlana Tchistiakova, Jelmer van der Linde, Pinzhen Chen, Sidharth Kashyap and Roman Grundkiewicz
10:30 – 12:00 HW-TSC’s Participation in the WMT 2021 Efficiency Shared Task
Hengchao Shang, Ting Hu, Daimeng Wei, Zongyao Li, Jianfei Feng, ZhengZhe Yu, Jiaxin Guo, Shaojun Li, Lizhi Lei, ShiMin Tao, Hao Yang, Jun Yao and Ying Qin
10:30 – 12:00 The NiuTrans System for the WMT 2021 Efficiency Task
Chenglong Wang, Chi Hu, Yongyu Mu, Zhongxiang Yan, Siming Wu, Yimin Hu, Hang Cao, Bei Li, Ye Lin, Tong Xiao and Jingbo Zhu
10:30 – 12:00 TenTrans High-Performance Inference Toolkit for WMT2021 Efficiency Task
Kaixin WU, Bojie Hu and Qi Ju
10:30 – 12:00 Terminology Translation Task
10:30 – 12:00 Lingua Custodia’s Participation at the WMT 2021 Machine Translation Using Terminologies Shared Task
Melissa Ailem, Jingshu Liu and Raheel Qader
10:30 – 12:00 Kakao Enterprise’s WMT21 Machine Translation Using Terminologies Task Submission
Yunju Bak, Jimin Sun, Jay Kim, Sungwon Lyu and Changmin Lee
10:30 – 12:00 The SPECTRANS System Description for the WMT21 Terminology Task
Nicolas Ballier, Dahn Cho, Bilal Faye, Zong-You Ke, Hanna Martikainen, Mojca Pecman, Guillaume Wisniewski, Jean-Baptiste Yunès, Lichao Zhu and Maria Zimina-Poirot
10:30 – 12:00 Dynamic Terminology Integration for COVID-19 and Other Emerging Domains
Toms Bergmanis and Mārcis Pinnis
10:30 – 12:00 CUNI Systems for WMT21: Terminology Translation Shared Task
Josef Jon, Michal Novák, João Paulo Aires, Dusan Varis and Ondřej Bojar
10:30 – 12:00 PROMT Systems for WMT21 Terminology Translation Task
Alexander Molchanov, Vladislav Kovalenko and Fedor Bykov
10:30 – 12:00 SYSTRAN @ WMT 2021: Terminology Task
Minh Quang Pham, Josep Crego, Antoine Senellart, Dan Berrebbi and Jean Senellart
10:30 – 12:00 TermMind: Alibaba’s WMT21 Machine Translation Using Terminologies Task Submission
Ke Wang, Shuqin Gu, Boxing Chen, Yu Zhao, Weihua Luo and Yuqi Zhang
10:30 – 12:00 Biomedical Translation Task
10:30 – 12:00 FJWU Participation for the WMT21 Biomedical Translation Task
Sumbal Naz, Sadaf Abdul Rauf and Sami Ul Haq
10:30 – 12:00 High Frequent In-domain Words Segmentation and Forward Translation for the WMT21 Biomedical Task
Bardia Rafieian and Marta Ruiz Costa Jussa
10:30 – 12:00 Huawei AARC’s Submissions to the WMT21 Biomedical Translation Task: Domain Adaption from a Practical Perspective
Weixuan Wang, Wei Peng, Xupeng Meng and Qun Liu
  Tencent AI Lab Machine Translation Systems for the WMT21 Biomedical Translation Task
Xing Wang, Zhaopeng Tu and Shuming Shi
10:30 – 12:00 HW-TSC’s Submissions to the WMT21 Biomedical Translation Task
Hao Yang, Zhanglin Wu, Zhengzhe Yu, Xiaoyu Chen, Daimeng Wei, Zongyao Li, Hengchao Shang, Minghan Wang, Jiaxin Guo, Lizhi Lei, chuanfei xu, Min Zhang and Ying Qin
10:30 - 12:00 LISN @ WMT 2021
Jitao Xu, Minh Quang Pham, Sadaf Abdul Rauf and François Yvon
10:30 – 12:00 Quality Estimation Task
10:30 – 12:00 RTM Super Learner Results at Quality Estimation Task
Ergun Biçici
10:30 – 12:00 HW-TSC’s Participation at WMT 2021 Quality Estimation Shared Task
Yimeng Chen, Chang Su, Yingtao Zhang, Yuxia Wang, Xiang Geng, Hao Yang, Shimin Tao, Guo Jiaxin, Wang Minghan, Min Zhang, Yujia Liu and Shujian Huang
10:30 – 12:00 Ensemble Fine-tuned mBERT for Translation Quality Estimation
Shaika Chowdhury, Naouel Baili and Brian Vannah
10:30 – 12:00 The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task
Shuoyang Ding, Marcin Junczys-Dowmunt, Matt Post, Christian Federmann and Philipp Koehn
10:30 – 12:00 TUDa at WMT21: Sentence-Level Direct Assessment with Adapters
Gregor Geigle, Jonas Stadtmüller, Wei Zhao, Jonas Pfeiffer and Steffen Eger
10:30 – 12:00 Quality Estimation Using Dual Encoders with Transfer Learning
Dam Heo, WonKee Lee, Baikjin Jung and Jong-Hyeok Lee
10:30 – 12:00 ICL’s Submission to the WMT21 Critical Error Detection Shared Task
Genze Jiang, Zhenhao Li and Lucia Specia
10:30 – 12:00 Papago’s Submission for the WMT21 Quality Estimation Shared Task
Seunghyun Lim, Hantae Kim and Hyunjoong Kim
10:30 – 12:00 NICT Kyoto Submission for the WMT’21 Quality Estimation Task: Multimetric Multilingual Pretraining for Critical Error Detection
Raphael Rubino, Atsushi Fujita and Benjamin Marie
10:30 – 12:00 QEMind: Alibaba’s Submission to the WMT21 Quality Estimation Shared Task
Jiayi Wang, Ke Wang, Boxing Chen, Yu Zhao, Weihua Luo and Yuqi Zhang
10:30 – 12:00 Direct Exploitation of Attention Weights for Translation Quality Estimation
Lisa Yankovskaya and Mark Fishel
10:30 – 12:00 IST-Unbabel 2021 Submission for the Quality Estimation Shared Task
Chrysoula Zerva, Daan van Stigt, Ricardo Rei, Ana C Farinha, Pedro Ramos, José G. C. de Souza, Taisiya Glushkova, Miguel Vera, Fabio Kepler and André F. T. Martins
10:30 – 12:00 Unsupervised and Very Low Resource Translation Task
10:30 – 12:00 The IICT-Yverdon System for the WMT 2021 Unsupervised MT and Very Low Resource Supervised MT Task
Àlex R. Atrio, Gabriel Luthier, Axel Fahy, Giorgos Vernikos, Andrei Popescu-Belis and Ljiljana Dolamic
10:30 – 12:00 Unsupervised Translation of German–Lower Sorbian: Exploring Training and Novel Transfer Methods on a Low-Resource Language
Lukas Edman, Ahmet Üstün, Antonio Toral and Gertjan van Noord
10:30 – 12:00 The LMU Munich Systems for the WMT21 Unsupervised and Very Low-Resource Translation Task
Jindřich Libovický and Alexander Fraser
10:30 – 12:00 Language Model Pretraining and Transfer Learning for Very Low Resource Languages
Jyotsana Khatri, Rudra Murthy and Pushpak Bhattacharyya
10:30 – 12:00 NRC-CNRC Systems for Upper Sorbian-German and Lower Sorbian-German Machine Translation 2021
Rebecca Knowles and Samuel Larkin
10:30 – 12:00 NoahNMT at WMT 2021: Dual Transfer for Very Low Resource Supervised Machine Translation
Meng Zhang, Minghao Wu, Pengfei Li, Liangyou Li and Qun Liu
10:30 – 12:00 Metrics Task
10:30 – 12:00 cushLEPOR: Customised hLEPOR Metric Using LABSE Distilled Knowledge Model to Improve Agreement with Human Judgements
Lifeng Han, Irina Sorokina, Gleb Erofeev and Serge Gladkoff
10:30 – 12:00 MTEQA at WMT21 Metrics Shared Task
Mateusz Krubiński, Erfan Ghadery, Marie-Francine Moens and Pavel Pecina
10:30 – 12:00 Are References Really Needed? Unbabel-IST 2021 Submission for the Metrics Shared Task
Ricardo Rei, Ana C Farinha, Chrysoula Zerva, Daan van Stigt, Craig Stewart, Pedro Ramos, Taisiya Glushkova, André F. T. Martins and Alon Lavie
10:30 – 12:00 Regressive Ensemble for Machine Translation Quality Evaluation
Michal Stefanik, Vít Novotný and Petr Sojka
10:30 – 12:00 Multilingual Machine Translation Evaluation Metrics Fine-tuned on Pseudo-Negative Examples for WMT 2021 Metrics Task
Kosuke Takahashi, Yoichi Ishibashi, Katsuhito Sudoh and Satoshi Nakamura
10:30 – 12:00 RoBLEURT Submission for WMT2021 Metrics Task
Yu Wan, Dayiheng Liu, Baosong Yang, Tianchi Bi, Haibo Zhang, Boxing Chen, Weihua Luo, Derek F. Wong and Lidia S. Chao
10:30 – 12:00 Test Suites
10:30 – 12:00 Linguistic Evaluation for the 2021 State-of-the-art Machine Translation Systems for German to English and English to German
Vivien Macketanz, Eleftherios Avramidis, Shushen Manakhimova and Sebastian Möller
12:00 – 13:00 🍴
13:00 – 14:15 Session 8: Research Papers on Training and Modelling
13:00 – 14:15 Pruning Neural Machine Translation for Speed Using Group Lasso
Maximiliana Behnke and Kenneth Heafield
13:00 – 14:15 Phrase-level Active Learning for Neural Machine Translation
Junjie Hu and Graham Neubig
13:00 – 14:15 Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora
Gaurav Kumar, Philipp Koehn and Sanjeev Khudanpur
13:00 – 14:15 Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement
HyoJung Han, Seokchan Ahn, Yoonjung Choi, Insoo Chung, Sangha Kim and Kyunghyun Cho
13:00 – 14:15 Simultaneous Neural Machine Translation with Constituent Label Prediction
Yasumasa Kano, Katsuhito Sudoh and Satoshi Nakamura
13:00 – 14:15 Contrastive Learning for Context-aware Neural Machine Translation Using Coreference Information
Yongkeun Hwang, Hyeongu Yun and Kyomin Jung
14:15–14:45 🕑
14:45 – 15:45 Session 9: Machine Translation Papers from the Findings of the EMNLP
14:45–15:00 The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
Orevaoghene Ahia, Julia Kreutzer and Sara Hooker
15:00 – 15:15 Subword Mapping and Anchoring across Languages
Giorgos Vernikos and Andrei Popescu-Belis
15:15 – 15:30 Uncertainty-Aware Machine Translation Evaluation
Taisiya Glushkova, Chrysoula Zerva, Ricardo Rei and André F. T. Martins
15:30 – 15:45 Sometimes We Want Ungrammatical Translations
Prasanna Parasarathi

Results

Full results of the shared tasks: Findings of the 2021 Conference on Machine Translation (WMT21)

News translation

The winner systems were listed according to their average score and average z-score.

→ English

The results were determined with a monolingual direct assessment with document context.

Language pair System Average score Average z-score
Czech → Facebook-AI 77.8 0.111
German → Borderline 71.9 0.126
Hausa → Facebook-AI 74.4 0.248
Icelandic → Facebook-AI 74.5 0.293
Japanese → HW-TSC 73.8 0.141
Russian → NVIDIA-NeMo 77.5 0.137
Chinese → NiuTrans 75.0 0.042

English →

The results were determined with a bilingual direct assessment in a full document.

Language pair System Average score Average z-score
→ Czech Facebook-AI 87.6 0.263
→ German Online-B 83.3 0.266
→ Hausa Facebook-AI 82.7 0.264
→ Icelandic Facebook-AI 84.5 0.594
→ Japanese Facebook-AI 86.4 0.430
→ Russian Online-W 83.3 0.277
→ Chinese HappyNewYear 74.9 0.284

French ↔︎ German

The results were determined with a bilingual direct assessment in a full document.

Language pair System Average score Average z-score
French → German Online-W 87.7 0.088
German → French Online-B 87.9 0.160

Hindi ↔︎ Bengali

The results were determined with a direct assessment without document context.

Language pair System Average score Average z-score
Hindi → Bengali HW-TSC 95.0 0.245
Bengali → Hindi GTCOM 82.1 0.202

Zulu ↔︎ Xhosa

The results were determined with a direct assessment without document context.

Language pair System Average score Average z-score
Zulu → Xhosa TRANSSION 80.7 0.502
Xhosa → Zulu HW-TSC 68.4 0.331

Edit this article →

Machine Translate is created and edited by contributors like you!

Learn more about contributing →

Licensed under CC-BY-SA-4.0.

Cite this article →