Publication:
Cross-lingual visual pre-training for multimodal machine translation

Thumbnail Image

Organizational Units

Program

KU-Authors

KU Authors

Co-Authors

Çağlayan, O.
Kuyu, M.
Amaç, M. S.
Madhyastha, P.
Erdem, E.
Specia, L.

Advisor

Publication Date

Language

English

Journal Title

Journal ISSN

Volume Title

Abstract

Pre-trained language models have been shown to improve performance in many natural language tasks substantially. Although the early focus of such models was single language pre-training, recent advances have resulted in cross-lingual and visual pre-training methods. In this paper, we combine these two approaches to learn visually-grounded cross-lingual representations. Specifically, we extend the translation language modelling (Lample and Conneau, 2019) with masked region classification and perform pre-training with three-way parallel vision & language corpora. We show that when fine-tuned for multimodal machine translation, these models obtain state-of-the-art performance. We also provide qualitative insights into the usefulness of the learned grounded representations.

Source:

EACL 2021 - 16th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference

Publisher:

Association for Computational Linguistics (ACL)

Keywords:

Subject

Visual languages

Citation

Endorsement

Review

Supplemented By

Referenced By

Copyrights Note

0

Views

1

Downloads

View PlumX Details