Efficient multi-lingual language model fine-tuning
Most of the world’s text is not in English. To enable researchers and practitioners to build impactful solutions in their domains, understanding how our NLP architectures fare in many languages needs to be more than an afterthought.
In this post, we introduce our latest paper that studies multilingual text classification and introduces #MultiFiT, a novel method based on #ULMFiT.
MultiFiT, trained on 100 labeled documents in the target language, outperforms multi-lingual BERT. It also outperforms the cutting-edge LASER algorithm-even though LASER requires a corpus of parallel texts, and MultiFiT does not.
Post: http://nlp.fast.ai/classification/2019/09/10/multifit.html…
Paper: https://arxiv.org/abs/1909.04761
Tweet: https://twitter.com/seb_ruder/status/1186744388908654597?s=20
#NLP #DL #FineTuning
Most of the world’s text is not in English. To enable researchers and practitioners to build impactful solutions in their domains, understanding how our NLP architectures fare in many languages needs to be more than an afterthought.
In this post, we introduce our latest paper that studies multilingual text classification and introduces #MultiFiT, a novel method based on #ULMFiT.
MultiFiT, trained on 100 labeled documents in the target language, outperforms multi-lingual BERT. It also outperforms the cutting-edge LASER algorithm-even though LASER requires a corpus of parallel texts, and MultiFiT does not.
Post: http://nlp.fast.ai/classification/2019/09/10/multifit.html…
Paper: https://arxiv.org/abs/1909.04761
Tweet: https://twitter.com/seb_ruder/status/1186744388908654597?s=20
#NLP #DL #FineTuning