An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining release_nr7e2axopnbvva6kocz6wkojfm

by Yifan Peng, Qingyu Chen, Zhiyong Lu

Released as a article .

2020  

Abstract

Multi-task learning (MTL) has achieved remarkable success in natural language processing applications. In this work, we study a multi-task learning model with multiple decoders on varieties of biomedical and clinical natural language processing tasks such as text similarity, relation extraction, named entity recognition, and text inference. Our empirical results demonstrate that the MTL fine-tuned models outperform state-of-the-art transformer models (e.g., BERT and its variants) by 2.0% and 1.3% in biomedical and clinical domains, respectively. Pairwise MTL further demonstrates more details about which tasks can improve or decrease others. This is particularly helpful in the context that researchers are in the hassle of choosing a suitable model for new problems. The code and models are publicly available at https://github.com/ncbi-nlp/bluebert
In text/plain format

Archived Files and Locations

application/pdf  557.4 kB
file_mzgsl5z3z5bwzfjrqh7sjz2wxa
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2020-05-06
Version   v1
Language   en ?
arXiv  2005.02799v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: bc4eea0f-1503-48df-8906-b4f0908f9900
API URL: JSON