An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
release_nr7e2axopnbvva6kocz6wkojfm
by
Yifan Peng, Qingyu Chen, Zhiyong Lu
2020
Abstract
Multi-task learning (MTL) has achieved remarkable success in natural language
processing applications. In this work, we study a multi-task learning model
with multiple decoders on varieties of biomedical and clinical natural language
processing tasks such as text similarity, relation extraction, named entity
recognition, and text inference. Our empirical results demonstrate that the MTL
fine-tuned models outperform state-of-the-art transformer models (e.g., BERT
and its variants) by 2.0% and 1.3% in biomedical and clinical domains,
respectively. Pairwise MTL further demonstrates more details about which tasks
can improve or decrease others. This is particularly helpful in the context
that researchers are in the hassle of choosing a suitable model for new
problems. The code and models are publicly available at
https://github.com/ncbi-nlp/bluebert
In text/plain
format
Archived Files and Locations
application/pdf 557.4 kB
file_mzgsl5z3z5bwzfjrqh7sjz2wxa
|
arxiv.org (repository) web.archive.org (webarchive) |
2005.02799v1
access all versions, variants, and formats of this works (eg, pre-prints)