Investigating cross-lingual training for offensive language detection release_lsjgz6n7mrgnnlkrwfpddnkhnq

by Andraž Pelicon, Ravi Shekhar, Blaž Škrlj, Matthew Purver, Senja Pollak

Published in PeerJ Computer Science by PeerJ.

2021   e559

Abstract

Platforms that feature user-generated content (social media, online forums, newspaper comment sections etc.) have to detect and filter offensive speech within large, fast-changing datasets. While many automatic methods have been proposed and achieve good accuracies, most of these focus on the English language, and are hard to apply directly to languages in which few labeled datasets exist. Recent work has therefore investigated the use of <jats:italic>cross-lingual transfer learning</jats:italic> to solve this problem, training a model in a well-resourced language and transferring to a less-resourced target language; but performance has so far been significantly less impressive. In this paper, we investigate the reasons for this performance drop, via a systematic comparison of pre-trained models and intermediate training regimes on five different languages. We show that using a better pre-trained language model results in a large gain in overall performance and in zero-shot transfer, and that intermediate training on other languages is effective when little target-language data is available. We then use multiple analyses of classifier confidence and language model vocabulary to shed light on exactly where these gains come from and gain insight into the sources of the most typical mistakes.
In application/xml+jats format

Archived Files and Locations

application/pdf  7.4 MB
file_gk7bdqjkyzc65fiolglufsw3se
peerj.com (publisher)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article-journal
Stage   published
Date   2021-06-25
Language   en ?
Container Metadata
Open Access Publication
In DOAJ
In Keepers Registry
ISSN-L:  2376-5992
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 3a5c00af-d3ec-43dd-8727-b77ff3fc346f
API URL: JSON