Text classification for assisting moderators in online health communities release_jk4vq46rnrfdxjsroadpqahxea

by Jina Huh, Meliha Yetisgen-Yildiz, Wanda Pratt

Published in Journal of Biomedical Informatics by Elsevier BV.

2013   Volume 46, Issue 6, p998-1005

Abstract

Patients increasingly visit online health communities to get help on managing health. The large scale of these online communities makes it impossible for the moderators to engage in all conversations; yet, some conversations need their expertise. Our work explores low-cost text classification methods to this new domain of determining whether a thread in an online health forum needs moderators' help. We employed a binary classifier on WebMD's online diabetes community data. To train the classifier, we considered three feature types: (1) word unigram, (2) sentiment analysis features, and (3) thread length. We applied feature selection methods based on χ² statistics and under sampling to account for unbalanced data. We then performed a qualitative error analysis to investigate the appropriateness of the gold standard. Using sentiment analysis features, feature selection methods, and balanced training data increased the AUC value up to 0.75 and the F1-score up to 0.54 compared to the baseline of using word unigrams with no feature selection methods on unbalanced data (0.65 AUC and 0.40 F1-score). The error analysis uncovered additional reasons for why moderators respond to patients' posts. We showed how feature selection methods and balanced training data can improve the overall classification performance. We present implications of weighing precision versus recall for assisting moderators of online health communities. Our error analysis uncovered social, legal, and ethical issues around addressing community members' needs. We also note challenges in producing a gold standard, and discuss potential solutions for addressing these challenges. Social media environments provide popular venues in which patients gain health-related information. Our work contributes to understanding scalable solutions for providing moderators' expertise in these large-scale, social media environments.
In text/plain format

Archived Files and Locations

application/pdf  653.4 kB
file_35f4ezre5nfclhjdua4nogn2ai
europepmc.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article-journal
Stage   published
Date   2013-09-08
Language   en ?
DOI  10.1016/j.jbi.2013.08.011
PubMed  24025513
PMC  PMC3874858
Wikidata  Q41847527
Journal Metadata
Not in DOAJ
In Keepers Registry
ISSN-L:  1532-0464
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: f021044c-7bb6-4c92-8484-39ba9cd58fe9
API URL: JSON