Neural Word Segmentation Learning for Chinese
release_vcmvbaaxhzc3pit2mefbb7ur6i
by
Deng Cai, Hai Zhao
2016
Abstract
Most previous approaches to Chinese word segmentation formalize this problem
as a character-based sequence labeling task where only contextual information
within fixed sized local windows and simple interactions between adjacent tags
can be captured. In this paper, we propose a novel neural framework which
thoroughly eliminates context windows and can utilize complete segmentation
history. Our model employs a gated combination neural network over characters
to produce distributed representations of word candidates, which are then given
to a long short-term memory (LSTM) language scoring model. Experiments on the
benchmark datasets show that without the help of feature engineering as most
existing approaches, our models achieve competitive or better performances with
previous state-of-the-art methods.
In text/plain
format
Archived Files and Locations
application/pdf 799.2 kB
file_3vuaibsbprb4tgfnav4hlp6pou
|
arxiv.org (repository) web.archive.org (webarchive) |
1606.04300v1
access all versions, variants, and formats of this works (eg, pre-prints)