[PDF][PDF] Effect of non-linear deep architecture in sequence labeling

M Wang, CD Manning - … of the Sixth International Joint Conference …, 2013 - aclanthology.org
Proceedings of the Sixth International Joint Conference on Natural …, 2013aclanthology.org
If we compare the widely used Conditional Random Fields (CRF) with newly proposed
“deep architecture” sequence models (Collobert et al., 2011), there are two things changing:
from linear architecture to non-linear, and from discrete feature representation to
distributional. It is unclear, however, what utility nonlinearity offers in conventional
featurebased models. In this study, we show the close connection between CRF and
“sequence model” neural nets, and present an empirical investigation to compare their …
Abstract
If we compare the widely used Conditional Random Fields (CRF) with newly proposed “deep architecture” sequence models (Collobert et al., 2011), there are two things changing: from linear architecture to non-linear, and from discrete feature representation to distributional. It is unclear, however, what utility nonlinearity offers in conventional featurebased models. In this study, we show the close connection between CRF and “sequence model” neural nets, and present an empirical investigation to compare their performance on two sequence labeling tasks–Named Entity Recognition and Syntactic Chunking. Our results suggest that non-linear models are highly effective in low-dimensional distributional spaces. Somewhat surprisingly, we find that a nonlinear architecture offers no benefits in a high-dimensional discrete feature space.
aclanthology.org
Showing the best result for this search. See all results