Publication Details

Is It Worth the Attention?

A Comparative Evaluation of Attention Layers for Argument Unit Segmentation

authored by
Maximilian Spliethöver, Jonas Klaff, Hendrik Heuer
Abstract

Attention mechanisms have seen some success for natural language processing downstream tasks in recent years and generated new state-of-the-art results. A thorough evaluation of the attention mechanism for the task of Argumentation Mining is missing. With this paper, we report a comparative evaluation of attention layers in combination with a bidirectional long short-term memory network, which is the current state-of-the-art approach for the unit segmentation task. We also compare sentence-level contextualized word embeddings to pre-generated ones. Our findings suggest that for this task, the additional attention layer does not improve the performance. In most cases, contextualized embeddings do also not show an improvement on the score achieved by pre-defined embeddings.

Organisation(s)
Natural Language Processing Section
External Organisation(s)
University of Bremen
Type
Conference contribution
Pages
74-82
No. of pages
9
Publication date
08.2019
Publication status
Published
Peer reviewed
Yes
ASJC Scopus subject areas
Software, Language and Linguistics, Linguistics and Language
Electronic version(s)
https://doi.org/10.18653/v1/W19-4509 (Access: Open)