From d4da53795d2d3330bc57fd1d8e3bc30480192dc2 Mon Sep 17 00:00:00 2001 From: Matt Post Date: Sun, 2 Feb 2025 20:13:04 -0500 Subject: [PATCH] Processed metadata corrections (closes #4522) --- data/xml/2025.comedi.xml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/data/xml/2025.comedi.xml b/data/xml/2025.comedi.xml index 8d7fa9b900..597c3d97c2 100644 --- a/data/xml/2025.comedi.xml +++ b/data/xml/2025.comedi.xml @@ -156,10 +156,10 @@ Disagreement in Metaphor Annotation of <fixed-case>M</fixed-case>exican <fixed-case>S</fixed-case>panish Science Tweets - Alec M.Sanchez-Montero + AlecSánchez-Montero GemmaBel-Enguix - Sergio LuisOjeda Trueba - GerardoSierra Martínez + Sergio-LuisOjeda-Trueba + GerardoSierra 155–164 Traditional linguistic annotation methods often strive for a gold standard with hard labels as input for natural language processing models, assuming an underlying objective truth for all tasks. However, disagreement among annotators is a common scenario, even for seemingly objective linguistic tasks, and is particularly prominent in figurative language annotation, since multiple valid interpretations can sometimes coexist. This study presents the annotation process for identifying metaphorical tweets within a corpus of 3733 Public Communication of Science texts written in Mexican Spanish, emphasizing inter-annotator disagreement. Using Fleiss’ and Cohen’s Kappa alongside agreement percentages, we evaluated metaphorical language detection through binary classification in three situations: two subsets of the corpus labeled by three different non-expert annotators each, and a subset of disagreement tweets, identified in the non-expert annotation phase, re-labeled by three expert annotators. Our results suggest that expert annotation may improve agreement levels, but does not exclude disagreement, likely due to factors such as the relatively novelty of the genre, the presence of multiple scientific topics, and the blending of specialized and non-specialized discourse. Going further, we propose adopting a learning-from-disagreement approach for capturing diverse annotation perspectives to enhance computational metaphor detection in Mexican Spanish. 2025.comedi-1.15