Investigating the Impact of Conceptual Metaphors on LLM-based NLI through Shapley Interactions
Abstract
Metaphorical language is prevalent in everyday communication, often used unconsciously, as in “rising crime.” While LLMs excel at identifying metaphors in text, they struggle with downstream tasks that implicitly require correct metaphor interpretation, such as natural language inference (NLI). This work explores how LLMs perform on NLI with metaphorical input. Particularly, we investigate whether incorporating conceptual metaphors (source and target domains) enhances performance in zero-shot and few-shot settings. Our contributions are two-fold: (1) We create a new dataset, FLUTE.st, extending metaphorical texts in an existing NLI corpus by annotations of source and target domains; and (2) we conduct an ablation study using Shapley values and interactions to assess the extent to which LLMs interpret metaphorical language correctly in NLI. Our results indicate that incorporating conceptual metaphors often improves task performance.
Details
- Organisationseinheit(en)
-
Fachgebiet Maschinelle Sprachverarbeitung
- Externe Organisation(en)
-
Ludwig-Maximilians-Universität München (LMU)
Universität Bielefeld
- Typ
- Aufsatz in Konferenzband
- Seiten
- 17393-17403
- Anzahl der Seiten
- 11
- Publikationsdatum
- 11.2025
- Publikationsstatus
- Veröffentlicht
- Peer-reviewed
- Ja
- ASJC Scopus Sachgebiete
- Theoretische Informatik und Mathematik, Angewandte Informatik, Information systems, Linguistik und Sprache
- Ziele für nachhaltige Entwicklung
- SDG 16 – Frieden, Gerechtigkeit und starke Institutionen
- Elektronische Version(en)
-
https://doi.org/10.18653/v1/2025.findings-emnlp.942 (Zugang:
Offen
)
