Improving Argument Effectiveness Across Ideologies using Instruction-tuned Large Language Models

authored by

Roxanne El Baff, Khalid Al Khatib, Milad Alshomary, Kai Konen, Benno Stein, Henning Wachsmuth

Abstract

Different political ideologies (e.g., liberal and conservative Americans) hold different worldviews, which leads to opposing stances on different issues (e.g., gun control) and, thereby, fostering societal polarization. Arguments are a means of bringing the perspectives of people with different ideologies closer together, depending on how well they reach their audience. In this paper, we study how to computationally turn ineffective arguments into effective arguments for people with certain ideologies by using instruction-tuned large language models (LLMs), looking closely at style features. For development and evaluation, we collect ineffective arguments per ideology from debate.org, and we generate about 30k, which we rewrite using three LLM methods tailored to our task: zero-shot prompting, few-shot prompting, and LLM steering. Our experiments provide evidence that LLMs naturally improve argument effectiveness for liberals. Our LLM-based and human evaluation show a clear preference towards the rewritten arguments. Code and link to the data are available here: github.com/roxanneelbaff/emnlp2024-iesta.

Organisation(s)

Institute of Artificial Intelligence
Natural Language Processing Section

External Organisation(s)

DLR-Institute of Space Systems
Bauhaus-Universität Weimar
University of Groningen

Type

Conference contribution

Pages

4604-4622

No. of pages

19

Publication date

01.11.2024

Publication status

Published

Peer reviewed

Yes

Electronic version(s)

https://doi.org/10.18653/v1/2024.findings-emnlp.265 (Access: Closed)