CLEAR: A Comprehensive Linguistic Evaluation of Argument Rewriting by Large Language Models
Journal
Findings of the Association for Computational Linguistics: EMNLP 2025
Type
conference paper
Date Issued
2025
Author(s)
Abstract
While LLMs have been extensively studied on general text generation tasks, there is less research on text rewriting, a task related to general text generation, and particularly on the behavior of models on this task. In this paper we analyze what changes LLMs make in a text rewriting setting. We focus specifically on argumentative texts and their improvement, a task named Argument Improvement (ArgImp). We present CLEAR: an evaluation pipeline consisting of 57 metrics mapped to four linguistic levels: lexical, syntactic, semantic and pragmatic. This pipeline is used to examine the qualities of LLM-rewritten arguments on a broad set of argumentation corpora and compare the behavior of different LLMs on this task and analyze the behavior of different LLMs on this task in terms of linguistic levels. By taking all four linguistic levels into consideration, we find that the models perform ArgImp by shortening the texts while simultaneously increasing average word length and merging sentences. Overall we note an increase in the persuasion and coherence dimensions.
Publisher
Association for Computational Linguistics
File(s)![Thumbnail Image]()
Name
clear_camera_ready.pdf
Size
393.76 KB
Format
Adobe PDF
Checksum (MD5)
80dcc18d3edbd444c5a3b3e1db544070