The Secret Power of Syntax: Improving ChatGPT Translation Quality Through Sentence Constituent Analysis?

The synergy between AI efficiency and human expertise can result in high-quality translations that benefit a wide range of applications, from aca-demic writing and administrative texts to legal documents. Human translators employ a variety of strategies based on their expertise and understanding of both the source and target languages. This paper uses an explorative setting to inves-tigate the potential of CoT prompting including steps of linguistic analysis, like phrase structure analysis and sentence constituent analysis for English to German translations of text passages containing complex noun phrases. Text passages of academic, administrative and law texts containing complex noun phrases were translated by ChatGPT using a zero-shot one question prompt first and a Chain-of-thought (CoT) prompt including steps of linguistic analysis second. To evaluate translation quality, we compared ChatGPT translations with authorized transla-tions of the sample texts. The results show that CoT prompting could dissolve syntactic ambiguities caused by complex noun phrases. In addition to that, the model’s documentation of phrase structure analysis displayed that ChatGPT cor-rectly figured out the noun phrases’ heads in the sample texts. However, ChatGPT translations contain critical mistakes in every example that impair translation qual-ity and underline the unconditional need for human oversight and correction by expert translators. Despite its strong advancements, ChatGPT-4 does not replicate the intuitive reasoning capabilities of human linguistic experts.