Regresar

SINAI in SimpleText CLEF 2025: Simplifying Biomedical Scientific Texts and Identifying Hallucinations Using GPT-4.1 and Pattern Detection

Abstract:

This paper presents our participation in three different tasks of the CLEF 2025 SimpleText track. For tasks 1.1 and 1.2, we explore the application of advanced language models, specifically GPT-4.1, for the automatic simplification of biomedical texts in English using zero-shot learning. Two versions of prompts were designed and implemented on the Cochrane-auto dataset in Task 1.1 and Task 1.2, with the aim of generating texts that are more understandable for non-specialist audiences. The results show that the model successfully preserves the original semantic structure, identifying complex terms and providing clear restructuring, including brief explanations when necessary. Furthermore, an accurate listing of key elements and effective reorganization of difficult grammatical structures were observed. These characteristics indicate the adaptability of the model to facilitate access to technical information without affecting its accuracy. Finally, the results preliminarily support the effectiveness of prompt design as an approach to improve text comprehension in the biomedical field, without the need for additional supervised training. In Task 2.1, we addressed the problem of detecting creative generation in simplification outputs at the document level. Our approach combined rule-based pattern matching—developed through exploratory analysis of the training set—with the use of the llama-3.1-8b-instruct language model. Surface-level patterns such as one-word sentences, double-space endings, and near-literal context matches were leveraged to pre-label data, while remaining cases were evaluated by the language model using a token-level confidence threshold. The highest-performing run in the sourced subtask achieved an F1-score of 0.976 whereas in the posthoc subtask it was 0.953.