Regresar

Linguistic Capabilities for a Checklist-based evaluation in Automatic Text Simplification

Abstract:

Evaluation in Automatic Text Simplification (ATS) has been carried out by means of automatic metrics such as SARI, BLEU, by manual analysis that takes into account the grammar/fluency, meaning preservation and simplicity of the outputs, readability metrics or by extrinsic evaluation via NLP tasks. These metrics and dimensions give an overviewof what the systems are doing, butwe do not exactly which are the strong and weak points. Inspired by recent literature of Natural Language Processing tasks for classification, in this paper we explore the checklist-based evaluation of the linguistic capabilitiesATS systems need to meet. We apply this evaluation to a syntax aware edit-based ATS system and we point out which are the weakness and the strength of the system, which can also lead to improvements of the system.