Regresar

Word Overlap Artifacts in Textual Entailment Models: A Study with Multilingual Back Translation on the SNLI Dataset †

Abstract:

This paper investigates the domain of Natural Language Inference (NLI), with an emphasis on Recognizing Textual Entailment (RTE). We utilize the Stanford Natural Language Inference (SNLI) dataset, a benchmark for RTE tasks, to examine the efficacy of machine back translation and model performance in textual entailment. Our methodology employs a cost-effective approach using an open-source machine translation library like MarianMT with Helsinki-NLP/opus-mt models for back translation, applied to the comprehensive SNLI dataset. The concluding analysis demonstrates that no single model, whether back translated or augmented, consistently outperforms the reference English model in all aspects. The performance variations are particular to certain word overlap ranges and categories, suggesting that these models are essentially equivalent to the reference. This study contributes to the comprehension of machine translation′s impact on textual entailment models, emphasizing the complexities in multilingual NLI tasks.