Regresar

A study on the impact of pre-processing techniques in Spanish and english text classification over short and large text documents

Abstract:

Nowadays, text mining is a long studied field in science, the vast amount of text resources available has made scientist explore several domains through many different techniques. One of the main processes in text mining are cleaning, reduction and transformation before the application of a classification algorithm. Preprocessing has a large impact in classification algorithms because text is an unstructured form of data with very large number of dimensions and therefore it can be seen as a very sparse matrix. These characteristics that make text so complex are addressed by preprocessing algorithms which extract the main data features. We present a work with a comparison of the performance of the different preprocessing algorithms for a classification problem in two datasets written in Spanish and English.