Enhancing Classification Performance through Rough Set Theory Feature Selection: A Comparative Study across Multiple Datasets

Abstract

In Machine Learning (ML), handling high-dimensional data with redundant or irrelevant features presents significant challenges. Effective feature selection is essential for enhancing model performance, reducing computational complexity, and improving interpretability. Rough Set Theory (RST) provides a powerful mathematical framework for managing uncertainty, making it a valuable tool for feature selection. This study applies RST-based feature selection to five diverse datasets, aiming to eliminate insignificant attributes. We evaluate the performance of various ML models, including Logistic Regression (LR), K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Kernel SVM, Naïve Bayes (NB), Decision Tree (DT) and Random Forest (RF), on both the original and RST-selected datasets. Standard metrics such as accuracy, precision, recall, F1-score and Mean Absolute Error (MAE) are used for evaluation. Our results demonstrate that RST effectively selects relevant features without significant information loss. Models trained on RST-selected datasets exhibit comparable or improved performance, with RF and SVM models showing notable gains in accuracy and efficiency. These findings highlight the potential of RST-based feature selection to enhance ML model performance while reducing computational complexity, making it a valuable approach for various ML applications.