Evaluating Ensemble Neural Networks as an Alternative to Tree-Based Ensemble Methods for Heart Disease Prediction Using Oversampling Methods

Abstract

Ensemble learning enhances predictive accuracy by combining multiple models, but it often struggles with imbalanced data, which can lead to biased results. To address this challenge, this study explores whether Ensemble Neural Networks (ENN) can be an alternative model to treebased methods like Random Forest (RF), Gradient Boosting (GB), and Extreme Gradient Boosting (XGB) in predicting cardiovascular disease (CVD) and whether it can provide improved results. Unlike single neural networks, ENN combines multiple neural network architectures, like how tree-based models use ensembles of decision trees. This approach might allow ENN to better capture and understand data patterns. To mitigate class imbalance, oversampling techniques such as Random oversampling (ROS), Synthetic minority oversampling technique (SMOTE), Borderline-smote (B-SMOTE), and Adaptive synthetic sampling (ADASYN) are applied. Performance is evaluated using accuracy, F-score, geometric mean (G-mean), and area under the curve (AUC) on three CVD datasets: Heart Disease Health Indicators, Framingham, and Statlog. Results show that ENN, when combined with SMOTE and B-SMOTE, offers a strong alternative for imbalanced classification tasks, though tree-based methods remain more robust in terms of overall performance.