International Journal of Statistics and Applications
p-ISSN: 2168-5193 e-ISSN: 2168-5215
2024; 14(1): 7-12
doi:10.5923/j.statistics.20241401.02
Received: Feb. 20, 2024; Accepted: Mar. 5, 2024; Published: Mar. 22, 2024
Sanjib Ghosh
Assistant Professor, Department of Statistics, University of Chittagong, Chittagong, Bangladesh
Correspondence to: Sanjib Ghosh, Assistant Professor, Department of Statistics, University of Chittagong, Chittagong, Bangladesh.
Email: |
Copyright © 2024 The Author(s). Published by Scientific & Academic Publishing.
This work is licensed under the Creative Commons Attribution International License (CC BY).
http://creativecommons.org/licenses/by/4.0/
Several studies have demonstrated that effectively combining machine learning models can improve the individual predictions made by the base models. Random forests allow for the selection of a random number of features while bagging increases diversity by sampling with replacement and generating multiple training data sets. As a result, random forest has become a strong contender for various machine learning applications. Assuming equal weights for each base decision tree, however, seems unreasonable because different base decision trees may have varying decision-making abilities due to randomization in sampling and input feature selection. As a result, we offer several methods to enhance the regular random forest's weighting approach and prediction quality. The developed weighting frameworks include multiple stacking-based weighted random forest models, optimal weighted random forest based on area under the curve (AUC), and ideal weighted random forest based on accuracy. The numerical result shows that the stacking-based random forest with binary prediction can introduce significant improvements compared to regular random forest.
Keywords: Optimization, Stacking, Weighted random forest, Out-of-bag prediction, Ensemble
Cite this paper: Sanjib Ghosh, Comparing Regular Random Forest Model with Weighted Random Forest Model for Classification Problem, International Journal of Statistics and Applications, Vol. 14 No. 1, 2024, pp. 7-12. doi: 10.5923/j.statistics.20241401.02.
Figure 1. Random forest classifier uses majority voting of the predictions made by randomly created decision trees to make the final predictions |
(1) |
Figure 2. The optimal weighted random forest classifier utilizes out-of-bag (OOB) binary predictions from the randomly generated decision trees to enhance prediction accuracy |
(2) |
|
Figure 3. Comparing weighted random forest classifier with regular random forest |