Minimum Covariance Distance-Based SMOTE Approaches for Zero-Inflated Datasets with High-Dimensional Heterogeneous Features in Big Data Analytics

Abstract

Big data in the credit risk landscape is often characterized by zero-inflated datasets, heterogeneity, and high dimensionality. These data aberrations adversely diminish the computational efficacy of the conventional predictive classifiers. To ensure accurate and reliable predictions, it is crucial to remedy these aberrations, as they may result in bias towards the majority class, sparsity, and computational complexity. The modified Euclidean distance (MED)-based synthetic minority oversampling technique (SMOTE) approaches have been suggested in contemporary literature as countermeasures for zero-inflated datasets coupled with heterogeneity. Despite their mathematical tractability, these approaches substantially fail to effectively capture correlations and variability among features. They are also susceptible to heavy-tailed error distributed data points (outliers) and collinearity, rendering them computationally suboptimal in high-dimensional data spaces. In this study, authors present a novelty of supplanting the MED with modified Mahalanobis distance (MMD) to the variants of SMOTE, enhancing their ability to adequately capture correlations, variability, and heterogeneous features. To mitigate the intricacies posed by these multifaceted data aberrations in high-dimensional data settings, the authors propose the fast minimum regularized coefficient determinant (FMRCD) approach to estimate the parameters of the MMD measure. Therefore, this paper enhances the robustness and computational efficiency of SMOTE-based approaches, by leveraging MMD computed intrinsically to the FMRCD approach, in conjunction with classical predictive classifiers. The empirical evidence suggests that our novelty, demonstrates superior predictions and offers computational stability edge over traditional approaches. These contributions circumvent overwhelming data complexities presented by zero-inflated datasets combined with high-dimensional heterogeneity in modelling big data phenomena.