Fuzzified Clustering and Sample Reduction for Intelligent High Performance Distributed Classification of Heterogeneous Uncertain Big Data

Abstract

diverse datasets efficiently. This paper introduces a Fuzzified Clustering technique with sample reduction and distributed Parallel Classification (FCPC). Fuzzified clustering is particularly well-suited for Big Data as it enables the intelligent partitioning of datasets while managing uncertainties and overlapping data points. The FCPC technique takes advantage of this capability to reduce dataset size, capturing essential data structures and enhancing classification performance. Benchmark Big Data sets are used to compare FCPC with traditional classifiers, which require the entire dataset to fit in memory. Four classification techniques were evaluated in terms of classification evaluation metrics, namely, Accuracy, Area Under the ROC Curve, and F1 Score. The proposed model demonstrated improved classification predictive power with a sample reduction of approximately 90%, leading to enhanced performance and potential reductions in computational resources.