Toward Multidiversified Ensemble Clustering of High-Dimensional Data: From Subspaces to Metrics and Beyond

Area of Science:

• Data Science
• Machine Learning
• Computational Biology

Background:

• High-dimensional data presents challenges for ensemble clustering.
• Existing methods often focus on subspaces, neglecting metric diversity.
• A unified framework for metric, subspace, and cluster diversity is lacking.

Purpose of the Study:

• To propose a novel multidiversified ensemble clustering approach.
• To address the curse of dimensionality in clustering.
• To investigate multilevel diversity in metrics, subspaces, and clusters.

Main Methods:

• Created diversified metrics by randomizing a scaled exponential similarity kernel.
• Coupled metrics with random subspaces to form metric-subspace pairs.
• Utilized an entropy-based criterion for cluster-wise diversity and consensus functions.

Main Results:

• Developed an ensemble of diversified base clusterings.
• Demonstrated superior performance on 30 high-dimensional datasets.
• Outperformed state-of-the-art algorithms in experiments.

Conclusions:

• The proposed multidiversified approach effectively handles high-dimensional data.
• Integrating metric and subspace diversity improves ensemble clustering.
• The method shows significant potential for applications in gene expression and image/speech analysis.