Clustering Algorithms Comparison: Choosing the Right Method for Your Data

Key takeaways

Clustering algorithms are a machine learning technique that helps identify natural groupings within data without requiring predefined labels.

• Clustering algorithms can assign data into “hard clusters” where each point belongs to exactly one group, or “soft clusters” where each point has probabilities of belonging to multiple groups.

• Common types of clustering algorithms include centroid-based, hierarchical, density-based, model-based, and grid-based methods. 

• You can select the clustering algorithm that best suits your data by considering your underlying data distribution, whether or not you want to predefine the number of clusters, and your available resources.

Explore popular clustering approaches, including general categories, specific algorithms, and factors to consider when choosing the right method for your data. Or, start building your expertise in machine learning and data analytics with the IBM Machine Learning Professional Certificate. In as little as three months, you can learn how to compare and contrast different machine learning algorithms by creating recommender systems in Python. 

What is clustering in machine learning?

Clustering is a type of unsupervised machine learning that organizes unlabeled data into groups based on similar characteristics. Unlike supervised learning, where algorithms learn from labeled examples, clustering algorithms examine the relationships between data points to create groups (clusters) where points within each group are more similar to each other than the points in other clusters. The algorithm can base this grouping process on just one feature of your data, or it can use all features present, depending on the type of algorithm you choose [1].

You can use clustering in a variety of professional fields to make sense of complex, uncategorized data and gain insights by observing the types of clusters that form. For example, you might use clustering algorithms to segment customers for behavior-based marketing, group documents by topic or theme, detect anomalies in credit card transactions to identify fraud, group patients by subtypes, or even just reduce the complexity of a large data set [2].

Types of clustering algorithms

While each clustering algorithm will result in natural groupings of your data, selecting the ideal clustering algorithm requires an understanding of your data’s characteristics and the underlying mechanics of your algorithm choice. For example, some clustering algorithms might assign each data point to only one cluster (sometimes called “hard clustering”), while others assign data points with a probability of belonging to each cluster (called “soft clustering”). 

In general, your clustering algorithm is likely to fall within several major categories:

• Centroid-based methods: Centroid-based methods, also called partitioning methods, divide your data into a predefined number of non-overlapping clusters by identifying central points (centroids). With these algorithms, you typically optimize a specific measure of cluster quality, like minimizing within-cluster variance. 

• Hierarchical methods: Hierarchical cluster analysis creates tree-like structures of clusters. This can be through progressively merging smaller clusters into larger ones, called agglomerative clustering, or by progressively splitting larger clusters into smaller ones, called divisive clustering. These methods don’t typically have a pre-defined number of clusters and reveal the relationship between clusters at different scales.

• Density-based methods: Density-based clusters identify clusters as regions where data points pack together, separated by lower-density regions. These algorithms identify clusters of any shape and tag noise points that don’t belong to any cluster. 

• Model-based methods: Model-based methods, also called distribution-based methods, assume data comes from a mixture of several groups, where each group follows a statistical pattern, like a bell-curve or normal distribution. Instead of creating clusters with firm boundaries, these algorithms determine the statistical properties defining each group. This method requires assuming the underlying distribution of your data.

• Grid-based methods: Grid-based methods split the data into a grid structure, assigning data points to corresponding cells. The algorithm then performs clustering operations on the grid cells rather than individual data points, merging cells to obtain clusters. This is advantageous for large and dynamic data sets, as the algorithm can use cells to more quickly perform cluster analysis and remain robust against outliers.

Is PCA a clustering algorithm?

No, principal component analysis (PCA) is a dimensionality reduction technique, not a clustering algorithm. You can use PCA to transform your data into a smaller set of features, known as principal components. This is often used as a pre-processing step to improve the efficacy of downstream cluster analysis. PCA helps to reduce the computational complexity of your data set by decreasing the number of features, removing noise and irrelevant dimensions while retaining the most important information.

Clustering algorithms comparison: Popular approaches 

Once you understand the broader clustering algorithm types, you can explore specific clustering algorithms to determine the right one for your data. Each algorithm has strengths, weaknesses, and a unique approach to segmenting your data. Consider the following comparison of clustering algorithms, which includes some of the most popular.

K-means algorithm 

The k-means algorithm is a widely used centroid-based clustering algorithm. The algorithm works by identifying a set number of “center points,” or centroids, then groups points based on which centroid they are closest to. During the grouping process, the algorithm adjusts the centers and refines groups until everything settles into stable clusters. Common applications include organizing documents by type and image compression.

Strengths:

This algorithm works quickly and efficiently, making it practical even for large data sets. It’s considered to be straightforward to understand and implement, and can provide a solid foundation for clustering analysis. If your natural groups are roughly circular and similarly sized, like geographic regions with even population distribution, this algorithm can be a good choice.

Weaknesses:

With this algorithm, you have to decide how many clusters you want before running it, which can be difficult when you don’t know the patterns in your data. The k-means algorithm also expects round, compact clusters, which means other algorithms (or extended versions of k-means) may be a better choice if you have clusters with unusual shapes or very different sizes.

DBSCAN (density-based spatial clustering of applications with noise)

The DBSCAN algorithm looks for areas where data points cluster together and classify points as either core points (surrounded by many neighboring points), border points (on the edge of clusters), or noise points (isolated points). 

Strengths:

A DBSCAN algorithm can find clusters of any shape, which makes it a more flexible algorithm than k-means. Instead of forcing every point into a group, it can identify outliers and strange data points and determine independently how many clusters exist. This makes it a good choice if your data is messy or if not everything fits into a category.

Weaknesses:

For DBSCAN algorithms, you determine how close points need to be to count as neighbors and how many neighborhoods form a core point. This can take trial and error to determine the best setup for your data. This algorithm may also struggle if different parts of your data have different densities.

Hierarchical clustering

Hierarchical clustering builds a family tree of your data, showing how data points gradually merge or divide into groups of different sizes. For data with a naturally nested structure (such as taxonomic classification or gene expression patterns), this type of algorithm can help you organize your information effectively.

Strengths:

The tree visualization gives you multiple levels of detail so you can see both fine-grained categorizations and broad groupings in the same output. Similarly to DBSCAN, you don’t need to pre-specify group size. Instead, you can decide how to “cut” the tree based on your desired level of detail and application.

Weaknesses:

This approach can be slow for very large amounts of information, making it a better choice for small to medium-sized data sets. In addition, the way you choose to measure distances and merge clusters significantly impacts results and often relies on subjective decisions.

Expectation-maximization (EM) for Gaussian mixture models

EM models assume your data comes from several overlapping groups, where each group follows a bell-curve pattern around some form of central tendency. The algorithm alternates between two steps: the “expectation” step calculates the probability that each data point belongs to each group, while the “maximization” step updates the group definition based on those probabilities.

Strengths:

This algorithm created “soft” clusters where data points can partially belong to several groups. This may reflect real-world boundaries more accurately and can be ideal if your data has several possible groupings. This type of model is typically less sensitive to scale. It is also able to handle clusters of different sizes, which makes it great for complex data (such as medical data).

Weaknesses: 

This algorithm requires you to define groups ahead of time and assumes each group’s data follows a bell curve, which may not match reality in many cases. This algorithm may not be the best choice for binary or categorical data, which often require extended approaches using Bernoulli or multinomial distributions. 

Spectral clustering

Spectral clustering uses a graph-based algorithm to find groups in your data. This algorithm works by representing your data as a network where each of your data points connects to similar nearby points. From there, the algorithm determines which points most strongly connect to one another, finding natural groupings in the data and creating clusters based on connection patterns rather than only geometric proximity.

Strengths:

Spectral clustering can identify clusters with unusual shapes (like crescents or concentric circles) that may be missed with simpler algorithms. You can also choose to determine the number of clusters, or have the algorithm determine the appropriate number for itself. In practice, spectral clustering is relatively simple to implement and often performs better than traditional algorithms like k-means.

Weaknesses:

Spectral clustering can be resource-intensive, which makes it impractical for very large data sets. In addition, creating a similarity network requires intentional decisions for how to measure similarity between points and what distance thresholds should be, each of which can significantly impact results. In some cases, spectral algorithms may struggle with data sets with clusters of different densities and sizes. 

How to evaluate clustering algorithm performance

You can evaluate the quality of your clusters using several metrics, primarily focused on cluster variance and point similarity within each cluster. A few metrics you might decide to measure include:

• Silhouette score: How similar each point is to its own cluster versus other clusters (-1 to 1 value)

• Davies-Bouldin index: The average similarity between each cluster and the most similar cluster to it

• Calinski-Harabasz index: The ratio of between-cluster variance to within-cluster variance

• Within-cluster sum of squares: The measure of compactness, found by summing squared distances from points to their cluster centers 

How to choose the right clustering algorithm

Choosing the right clustering algorithm requires evaluating several key factors related to your data, constraints, and overall objectives. You’ll want to start by examining your data’s properties: Do your clusters appear with roughly similar sizes, or do you have more random or skewed shapes? Is your data high-dimensional? Does it have a lot of noise? Each property can help you narrow down to the right type of algorithm. 

Following this, consider the number of clusters: Do you want to define this yourself, or have the algorithm determine the number of clusters? Do you have varying densities in your cluster? Having varying densities may require more complex algorithms, while simpler methods may be best for clusters of similar densities.

Next, think about: What type of computational resources do you have? Do you need your algorithm to scale well to very large data sets? In some cases, you might want to start with a simpler algorithm like k-means to establish a baseline, then experiment with more sophisticated algorithms to find the one that fits your data best. In many cases, comparing multiple algorithms can give you a better understanding of your data and what method will yield the best results [1].

Explore clustering and other machine learning techniques with our free resources

If you’d like to learn more about machine learning and data analytics techniques or get helpful career advice, consider subscribing to our LinkedIn newsletter, Career Chat. You can also explore more through our free resources below:

• Explore careers: Machine Learning Career Paths: Explore Roles & Specializations 

• Earn a certificate: 6 Machine Learning Certificates + How to Choose the Right One for You

• Learn from experts: 7 Questions with a Data Analytics Professor 

With Coursera Plus, you can learn and earn credentials at your own pace from over 350 leading companies and universities. With a monthly or annual subscription, you’ll gain access to over 10,000 programs—just check the course page to confirm your selection is included.

Build job-ready skills with Coursera Plus

Start 7-day free trial

• 
• 
• 
• 

Start 7-day free trial