Data mining: frequent patterns and clustering

Two classic halves of data mining, each with its working shown. Frequent patterns & rules mines transactions — baskets, sessions, sets of tags — for itemsets that occur together and the rules between them. Clustering groups points, on a plane you can edit or on the numeric columns of your own CSV. The numbers match the Python libraries most people would use for the same job (see How it was checked).

How to use it

Frequent patterns and rules

Pick an example — 400 made-up grocery baskets with some planted associations (pasta and tomatoes, nappies and beer), or the nine-transaction AllElectronics table from Han, Kamber and Pei’s Data Mining — or paste or open your own. Three shapes are read: one transaction per line (items separated by commas, semicolons or tabs), a one-hot CSV (a header of item names and 0/1, true/false or yes/no cells; an id column is ignored) and a long CSV of (transaction, item) rows. Auto works out which.

Set the minimum support as a fraction of transactions or, ticking as a count, as a number of them, and the longest itemset to look for. Then:

Apriori, level by level shows each level k: how many candidates were joined from the frequent (k−1)-itemsets, which were pruned without being counted because one of their subsets is not frequent (hover to see which), and the counts of the rest — frequent ones in green.
FP-Growth tree draws the FP-tree (each transaction’s frequent items, most frequent first, inserted as a path so common prefixes share nodes), with the header table’s node links dashed, and lists every item’s conditional pattern base — the prefix paths FP-Growth mines recursively. Both algorithms run every time, and the tiles say whether they agree.
Association rules lists A → C for every split of every frequent itemset, with support, confidence, lift, leverage, conviction and Zhang’s metric. Filter by confidence, lift or an item; sort by any measure; download the table as CSV.

For larger baskets data than this page handles comfortably (it holds the transactions as bitsets, so tens of thousands are fine; a very low support can make millions of candidates), filter or sample first in the data wrangler.

Clustering

Choose 2-D points — blobs, two moons, circles, stretched blobs, blobs of different densities or uniform noise, with optional outliers — and edit them: Add, Move or Delete by clicking the plot (it works with a finger too). Or choose CSV: tick the numeric columns to cluster on, choose the scaling (standardise to z-scores, min–max to [0, 1], or none), and optionally a column of known labels to compare against. With more than two columns the plot shows the first two principal components, with the share of variance they hold and their loadings; the clustering itself uses all the columns you ticked.

k-means: k-means++ start picks the initial centres (the first at random, each next one with probability proportional to its squared distance from the nearest centre so far); Assign and Move centres run one half-step of Lloyd’s algorithm each, with the centres' paths drawn; Run to convergence finishes; Best of 10 starts keeps the run with the lowest inertia. Under the plot: the elbow chart (inertia for k = 1…10) and the mean silhouette for each k, each with its suggested k.
DBSCAN: eps and minPts. Core points are filled, border points are rings, noise is ×. The k-distance plot (each point’s distance to its (minPts−1)-th nearest neighbour, sorted) shows where density drops; eps from the knee sets eps there.
Hierarchical: single, complete, average or Ward linkage; cut into a number of clusters or at a height, or click the dendrogram to cut where you clicked. Links below the cut are coloured by cluster.

For every result the tiles give the mean silhouette and, when the true groups are known (generated data, or a label column), the adjusted Rand index against them; the silhouette plot shows every point’s value, grouped by cluster. Two moons and circles are where k-means fails and DBSCAN or single linkage succeed; different densities are where DBSCAN’s single eps struggles.

Settings, your own points, transactions and CSV are kept in this browser; Share puts the settings in a link.

How it was checked

The tests (tests/tools/data-mining.test.js) compare this page’s engine with reference output saved from mlxtend 0.25 (apriori, fpgrowth and association_rules on three data sets — every itemset’s support and every rule’s support, confidence, lift, leverage, conviction and Zhang’s metric agree to 10⁻⁹), scikit-learn 1.9 (k-means from the same starting centres gives the same labels, centres and inertia; silhouette_score and silhouette_samples; DBSCAN labels point for point; StandardScaler and PCA variances, components and projections) and scipy 1.17 (linkage heights and merge sizes for all four methods, and fcluster cuts). The AllElectronics example reproduces the textbook’s levels, its pruned candidates, its FP-tree and the conditional pattern base of I5.

Differences to know about: an empty k-means cluster keeps its old centre here (scikit-learn moves it to a far point); k-means++ picks each centre by plain D² sampling (scikit-learn also tries a few candidates and keeps the best), so the same seed does not give scikit-learn’s starting points.

Limits

Clustering keeps all pairwise distances in memory, so DBSCAN, silhouettes and the elbow chart are for up to 3,000 points and hierarchical clustering for up to 2,000. The FP-tree is drawn when it has at most 400 nodes.

Classic machine learning — decision trees, k-NN, SVMs and friends on 2-D data.
Embedding explorer — clustering and projecting embeddings.
Statistics — descriptive statistics and tests for the columns you cluster.
Probabilistic data structures — counting at scale (Count-Min, HyperLogLog).
Near-duplicate detection — MinHash, LSH and SimHash.
Data wrangler — clean and reshape the CSV first.

Browser support

Any current browser. Editing points works with a mouse, pen or touch.

Privacy

Everything is computed in this tab: no data is uploaded and nothing is loaded from other sites. Your transactions, CSV and points are kept in this browser’s local storage so they are there next time; share links carry only the settings.