Clustering 4 Fairness

Published as a research software artefact at BNAIC/BeNeLearn 2026 (demo track): c4fairness: A Python Package for Clustering-based Fairness Analysis, Filip Muntean and Emma Beauxis-Aussalet. Video demo.

c4fairness is an open-source Python package that finds where a model's errors fall unevenly. It clusters the rows of a model's test set and reports how prediction-error disparities and sensitive-attribute composition vary across the discovered clusters. This surfaces under-served subgroups without pre-specifying the protected group. Works for binary, multi-class, and regression tasks.

Built at Vrije Universiteit Amsterdam (VU), co-funded by University of Twente (UT).

• PyPI: pypi.org/project/c4fairness
• Website: filipmuntean.github.io/c4fairness_website
• Source: github.com/emma-ba/Clustering_4_Fairness

Install

pip install c4fairness              # from PyPI
pip install "c4fairness[web]"       # + the Gradio web UI
pip install "c4fairness[r]"         # + rpy2, for the exact r×c Fisher test

Python 3.10+. The import name is c4fairness; the CLI command is c4fairness.

Quick start

c4fairness --data_path docs/datasets/compas_audit.csv \
    --regular_cols age,priors_count \
    --sensitive_cols sex,race,age --continuous_sensitive_cols age \
    --error_col errors --error_type binary \
    --algorithm kmeans --n_clusters 4 --seed 42

This clusters the test set on age/priors_count (plus the sensitive columns), then writes a per-cluster recap and heatmap showing each cluster's error rate and its sex/race/age make-up. String sensitive columns are one-hot encoded automatically.

What you get

For each discovered cluster the tool reports its size, its error magnitude or rate, the gap against the rest of the test set with a significance test, and the composition of every sensitive attribute you passed in, so a disparity localised to a pocket of the feature space stays visible instead of being flattened by a single per-group average.

Output lands as heatmaps and CSVs: a detailed recap with one row per cluster, and, in experiment mode, an overview with one row per feature-group condition, so you can check whether something survives the choice of what to cluster on.

Features

Cluster-level fairness auditing

• No protected group needs to be specified up front; disparities are discovered, not queried.
• Per-cluster error gap against the rest of the test set, plus omnibus separability tests.
• Supports binary (raw, FPR, FNR, precision), multi-class (per-class, accuracy, confusion cell, one-hot, classwise), and regression errors.

Six clustering algorithms

KMeans, BisectingKMeans, KMedoids, KPrototypes, DBSCAN, HDBSCAN, with Euclidean, Manhattan, or Gower distance for mixed numeric/categorical data. Fixed k, or automatic selection over a range using a silhouette / chi-square / composite objective with tunable weights.

Statistical tests

• Fisher, chi-square, Mann-Whitney, and ANOVA tests picked automatically per error and feature type.
• Optional exact r×c Fisher–Freeman–Halton test via R's fisher.test (through rpy2), with an automatic scipy fallback when R isn't present.
• Benjamini-Hochberg correction across sensitive features in experiment mode.

Experiment mode

Reruns the audit for every combination of feature groups (regular / sensitive / error), so you can check whether a finding survives the choice of what to cluster on. Multi-seed runs supported.

Gradio web UI

c4fairness-web wraps the CLI in a browser app: upload a CSV, assign column roles, run single audits or a full sweep, and get heatmaps, an overview table, and downloadable CSVs, with a streaming run log.

Output

Recap and overview heatmaps colour-coded by family (size / error / sensitive composition, with p-values shaded by significance), PCA / t-SNE / MDS projections, per-attribute composition plots, and CSVs for every table.

Careful defaults

Binary one-hot columns are kept out of StandardScaler: scaling them distorts Euclidean distances, inflating the weight of rarer categories regardless of their importance. For kprototypes, where silhouette can't be computed directly on a mixed distance, the full pairwise distance matrix is precomputed with the same distance the algorithm fits (squared Euclidean + Hamming, weighted by the fitted gamma) and passed to silhouette_score(metric='precomputed').

Datasets

Two audit-ready extracts ship with the repo: a COMPAS classification extract (5050 rows; sex, race, age) and a UCI Student Performance regression extract (670 rows; sex, mother's education, age). The package was also evaluated on Open University (OULAD), German Credit, and Communities & Crime.

Publications and talks

• Demo paper at BNAIC/BeNeLearn 2026: OpenReview.
• Poster at VPH 2026, Milan, Italy: PDF.