Adaptive Bias Discovery for Learning Debiased Classifier

Abstract

Training deep neural networks with empirical risk minimization (ERM) often captures dataset biases, hindering generalization to new or unseen data. Previous solutions either require prior knowledge of biases or utilize training intentionally biased models as auxiliaries; however, they still suffer from multiple biases. To address this, we introduce Adaptive Bias Discovery (ABD), a novel learning framework designed to mitigate the impact of multiple unknown biases. ABD trains an auxiliary model to be adapted to biases based on the debiased parameters from the debiasing phase, allowing it to navigate through multiple biases. Then, samples are reweighted based on the discovered biases to update debiased parameters. Extensive evaluations of synthetic experiments and real-world datasets demonstrate that ABD consistently outperforms existing methods, particularly in real-world applications where multiple unknown biases are prevalent.

Overview

We propose a learning framework that sequentially discovers and removes multiple biases in data without any prior bias information.

1. Bias-adapted model — Generate a bias-sensitive auxiliary model \(f_\phi\) via one-step gradient descent from the debiased parameters \(\theta\) .
2. Adaptive group formation — Partition data into a bias-aligned group (\(G^\odot\) ) and a bias-conflicting group (\(G^\otimes\) ) based on \(f_\phi\) ’s predictions.
3. Iterative debiasing — Minimize worst-case group loss via group DRO; as \(\theta\) becomes robust to one bias, \(\phi\) naturally discovers the next.

Overview of the ABD framework. Illustrated with two biases (Bias1, Bias2) and two learning steps.

Method

Models trained with ERM readily capture spurious correlations in the data, degrading generalization. Existing debiasing methods either require prior knowledge of biases (Group DRO) or can only handle a single bias type (PI, JTT).

ABD operates in two stages. First, bias-adapted parameters \(\phi = \theta - \alpha \nabla_\theta \mathcal{L}(f_\theta)\) are obtained via one-step gradient descent from the debiased parameters \(\theta\) . Since \(f_\phi\) is sensitive to superficial patterns, its predictions partition the data into a bias-aligned group (\(G^\odot\) ) and a bias-conflicting group (\(G^\otimes\) ). The debiased parameters \(\theta\) are then updated by minimizing the worst-case group loss via group DRO.

The key insight is that \(\phi\) is regenerated from \(\theta\) at every step. Once \(\theta\) becomes robust to the first bias, \(\phi\) naturally discovers the next most prominent bias. This MAML-like structure enables sequential discovery and removal of multiple biases without any prior bias information.

The GradCAM visualizations below show how the biased model \(f_\phi\) ’s attention shifts to different regions as training progresses, providing visual evidence that ABD adaptively discovers diverse biases during learning.

GradCAM visualizations of ERM and ABD's biased model $f_\phi$. As training progresses, $f_\phi$'s attention shifts to different bias features.

Results

Colored MNIST — Handling Multiple Biases

This experiment best demonstrates the difference from existing methods, comparing single-bias (Color) and dual-bias (Color + Patch) settings.

OoD test accuracy (%). Single bias vs. multiple biases.

PI discovers only the Color bias and fails to capture Patch, effectively collapsing to 15.3% (a 54.9pp drop) when two biases are present. IRM and Group DRO also remain at the ERM level in the multi-bias setting. ABD sequentially discovers Color then Patch, making it the only method to achieve meaningful OoD performance (62.3%) under multiple biases.

Pearson correlation coefficients within groups.

Real-World Tasks

Datasets with bias annotations — CivilComments & MultiNLI

Worst-case test accuracy (%). The Group column for CivilComments lists the demographic information each algorithm uses for grouping. On MultiNLI, Group DRO* is an oracle setting using groups hand-crafted from prior bias knowledge.

Datasets without bias annotations — Camelyon17 & FMoW (WILDS)

Camelyon17-wilds reports OoD average accuracy; FMoW-wilds reports worst-region accuracy (%).

On CivilComments, ABD (71.1%) surpasses Group DRO (label × Black, 70.0%) which uses bias annotations, without any annotations of its own. On MultiNLI, ABD (67.1%) lands within 0.4pp of the oracle Group DRO* (67.5%) that relies on hand-crafted groups — matching it without any bias information. On Camelyon17, ABD achieves +4.0pp over LISA (81.1% vs 77.1%), and remains competitive with LISA on FMoW. Distribution-alignment and invariance baselines (IRM, CORAL, CGD) stay at or below the ERM level on the WILDS benchmarks.

Composition of bias features in the misclassified group $G^\otimes$ on MultiNLI. After discovering Negation bias, Overlap bias gradually emerges.

MetaShift — Performance Across Distributional Distances

On MetaShift, we vary the distributional distance between training and test to evaluate the robustness of each method. ABD’s advantage becomes more pronounced as the distributional distance increases.

MetaShift test accuracy (%). Larger distance indicates greater distributional shift.

At the smallest distance (0.44), LISA (81.3%) holds a slight edge, but as distance increases, ABD’s advantage becomes clear — at 1.43, ABD (41.8%) surpasses LISA (37.5%) by 4.3pp. This demonstrates that ABD is particularly well-suited for real-world OoD scenarios with large distributional shifts.

GradCAM visualizations on MetaShift test data. ERM relies on background features, while ABD focuses on the target object.