NeurIPS 2023 · Robust Bayesian Optimization

AIRBO

Efficient Robust Bayesian Optimization for Arbitrary Uncertain Inputs

A practical way to optimize expensive black-box systems when the input you choose is not exactly the input the system receives.

Lin Yang, Junlong Lyu, Wenlong Lyu, Zhitang Chen

Read the paper ↗
Code ↗

AIRBO robust prediction curve with uncertain input distributions — AIRBO searches for a solution that stays good when the actual input is drawn from a distribution.

The one-sentence ideaRepresent each uncertain input as a probability distribution, compare distributions with an MMD-based kernel, and let a Gaussian Process guide the search for a robust optimum.

Why this problem matters

The input drifts

Manufacturing tolerance, execution noise, wind, or changing context can move a selected design x to an unobserved actual input x′.

The best point can be fragile

Optimizing only the nominal function may find a sharp peak that performs poorly after a small perturbation. A robust solution should perform well on average under its whole uncertainty distribution.

AIRBO optimizes what is actually experienced

It uses samples from arbitrary input distributions without assuming they are Gaussian or that the exact perturbed input is observable.

From a point estimate to a distribution-aware search

1Choose a design

Propose a nominal input x.

→

2Model uncertainty

Represent the possible actual inputs as samples from P(x).

→

3Compare distributions

Use Maximum Mean Discrepancy (MMD) to build a kernel over distributions.

→

4Find a robust optimum

Use the GP posterior and an acquisition function to select the next query.

Robust objective

$$\underset{x}{\operatorname{argmin}}\;\mathbb{E}_{x’\sim P(x)}[f(x’)]$$

The important shift is conceptual: the object being optimized is not a single point evaluation, but the expected behavior induced by uncertain inputs.

What makes AIRBO efficient

01 · MMD kernel

Compare arbitrary uncertainty directly

Maximum Mean Discrepancy compares probability distributions through their embeddings in a reproducing kernel Hilbert space. This lets the GP propagate input uncertainty into its posterior without imposing a Gaussian shape.

P ↔ Q

02 · Nyström approximation

Keep posterior inference practical

Rather than forming the full quadratic MMD estimate, AIRBO uses a smaller set of landmark samples. The paper reduces the space complexity from O(m²) to O(mh), where h ≪ m.

m × m→m × hlandmark samples keep the kernel compact

AIRBO NeurIPS poster summarizing motivation, MMD kernel, Nyström approximation, and evaluations — The original NeurIPS poster: motivation, distribution-aware GP, efficient inference, and benchmark results in one view.

Evidence from the paper

Arbitrary distributionsEvaluated on complex synthetic uncertainties, including changing and multi-modal distributions.

0.78 s vs. 8.12 sPosterior inference time reported for Nyström (100 samples, 10 landmarks) versus a full empirical estimate at the comparable setting.

Theory + practiceA regret bound is established under MMD estimation error, then tested on synthetic functions and a robot pushing problem.

Why AIRBO is useful

Robustness becomes part of the model, not an afterthought.

When every expensive experiment is exposed to input variation, AIRBO turns that variation into a first-class object for Bayesian optimization. The result is a search procedure designed for the behavior a system will actually experience.

arXiv abstract ↗
NeurIPS record ↗
Implementation ↗
Poster source ↗

Back to Home

Lin YANG

AIRBO - Efficient Robust Bayesian Optimization for Arbitrary Uncertain Inputs