Bayesian optimization

Installation

pip install "tabpfn-extensions[bayesian_optimization]"

`expected_improvement`

View source

EI over best_f for a batch of points, differentiable w.r.t. x. forward returns bar-distribution logits as [N_borders, N_samples]; after transposing, raw_space_bardist_.ei integrates the improvement over the predicted distribution in closed form. Because the raw-space borders are an affine rescaling of the z-normalized ones, the logits can be used with the raw-space criterion directly and best_f is passed in the original (unnormalized) target space.

expected_improvement(
    reg: TabPFNRegressor,
    x: torch.Tensor,
    best_f: float,
) -> torch.Tensor

Parameters

Parameter	Type	Default	Description
`reg`	`TabPFNRegressor`	Required	A fitted `TabPFNRegressor` with `differentiable_input=True`.
`x`	`torch.Tensor`	Required	Candidate points of shape `(n, d)`.
`best_f`	`float`	Required	Best objective value observed so far, in the original (unnormalized) target space.

Returns

Type	Description
`torch.Tensor`	EI values of shape `(n,)`.

`propose_next_point`

View source

One acquisition round: screen random candidates, refine the best by EI ascent. Fits reg on the observations via fit_with_differentiable_input, screens n_candidates uniform random points in [0, 1]^d with EI in a single forward pass, then refines the top_k most promising ones by gradient ascent on EI w.r.t. the candidate coordinates.

propose_next_point(
    reg: TabPFNRegressor,
    train_x: torch.Tensor,
    train_y: torch.Tensor,
    *,
    n_candidates: int = 512,
    top_k: int = 4,
    n_refine_steps: int = 8,
    refine_lr: float = 0.05,
) -> torch.Tensor

Parameters

Parameter	Type	Default	Description
`reg`	`TabPFNRegressor`	Required	A `TabPFNRegressor` constructed with `differentiable_input=True`.
`train_x`	`torch.Tensor`	Required	Observed inputs of shape `(n, d)`, scaled to `[0, 1]^d`.
`train_y`	`torch.Tensor`	Required	Observed objective values of shape `(n,)`.
`n_candidates`	`int`	`512`	Number of random candidates screened per round.
`top_k`	`int`	`4`	Number of candidates refined by gradient ascent on EI.
`n_refine_steps`	`int`	`8`	Gradient steps on the candidate coordinates.
`refine_lr`	`float`	`0.05`	Learning rate for the refinement steps.

Returns

Type	Description
`torch.Tensor`	The proposed next point of shape `(d,)`, detached.

Reference

tabpfn

tabpfn-client

tabpfn-extensions

REST API

Bayesian optimization

Installation

`expected_improvement`

`propose_next_point`

​Installation

​expected_improvement

​propose_next_point

Installation

`expected_improvement`

`propose_next_point`