Decoder readout

Looking for usage documentation? Check out Interpretability and Classification.

Read out TabPFN’s classification head as a label-vote over training rows. TabPFN classifies with an attention-based retrieval head (ManyClassDecoder): each test row attends to the training rows, and the prediction is the average of their one-hot labels weighted by that attention. The prediction is therefore a weighted vote, and P(class c) for a test row is the sum of its attention weights over the training rows whose label is c. get_decoder_readout recovers those per-training-row attention weights, so you can see which training points drive a prediction and by how much. The weights come from the head’s own ManyClassDecoder.attention_weights (tabpfn>=8.3.0), read off a forward pre-hook during predict; forward itself fuses the attention into a single kernel and never materializes them. For each test row the weights sum to 1 (averaged over the decoder’s attention heads and over the ensemble members). Collapsing them by training label with class_vote reproduces the model’s predict_proba up to the head’s log-clamping when softmax_temperature=1.0 and balance_probabilities=False. Both are applied to the decoder’s logits after this readout, so at the library default softmax_temperature=0.9 the temperature sharpens the vote (per estimator, predict_proba ∝ vote ** (1 / T)); predict_proba then differs from the vote by up to ~2 percentage points for binary and ~6 at 10 classes, and balance_probabilities=True widens it further. Only the local tabpfn backend is supported: the client/API backend does not expose the model internals this reads from. Row subsampling (TabPFNClassifier(..., subsample_samples=...)) is not supported, since the weight columns would no longer align to a single set of training rows.

`decoder_readout.get_decoder_readout`

View source

Extract the decoder-head attention weights over training rows.

decoder_readout.get_decoder_readout(
    estimator: BaseEstimator,
    X: np.ndarray,
    *,
    average_over_estimators: bool = True,
) -> tuple[np.ndarray, np.ndarray]

Parameters

Parameter	Type	Default	Description
`estimator`	`BaseEstimator`	Required	A fitted local `TabPFNClassifier`.
`X`	`np.ndarray`	Required	Test inputs, shape `(n_test, n_features)`.
`average_over_estimators`	`bool`	`True`	If `True` (default), average the weights over the preprocessing ensemble members, returning one weight matrix. If `False`, keep the per-member weights on a leading axis.

Returns

Type	Description
`tuple[np.ndarray, np.ndarray]`	`(weights, train_indices)`. `weights` has shape `(n_test, n_train)` when `average_over_estimators` is `True`, else `(n_estimators, n_test, n_train)`. Along the training axis the weights are non-negative and sum to 1 for each test row: `weights[i, j]` is the attention test row `i` pays to training row `j`. `train_indices` has shape `(n_train,)` and indexes the columns of `weights` into the rows of the data the model was fit on (i.e. `weights[:, k]` refers to training row `train_indices[k]`).

`decoder_readout.class_vote`

View source

Collapse per-row readout weights into a per-class vote. Sums the attention weights within each training label, turning the readout into a class distribution. Averaged over the ensemble, this reproduces the model’s predict_proba up to the head’s log-clamping when softmax_temperature=1.0 and balance_probabilities=False. Both are applied downstream of this readout, so at the library default softmax_temperature=0.9 the vote is sharpened (per estimator, predict_proba ∝ vote ** (1 / T)), differing by up to ~2 percentage points for binary and ~6 at 10 classes. It is also exact only in full precision; reduced precision costs ~1e-2 relative, so fit with inference_precision=torch.float32 for a tight match.

decoder_readout.class_vote(
    weights: np.ndarray,
    y_train: np.ndarray,
    classes: np.ndarray | None = None,
) -> tuple[np.ndarray, np.ndarray]

Parameters

Parameter	Type	Default	Description
`weights`	`np.ndarray`	Required	Readout weights `(n_test, n_train)` from `get_decoder_readout`.
`y_train`	`np.ndarray`	Required	Training labels aligned to the weight columns, shape `(n_train,)`.
`classes`	`np.ndarray \| None`	`None`	Class order for the output columns. Defaults to the sorted unique labels of `y_train`.

Returns

Type	Description
`tuple[np.ndarray, np.ndarray]`	`(votes, classes)` where `votes` has shape `(n_test, n_classes)` and each row sums to 1, and `classes` is the class order of its columns.

`decoder_readout.plot_decoder_readout`

View source

Draw the decoder readout for a set of queries over a 2D projection. Each panel places one query and its top_k most-attended training rows on a 2D projection of the rows, drawing a line from the query to each attended row colored by the row’s class and scaled by its vote weight; the query star is colored by the predicted class (argmax of the class votes). The projection uses embeddings (a (train_vecs, test_vecs) pair, e.g. TabPFN’s target-conditioned embeddings from get_embeddings) when given, else UMAP (falling back to PCA) over the raw train_features/test_features. Contrasting the two shows what the head keys on: distance in the embedding space, where votes concentrate on the query’s own class, versus the raw feature space, where that locality is weaker. Works for any number of classes. All test-row arrays (weights, test_features, the test vectors in embeddings, y_test) span the full test set; queries indexes into them to select the rows to draw.

decoder_readout.plot_decoder_readout(
    weights: np.ndarray,
    queries: list[int],
    train_features: np.ndarray,
    test_features: np.ndarray,
    y_train: np.ndarray,
    class_names: list[str],
    *,
    y_test: np.ndarray | None = None,
    embeddings: tuple[np.ndarray, np.ndarray] | None = None,
    query_titles: list[str] | None = None,
    colors: list[str] | None = None,
    title: str = "TabPFN decoder-head readout",
    top_k: int = 20,
) -> Figure

Parameters

Parameter	Type	Default	Description
`weights`	`np.ndarray`	Required	Readout weights `(n_test, n_train)` from `get_decoder_readout`.
`queries`	`list[int]`	Required	Indices into the test set of the rows to draw, one per panel.
`train_features`	`np.ndarray`	Required	Raw training features `(n_train, n_features)`.
`test_features`	`np.ndarray`	Required	Raw test features `(n_test, n_features)`; the queried rows are selected internally.
`y_train`	`np.ndarray`	Required	Training labels aligned to the weight columns, `(n_train,)`.
`class_names`	`list[str]`	Required	Per-class names, aligned to the sorted class labels (same order as `colors`), so `class_names[i]` names the `i`-th class in `sorted(unique(y_train))` rather than the label whose value is `i`. This keeps string and non-0-indexed integer labels working.
`y_test`	`np.ndarray \| None`	`None`	Optional test labels `(n_test,)`; when given, each panel is annotated with the query’s true class.
`embeddings`	`tuple[np.ndarray, np.ndarray] \| None`	`None`	Optional `(train_vecs, test_vecs)` with `test_vecs` spanning the full test set, projected instead of the raw features.
`query_titles`	`list[str] \| None`	`None`	Optional per-panel labels, aligned to `queries`.
`colors`	`list[str] \| None`	`None`	Optional per-class colors, aligned to the sorted class labels. Defaults to a colorblind-validated categorical palette.
`title`	`str`	`"TabPFN decoder-head readout"`	`Figure` title; the projection name is appended.
`top_k`	`int`	`20`	Number of top-voting training rows to draw per query.

Returns

Type	Description
`Figure`	`Figure` with one panel per query, showing its top-voting training rows.

Reference

tabpfn

tabpfn-client

tabpfn-extensions

REST API

`decoder_readout.get_decoder_readout`

`decoder_readout.class_vote`

`decoder_readout.plot_decoder_readout`

​decoder_readout.get_decoder_readout

​decoder_readout.class_vote

​decoder_readout.plot_decoder_readout

`decoder_readout.get_decoder_readout`

`decoder_readout.class_vote`

`decoder_readout.plot_decoder_readout`