KV cache

TabPFN makes a full forward pass over all the rows when it makes a prediction. Without cache, every call to predict() recomputes attention states for the training rows, even if training set had not changed. Set fit_mode="fit_with_cache" to compute attention state during fit() and reuse it during prediction. This makes fitting slower and uses more memory, but drastically reduces latency when you predict repeatedly against the same training set.

Quickstart

TabPFN API
Local package

pip install --upgrade tabpfn-client

from tabpfn_client import TabPFNClassifier

model = TabPFNClassifier(fit_mode="fit_with_cache")
model.fit(X_train, y_train)

# Both calls reuse the attention state computed during fit.
predictions = model.predict(X_batch_1)
probabilities = model.predict_proba(X_batch_2)

pip install --upgrade tabpfn

from tabpfn import TabPFNClassifier

model = TabPFNClassifier(fit_mode="fit_with_cache")
model.fit(X_train, y_train)

# Both calls reuse the attention state computed during fit.
predictions = model.predict(X_batch_1)
probabilities = model.predict_proba(X_batch_2)

fit_mode="fit_with_cache" is also available on TabPFNRegressor.

How it works

Fit

TabPFN preprocesses the training set and runs the training-side transformer computation once. The KV cache stores the resulting attention state. Locally, the cache belongs to the fitted estimator. The managed API stores it using a model ID.

Predict

TabPFN reuses the cached keys and values and runs only the test-side computation. With tabpfn-client, the fitted model ID identifies the cache used for the request.

With the managed API, Prior Labs stores and loads the cache for you. Use the fitted model ID for later predictions; you do not need to manage cache files or infrastructure.

When to use it

	Without cache	With cache (`fit_with_cache`)
Fit	Preprocesses the training data	Also computes and stores the attention state
Predict	Computes the attention state	Reuses the cached attention state
Memory	Lower	Higher; scales with training-set size and number of estimators
Use when	You predict once per fit or change the training set frequently	You make repeated predictions against the same fit, including repeated calls from explainability tools

The break-even point depends on training-set size, ensemble size, hardware, and the number of prediction calls.

Reuse an API fit in another process

After an API fit, model_id_ references the server-side fitted state. Store the ID with your application metadata to use the same fit from another process or machine.

from tabpfn_client import TabPFNClassifier

# Returned by an earlier call to model.fit(...)
cached_model_id = model.model_id_

reused_model = TabPFNClassifier(fit_mode="fit_with_cache")
reused_model.model_id_ = cached_model_id
reused_model.predict(X_new)

Use the same estimator type and model configuration as the original fit. Assigning model_id_ does not download the model or training data, the model stays on Prior Labs servers.

Limits and lifecycle

A cache is valid only for the training data and model configuration used to create it. Call fit() again after either changes.
The managed API supports KV caching only for TabPFN-3 and later models.
KV caching is not compatible with Thinking mode in the managed API.
Cached API predictions accept up to 10,000 test rows per call. Split larger test sets across multiple predict() calls.
In the local package, the cache lives with the fitted estimator and consumes local RAM or VRAM. Releasing the estimator releases the cache.

Benchmark KV cache

Compare fit and prediction time with and without caching.

Interpretability

Speed up repeated explanation calls against a fixed training set.

Out-of-memory errors

Reduce memory pressure and choose an appropriate fit mode.

API getting started

Set up authentication and make your first API call.

Getting Started

API

Capabilities

Agentic

Integrations

Troubleshooting

Use Cases

Changelog

Quickstart

How it works

When to use it

Reuse an API fit in another process

Limits and lifecycle

Benchmark KV cache

Interpretability

Out-of-memory errors

API getting started

​Quickstart

​How it works

​When to use it

​Reuse an API fit in another process

​Limits and lifecycle

Benchmark KV cache

Interpretability

Out-of-memory errors

API getting started

Quickstart

How it works

When to use it

Reuse an API fit in another process

Limits and lifecycle