> ## Documentation Index
> Fetch the complete documentation index at: https://docs.priorlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# KV cache
> Compute TabPFN's training-set attention state once and reuse it for faster repeated predictions.
TabPFN makes a full forward pass over all the rows when it makes a prediction. Without
cache, every call to `predict()` recomputes attention states for the training rows, even if training set had not changed.
Set `fit_mode="fit_with_cache"` to compute attention state during `fit()` and reuse it during prediction. This makes fitting slower and
uses more memory, but drastically reduces latency when you predict repeatedly against the same training set.
## Quickstart
```bash theme={null}
pip install --upgrade tabpfn-client
```
```python theme={null}
from tabpfn_client import TabPFNClassifier
model = TabPFNClassifier(fit_mode="fit_with_cache")
model.fit(X_train, y_train)
# Both calls reuse the attention state computed during fit.
predictions = model.predict(X_batch_1)
probabilities = model.predict_proba(X_batch_2)
```
```bash theme={null}
pip install --upgrade tabpfn
```
```python theme={null}
from tabpfn import TabPFNClassifier
model = TabPFNClassifier(fit_mode="fit_with_cache")
model.fit(X_train, y_train)
# Both calls reuse the attention state computed during fit.
predictions = model.predict(X_batch_1)
probabilities = model.predict_proba(X_batch_2)
```
`fit_mode="fit_with_cache"` is also available on `TabPFNRegressor`.
## How it works
TabPFN preprocesses the training set and runs the training-side transformer
computation once. The KV cache stores the resulting attention state. Locally, the cache belongs to the fitted
estimator. The managed API stores it using a model ID.
TabPFN reuses the cached keys and values and runs only the test-side
computation. With `tabpfn-client`, the fitted model ID identifies the cache
used for the request.
With the managed API, Prior Labs stores and loads the cache for you. Use the
fitted model ID for later predictions; you do not need to manage cache files or
infrastructure.
## When to use it
| | Without cache | With cache (`fit_with_cache`) |
| -------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| Fit | Preprocesses the training data | Also computes and stores the attention state |
| Predict | Computes the attention state | Reuses the cached attention state |
| Memory | Lower | Higher; scales with training-set size and number of estimators |
| Use when | You predict once per fit or change the training set frequently | You make repeated predictions against the same fit, including repeated calls from explainability tools |
The break-even point depends on training-set size, ensemble size, hardware, and
the number of prediction calls.
## Reuse an API fit in another process
After an API fit, `model_id_` references the server-side fitted state. Store the
ID with your application metadata to use the same fit from another process or
machine.
```python theme={null}
from tabpfn_client import TabPFNClassifier
# Returned by an earlier call to model.fit(...)
cached_model_id = model.model_id_
reused_model = TabPFNClassifier(fit_mode="fit_with_cache")
reused_model.model_id_ = cached_model_id
reused_model.predict(X_new)
```
Use the same estimator type and model configuration as the original fit.
Assigning `model_id_` does not download the model or training data, the model stays on Prior Labs servers.
## Limits and lifecycle
* A cache is valid only for the training data and model configuration used to
create it. Call `fit()` again after either changes.
* The managed API supports KV caching only for TabPFN-3 and later models.
* KV caching is not compatible with [Thinking mode](/capabilities/thinking-mode)
in the managed API.
* Cached API predictions accept up to 10,000 test rows per call. Split larger
test sets across multiple `predict()` calls.
* In the local package, the cache lives with the fitted estimator and consumes
local RAM or VRAM. Releasing the estimator releases the cache.
***
Compare fit and prediction time with and without caching.
Speed up repeated explanation calls against a fixed training set.
Reduce memory pressure and choose an appropriate fit mode.
Set up authentication and make your first API call.