TabPFN vs XGBoost

XGBoost uses gradient-boosted decision trees, training a model specifically for each dataset. TabPFN is a pretrained transformer that makes predictions on new datasets without traditional task-specific training.

TabPFN vs XGBoost at a glance

	TabPFN	XGBoost
Approach	Pretrained transformer (TabPFN)	Gradient-boosted trees
Training	No training required	Trains on each dataset
Tuning	Minimal	Often needed
Prediction speed	Under 0.2ms on 1M rows	Typically fast
Best suited for	Best predictive performance & fast deployment	CPU-efficient deployment

Which should you choose?

Choose TabPFN if…

You need stronger predictive performance.
You don’t want to spend time tuning the model.
Time-to-model matters and your data fits the supported row and feature combinations.
Your table includes mixed feature types and you want to minimize manual feature engineering.

Choose XGBoost if…

You need CPU-only deployment.
Your dataset exceeds TabPFN’s supported row and feature combinations.

Predictive Performance Comparison

TabPFN and XGBoost take different approaches to learning from tabular data, but how does that translate into predictive performance? To compare them, we look at results from TabArena, a benchmark evaluating tabular models across classification and regression tasks. Its Elo ratings summarize relative predictive performance across benchmark tasks. Higher Elo indicates stronger relative performance. The results below compare TabPFN-3.5 and TabPFN-3.5-Fast against both default and tuned-and-ensembled XGBoost configurations.

Model	Elo	95% Confidence Interval
TabPFN-3.5, default	1861	+107/-70
TabPFN-3.5-Fast, default	1795	+97/-70
XGBoost,	1371	+40/-49
XGBoost,	1227	+39/-44

Higher Elo is better. Configurations are listed separately where the board reports default and tuned results. Data as of 2026-10-28. Source CSV. Revision: 68988c6e5b31e2ec9a2d193c555dd41b8cb01fd3.

Model	Elo	95% Confidence Interval
TabPFN-3.5, default	2170	+203/-107
TabPFN-3.5-Fast, default	1981	+146/-86
XGBoost,	1448	+53/-52
XGBoost,	1201	+94/-92

On TabArena, TabPFN-3.5 outperforms both default and tuned-and-ensembled XGBoost in classification and regression rankings. Notably, TabPFN achieves these results without dataset-specific hyperparameter tuning, while the tuned XGBoost configuration includes additional optimization and ensembling.

Training and prediction speed

Predictive performance is only part of the comparison. The time required to prepare a model and generate predictions also matters. XGBoost trains decision trees on each dataset. TabPFN uses a pretrained model, avoiding traditional task-specific weight training. This means that achieving stronger performance with XGBoost often involves hyperparameter tuning (testing different model configurations) and ensembling (combining predictions from multiple trained models), both of which can significantly increase training time.

Model	Fit/setup (s/1K)	Predict (s/1K)	Hardware
TabPFN-3.5, default	1.85	0.518	GPU
TabPFN-3.5-Fast, default	0.72	0.168	GPU
XGBoost,	685.87	1.454	CPU
XGBoost,	1.77	0.117	CPU

TabArena median times per 1,000 rows. Fit/setup time for TabPFN is per-dataset setup and context processing rather than weight training. Data as of 2026-10-28. Source CSV. Revision: 68988c6e5b31e2ec9a2d193c555dd41b8cb01fd3.

Model	Fit/setup (s/1K)	Predict (s/1K)	Hardware
TabPFN-3.5, default	1.57	0.376	GPU
TabPFN-3.5-Fast, default	0.65	0.12	GPU
XGBoost,	834.93	2.614	CPU
XGBoost,	2.24	0.242	CPU

Note: These timings reflect the hardware used in the benchmark: TabPFN runs on GPU, while XGBoost runs on CPU. They illustrate the benchmark configurations rather than a hardware-controlled speed comparison.

Native data types

TabPFN-3.5 supports common tabular feature types and includes built-in preprocessing for text and datetime columns, reducing the need for manual feature engineering:

Data type	TabPFN-3.5	XGBoost
Numerical	Native support	Native support
Categorical	Native support	Native support (`enable_categorical=True`)
Missing values	Native support	Native support
Text	Built-in preprocessing	Requires feature engineering
Datetime	Built-in preprocessing	Requires feature engineering

Dataset characteristics

Beyond raw feature types, dataset characteristics such as class imbalance, temporal structure, and distribution shift can affect model performance and the amount of additional configuration required. The table below compares how TabPFN and XGBoost approach these challenges.

Dataset characteristic	TabPFN-3.5	XGBoost
Small datasets	Strong out of the box	Prone to overfitting without careful regularization and tuning
Imbalanced classes	Robust by default, with optional metric tuning (`balance_probabilities`, `eval_metric`)	Requires manual handling (e.g. `scale_pos_weight`, resampling)
Time series / forecasting	Supported via TabPFN-TS and related-series forecasting	Requires feature engineering (lags, rolling windows, calendar features)
Grouped data	Strong results on grouped splits in BeyondArena; see grouped data comparison	Prone to overfitting groups without careful group-aware splits and tuning
Distribution shift	Adapts in-context	Requires retraining and re-tuning on new data

Performance depends on the dataset, evaluation split, and metric. BeyondArena evaluates tabular models across different dataset characteristics, including temporal and grouped splits.

See the difference in action

Compare TabPFN and XGBoost on a real dataset from OpenML.

Frequently asked questions

Is TabPFN better than XGBoost?

On TabArena, TabPFN-3.5 outperforms default and tuned-and-ensembled XGBoost in both classification and regression. However, performance depends on your dataset, so benchmark both models on your own data before choosing.

Does TabPFN require training?

TabPFN is pretrained and does not require traditional task-specific weight training. Its .fit() method prepares the dataset for inference rather than training model weights, unlike XGBoost, which trains decision trees on each dataset.

Can TabPFN replace XGBoost?

For many supervised tabular prediction tasks, TabPFN can be an alternative to XGBoost, particularly when you want strong performance with minimal tuning. However, XGBoost may be preferable for very large datasets, CPU-only environments, or applications requiring specific high-throughput inference.

Is TabPFN faster than XGBoost?

It depends on what you’re measuring. TabPFN avoids traditional model training, which can reduce time to a strong model. XGBoost typically offers efficient prediction after training, particularly on CPUs. Compare end-to-end runtime and prediction latency on your hardware.

How can I compare TabPFN and XGBoost on my own dataset?

Use identical train/test splits, evaluation metrics, and comparable computational budgets. Evaluate both models using validation or cross-validation, then report final performance on an untouched test set. Follow the benchmarking guide or the TabPFN vs XGBoost cookbook.

Getting Started

Capabilities

Using the API

Agentic

Integrations

Troubleshooting

TabPFN in Action

Customer References

Changelog