> ## Documentation Index > Fetch the complete documentation index at: https://docs.priorlabs.ai/llms.txt > Use this file to discover all available pages before exploring further. # TabPFN vs XGBoost > Compare TabPFN results with XGBoost using an OpenML Dataset

{/* cookbook-authors:start process_markdown.py */}

By Prior Labs

{/* cookbook-authors:end process_markdown.py */} {/* cookbook-colab:start process_markdown.py */}

Open in Colab

{/* cookbook-colab:end process_markdown.py */}

*A benchmark on a single tabular dataset.* XGBoost is a commonly-used model for tabular data, so the natural question is how TabPFN compares. This notebook runs both on the same dataset and the same split, measuring ROC AUC alongside fit and predict time. TabPFN runs through the hosted API client (`tabpfn-client`), so its timings include the network round-trip to Prior Labs' servers rather than pure local compute. To keep things fair on accuracy, XGBoost gets two chances: a sensible hand-picked configuration and a properly cross-validated one. Every result is collected into a single table at the end. ## Setup *Installing the TabPFN client and XGBoost.* ```python theme={null} !pip install tabpfn-client xgboost scikit-learn ``` ## Imports and Data *Loading the dataset from OpenML and creating a stratified train/test split.* We fetch the German Credit dataset directly from OpenML and split it with stratification so the class balance is preserved in both halves. A single `results` list accumulates each model's score and timing for the final comparison. ```python theme={null} import time import os import pandas as pd import xgboost as xgb from google.colab import userdata from sklearn.datasets import fetch_openml from sklearn.metrics import roc_auc_score from sklearn.model_selection import train_test_split from tabpfn_client import TabPFNClassifier X, y = fetch_openml(data_id=46562, as_frame=True, return_X_y=True) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, stratify=y, random_state=42 ) results = [] ``` ## Authentication *Setting `TABPFN_TOKEN` so the client can reach the API.* The client reads the API key from the `TABPFN_TOKEN` environment variable, we use the `set_access_token` helper. Here we pull it from Colab secrets. ```python theme={null} from tabpfn_client import set_access_token set_access_token(userdata.get('TABPFN_TOKEN')) ``` ## TabPFN, Default Settings *No configuration, no tuning, just fit and predict.* We fit `TabPFNClassifier` from the hosted client with its defaults and time both the fit and the prediction. Because the model runs on Prior Labs' servers, these timings include the round-trip to upload the data and return predictions. The ROC AUC and timings go straight into the results table. ```python theme={null} tabpfn = TabPFNClassifier() t0 = time.perf_counter() tabpfn.fit(X_train, y_train) fit_tabpfn = time.perf_counter() - t0 t0 = time.perf_counter() proba_tabpfn = tabpfn.predict_proba(X_test)[:, 1] predict_tabpfn = time.perf_counter() - t0 results.append( { "Model": "TabPFN v3 (client, default)", "ROC-AUC": f"{roc_auc_score(y_test, proba_tabpfn):.4f}", "Fit time": f"{fit_tabpfn:.2f}s", "Predict time": f"{predict_tabpfn:.2f}s", "Notes": "hosted API", } ) ``` ## XGBoost, Sensible Defaults *A reasonable hand-picked configuration with 100 boosting rounds.* This is the configuration most practitioners reach for as a starting point: a moderate learning rate, depth-six trees, light subsampling, and 100 rounds. It represents XGBoost used with no tuning effort. ```python theme={null} xgb_params = { "learning_rate": 0.05, "max_depth": 6, "min_child_weight": 1, "subsample": 0.9, "colsample_bytree": 0.9, "reg_lambda": 1.0, "tree_method": "hist", "eval_metric": "auc", "objective": "binary:logistic", "seed": 42, } dtrain = xgb.DMatrix(X_train, label=y_train) dtest = xgb.DMatrix(X_test, label=y_test) t0 = time.perf_counter() booster = xgb.train(xgb_params, dtrain, num_boost_round=100) fit_xgb = time.perf_counter() - t0 t0 = time.perf_counter() proba_xgb = booster.predict(dtest) predict_xgb = time.perf_counter() - t0 results.append( { "Model": "XGBoost (sensible defaults)", "ROC-AUC": f"{roc_auc_score(y_test, proba_xgb):.4f}", "Fit time": f"{fit_xgb:.2f}s", "Predict time": f"{predict_xgb:.2f}s", "Notes": "n_estimators=100", } ) ``` ## XGBoost, CV-Tuned *Choosing the number of trees with cross-validation and early stopping.* To give XGBoost a fair shot, we tune the number of boosting rounds using 5-fold cross-validation with early stopping. This mirrors what a careful practitioner would actually ship. ```python theme={null} t0 = time.perf_counter() cv_result = xgb.cv( xgb_params, dtrain, num_boost_round=1000, nfold=5, early_stopping_rounds=20, seed=42, ) best_rounds = len(cv_result) booster_tuned = xgb.train(xgb_params, dtrain, num_boost_round=best_rounds) fit_xgb_tuned = time.perf_counter() - t0 t0 = time.perf_counter() proba_xgb_tuned = booster_tuned.predict(dtest) predict_xgb_tuned = time.perf_counter() - t0 results.append( { "Model": "XGBoost (CV-tuned n_estimators)", "ROC-AUC": f"{roc_auc_score(y_test, proba_xgb_tuned):.4f}", "Fit time": f"{fit_xgb_tuned:.2f}s", "Predict time": f"{predict_xgb_tuned:.2f}s", "Notes": f"5-fold CV with early stopping, best_rounds={best_rounds}", } ) ``` ## Results *Comparing accuracy and speed across all three models.* The table below collects ROC AUC, fit time, and predict time for each model. ROC AUC measures how well a model ranks positive cases above negative ones across every decision threshold: 1.0 is a perfect ranking, 0.5 is no better than a coin flip, so higher is better. On this dataset TabPFN reaches the highest ROC AUC with no tuning at all. XGBoost is faster on data this small, but TabPFN's times here are dominated by the hosted API round-trip rather than compute, and even after cross-validation XGBoost does not close the accuracy gap. ```python theme={null} print(pd.DataFrame(results).to_string(index=False)) ``` ```console theme={null} Model ROC-AUC Fit time Predict time Notes TabPFN v3 (client, default) 0.8281 0.62s 1.23s hosted API XGBoost (sensible defaults) 0.8142 0.51s 0.01s n_estimators=100 XGBoost (CV-tuned n_estimators) 0.8154 4.78s 0.00s 5-fold CV with early stopping, best_rounds=76 ```