Credit Card Fraud Detection with TabPFN

ByPhilipp Singer

Fraud detection on the raw dataframe: fit, predict, done. The Credit Card Fraud Detection dataset is one of the most used tables on Kaggle. It holds 284,807 card transactions from two days in September 2013, of which only 492 (0.172%) are fraudulent. The features are 28 anonymized PCA components plus the transaction Time and Amount. The dataset authors recommend the area under the precision-recall curve (AUPRC) as the metric, because accuracy and even ROC AUC say little when one class is this rare. Most public notebooks on this dataset spend their effort on the imbalance: undersampling, SMOTE, class weights, threshold sweeps. TabPFN is a tabular foundation model that predicts in a single forward pass with no per-dataset training, and it is robust to imbalance out of the box. This notebook hands it the raw dataframe through the hosted API (tabpfn-client), so no GPU is needed, and compares the result with the reference models from the most popular Kaggle notebooks, trained on exactly the same columns with no feature engineering, so the comparison stays fair. Summary:

One fit, one predict, no imbalance tricks. TabPFN reaches an AUPRC of about 0.90 and a ROC AUC of about 0.99 on a held-out 20% split, straight from the 30 raw columns.
Ahead of the Kaggle reference models on the same columns. On the same split, the XGBoost configuration from the most popular modeling notebook reaches 0.84 AUPRC, a 100-tree random forest 0.87, and XGBoost with library defaults 0.78.
The ranking holds across folds. In a 5-fold stratified cross-validation TabPFN wins every fold, with a mean AUPRC of 0.89 against 0.85 for the random forest and 0.81 for the reference XGBoost, so the conclusion does not rest on a single set of 98 test frauds.

Setup

Installing the TabPFN client, the Kaggle download helper, and the reference models, pinned to the versions this notebook was run with.

!pip install tabpfn-client==0.6.0 kagglehub==1.0.2 xgboost==3.4.1 scikit-learn==1.9.0 pandas==2.3.3 matplotlib==3.11.2

Load the data

Downloading the dataset from Kaggle and taking a first look. kagglehub loads public datasets straight into a dataframe without a Kaggle account. The table has no missing values, and all 30 feature columns are numeric. We drop nothing and engineer nothing: Time (seconds since the first transaction), Amount, and V1 to V28 go into every model exactly as they arrive.

import warnings

import kagglehub
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from google.colab import userdata
from kagglehub import KaggleDatasetAdapter
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import average_precision_score, precision_recall_curve, roc_auc_score
from sklearn.model_selection import StratifiedKFold, train_test_split
from xgboost import XGBClassifier

# tabpfn-client warns that the full-output criterion needs the local tabpfn package; not needed here.
warnings.filterwarnings("ignore", message="Optional dependencies 'tabpfn' and 'torch'")

RANDOM_STATE = 42

df = kagglehub.dataset_load(KaggleDatasetAdapter.PANDAS, "mlg-ulb/creditcardfraud", "creditcard.csv")

X = df.drop(columns="Class")
y = df["Class"]
print(f"{len(df):,} transactions | {X.shape[1]} features | "
      f"{int(y.sum()):,} frauds ({100 * y.mean():.3f}%)")
df.head()

284,807 transactions | 30 features | 492 frauds (0.173%)
   Time        V1        V2        V3        V4        V5        V6        V7  \
 0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599
 0.0  1.191857  0.266151  0.166480  0.448154  0.060018 -0.082361 -0.078803
 1.0 -1.358354 -1.340163  1.773209  0.379780 -0.503198  1.800499  0.791461
 1.0 -0.966272 -0.185226  1.792993 -0.863291 -0.010309  1.247203  0.237609
 2.0 -1.158233  0.877737  1.548718  0.403034 -0.407193  0.095921  0.592941

         V8        V9  ...       V21       V22       V23       V24       V25  \
0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539
0.085102 -0.255425  ... -0.225775 -0.638672  0.101288 -0.339846  0.167170
0.247676 -1.514654  ...  0.247998  0.771679  0.909412 -0.689281 -0.327642
0.377436 -1.387024  ... -0.108300  0.005274 -0.190321 -1.175575  0.647376
-0.270533  0.817739  ... -0.009431  0.798278 -0.137458  0.141267 -0.206010

        V26       V27       V28  Amount  Class
-0.189115  0.133558 -0.021053  149.62      0
0.125895 -0.008983  0.014724    2.69      0
-0.139097 -0.055353 -0.059752  378.66      0
-0.221929  0.062723  0.061458  123.50      0
0.502292  0.219422  0.215153   69.99      0

[5 rows x 31 columns]

Train/test split

A stratified 80/20 split, the setup used by the Kaggle notebooks we compare against. Stratification keeps the fraud rate identical in both halves, which leaves 98 frauds in the test set. The public notebooks split at random, and we follow them so the reference numbers are comparable. A production fraud model would be validated out of time (train on the first day, test on the second) to catch drift; the cross-validation at the end of this notebook is the more robust check within this protocol.

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=RANDOM_STATE
)
print(f"train {len(X_train):,} rows / {int(y_train.sum())} frauds | "
      f"test {len(X_test):,} rows / {int(y_test.sum())} frauds")

train 227,845 rows / 394 frauds | test 56,962 rows / 98 frauds

Metrics

AUPRC as the headline, ROC AUC for reference, and one operational number.

AUPRC (average precision) summarizes precision against recall across all thresholds. A model with no information scores the fraud rate, here 0.0017, so every gain is visible. This is the metric the dataset authors recommend.
ROC AUC measures how well frauds rank above legitimate transactions. It is the most reported number on Kaggle, but on data this imbalanced it saturates near 1 and hides real differences.
Alerts to catch 90% of frauds: sort the 56,962 test transactions by score and count how many an analyst team has to review before 90% of the 98 frauds are found. This is the workload behind a recall target, and it is where models that look alike on the first metrics come apart.

Every model records its scores into one results list for the final table.

results = []


def alerts_for_recall(proba, target=0.9):
    """Number of highest-scored transactions to review before `target` of the frauds are found."""
    frauds_in_order = y_test.to_numpy()[np.argsort(-proba)]
    needed = int(np.ceil(target * y_test.sum()))
    return int(np.argmax(np.cumsum(frauds_in_order) >= needed)) + 1


def evaluate(name, proba, notes=""):
    alerts = alerts_for_recall(proba)
    results.append(
        {
            "Model": name,
            "AUPRC": average_precision_score(y_test, proba),
            "ROC AUC": roc_auc_score(y_test, proba),
            "Alerts for 90% recall": alerts,
            "Notes": notes,
        }
    )
    print(f"{name}: AUPRC {results[-1]['AUPRC']:.4f} | ROC AUC {results[-1]['ROC AUC']:.4f} | "
          f"{alerts:,} alerts to catch 90% of frauds")

Authentication

Setting the API token so the client can reach the hosted model. The client reads the API key through set_access_token. Here we pull it from Colab secrets; outside Colab, set the TABPFN_TOKEN environment variable. Get a token at platform.priorlabs.ai. We pin the model to TabPFN 3.5 with create_default_for_version, so the numbers in this notebook stay reproducible after future releases. Use TabPFNClassifier() instead to always get the current default model.

from tabpfn_client import TabPFNClassifier, set_access_token
from tabpfn_client.api_models import ModelVersion

set_access_token(userdata.get("TABPFN_TOKEN"))


def make_tabpfn():
    return TabPFNClassifier.create_default_for_version(ModelVersion.V3_5)

TabPFN

Default settings, the raw dataframe, no resampling. fit uploads the training table and predict_proba runs the in-context learning on Prior Labs’ servers. There are no hyperparameters to set and no class weights to pick.

tabpfn = make_tabpfn()
tabpfn.fit(X_train, y_train)
proba_tabpfn = tabpfn.predict_proba(X_test)[:, 1]

evaluate("TabPFN 3.5 (client, default)", proba_tabpfn, "hosted API")

TabPFN 3.5 (client, default): AUPRC 0.9014 | ROC AUC 0.9902 | 135 alerts to catch 90% of frauds

Reference models from Kaggle

The same 30 columns, the model configurations the community converged on. We take the reference models from public notebooks on this dataset, keeping their configurations and dropping any resampling or feature engineering:

XGBoost, Kaggle configuration. Gabriel Preda’s Credit Card Fraud Detection Predictive Models is the most upvoted modeling notebook on the dataset that trains on the raw columns. Its XGBoost uses shallow trees (max_depth=2), a learning rate of 0.039, row and column subsampling, and up to 1,000 boosting rounds with early stopping on a validation slice. We reproduce that recipe, carving the validation slice out of our training set so the test set stays untouched.
XGBoost, library defaults. Several notebooks, for example ANNs vs XGBoost, simply call XGBClassifier().
Random forest. The 100-tree RandomForestClassifier that opens Preda’s notebook, and a common first model in many others.

def fit_xgb_reference(X_fit, y_fit):
    """XGBoost as configured in the Kaggle reference notebook, early-stopped on a validation slice."""
    X_inner, X_valid, y_inner, y_valid = train_test_split(
        X_fit, y_fit, test_size=0.2, stratify=y_fit, random_state=RANDOM_STATE
    )
    model = XGBClassifier(
        n_estimators=1000,
        learning_rate=0.039,
        max_depth=2,
        subsample=0.8,
        colsample_bytree=0.9,
        eval_metric="auc",
        early_stopping_rounds=50,
        random_state=RANDOM_STATE,
        n_jobs=-1,
    )
    model.fit(X_inner, y_inner, eval_set=[(X_valid, y_valid)], verbose=False)
    return model


xgb_ref = fit_xgb_reference(X_train, y_train)
proba_xgb_ref = xgb_ref.predict_proba(X_test)[:, 1]
evaluate("XGBoost (Kaggle configuration)", proba_xgb_ref,
         f"max_depth=2, early stopping, best_iteration={xgb_ref.best_iteration}")

xgb_default = XGBClassifier(random_state=RANDOM_STATE, n_jobs=-1).fit(X_train, y_train)
proba_xgb_default = xgb_default.predict_proba(X_test)[:, 1]
evaluate("XGBoost (library defaults)", proba_xgb_default, "XGBClassifier()")

rf = RandomForestClassifier(n_estimators=100, random_state=RANDOM_STATE, n_jobs=-1).fit(X_train, y_train)
proba_rf = rf.predict_proba(X_test)[:, 1]
evaluate("Random forest (100 trees)", proba_rf, "n_estimators=100")

XGBoost (Kaggle configuration): AUPRC 0.8382 | ROC AUC 0.9719 | 647 alerts to catch 90% of frauds
XGBoost (library defaults): AUPRC 0.7813 | ROC AUC 0.9219 | 5,389 alerts to catch 90% of frauds
Random forest (100 trees): AUPRC 0.8734 | ROC AUC 0.9630 | 192 alerts to catch 90% of frauds

Results

One table, four models, the same 30 columns. TabPFN leads on both ranking metrics. Every ROC AUC sits above 0.92 and the top three are within three points of each other, which is exactly why the dataset authors ask for AUPRC: there the spread between the same models is twelve points. The workload column makes the difference concrete: to catch 90% of the frauds, analysts review about 135 alerts with TabPFN, roughly 190 with the random forest, and more than 600 with the reference XGBoost, with no threshold tuning for any model.

table = pd.DataFrame(results)
table["AUPRC"] = table["AUPRC"].map("{:.4f}".format)
table["ROC AUC"] = table["ROC AUC"].map("{:.4f}".format)
print(table.to_string(index=False))

                         Model  AUPRC ROC AUC  Alerts for 90% recall                                           Notes
  TabPFN 3.5 (client, default) 0.9014  0.9902                    135                                      hosted API
XGBoost (Kaggle configuration) 0.8382  0.9719                    647 max_depth=2, early stopping, best_iteration=110
    XGBoost (library defaults) 0.7813  0.9219                   5389                                 XGBClassifier()
     Random forest (100 trees) 0.8734  0.9630                    192                                n_estimators=100

Precision-recall curves

Where the AUPRC gap comes from. The curves show precision (how many flagged transactions are fraud) against recall (how many frauds are caught). TabPFN holds higher precision as recall increases, which is the regime a fraud team operates in: catch most of the fraud without drowning analysts in false alarms. The dashed line is the fraud rate, the precision of flagging at random.

curves = [
    ("TabPFN 3.5", proba_tabpfn, "#c0504d"),
    ("XGBoost (Kaggle configuration)", proba_xgb_ref, "#4f81bd"),
    ("Random forest (100 trees)", proba_rf, "#9bbb59"),
    ("XGBoost (library defaults)", proba_xgb_default, "#9aa7b0"),
]

fig, ax = plt.subplots(figsize=(7, 5))
for name, proba, color in curves:
    precision, recall, _ = precision_recall_curve(y_test, proba)
    ax.plot(recall, precision, color=color, lw=2,
            label=f"{name} (AUPRC {average_precision_score(y_test, proba):.3f})")
ax.axhline(y_test.mean(), ls="--", color="grey", lw=1, label=f"random (fraud rate {y_test.mean():.4f})")
ax.set(xlabel="recall (share of frauds caught)", ylabel="precision (share of flags that are fraud)",
       xlim=(0, 1), ylim=(0, 1.02), title="Precision-recall on the held-out 20%")
ax.grid(ls=":", alpha=0.4)
ax.spines["top"].set_visible(False)
ax.spines["right"].set_visible(False)
ax.legend(frameon=False, loc="lower left")
plt.tight_layout()
plt.show()

Is one split enough?

Repeating the comparison with 5-fold stratified cross-validation. A test set with 98 frauds is small, and the AUPRC of any model moves by a few points from split to split. To check that the ranking is not an artifact of one split, we run all models on five stratified folds. Each model is fit on 80% of the data and scored on the remaining 20%, so every transaction is a test row exactly once. Same columns, same configurations, no tuning on the folds.

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=RANDOM_STATE)
cv_rows = []
for fold, (train_idx, test_idx) in enumerate(skf.split(X, y), start=1):
    X_tr, X_te = X.iloc[train_idx], X.iloc[test_idx]
    y_tr, y_te = y.iloc[train_idx], y.iloc[test_idx]
    row = {"Fold": fold, "Test frauds": int(y_te.sum())}
    row["TabPFN 3.5"] = average_precision_score(
        y_te, make_tabpfn().fit(X_tr, y_tr).predict_proba(X_te)[:, 1])
    row["XGBoost (Kaggle configuration)"] = average_precision_score(
        y_te, fit_xgb_reference(X_tr, y_tr).predict_proba(X_te)[:, 1])
    row["XGBoost (library defaults)"] = average_precision_score(
        y_te, XGBClassifier(random_state=RANDOM_STATE, n_jobs=-1).fit(X_tr, y_tr).predict_proba(X_te)[:, 1])
    row["Random forest (100 trees)"] = average_precision_score(
        y_te, RandomForestClassifier(n_estimators=100, random_state=RANDOM_STATE, n_jobs=-1)
        .fit(X_tr, y_tr).predict_proba(X_te)[:, 1])
    cv_rows.append(row)
    print(f"fold {fold}: " + " | ".join(f"{k} {v:.4f}" for k, v in row.items() if k not in ("Fold", "Test frauds")))

cv = pd.DataFrame(cv_rows).set_index("Fold")
model_cols = [c for c in cv.columns if c != "Test frauds"]
summary = pd.DataFrame({"mean AUPRC": cv[model_cols].mean(), "std": cv[model_cols].std(),
                        "folds won": (cv[model_cols].T == cv[model_cols].max(axis=1)).sum(axis=1)})
print()
print(summary.sort_values("mean AUPRC", ascending=False).to_string(float_format="{:.4f}".format))

fold 1: TabPFN 3.5 0.8663 | XGBoost (Kaggle configuration) 0.7898 | XGBoost (library defaults) 0.7572 | Random forest (100 trees) 0.8146
fold 2: TabPFN 3.5 0.9175 | XGBoost (Kaggle configuration) 0.8556 | XGBoost (library defaults) 0.8601 | Random forest (100 trees) 0.8856
fold 3: TabPFN 3.5 0.9109 | XGBoost (Kaggle configuration) 0.8160 | XGBoost (library defaults) 0.8211 | Random forest (100 trees) 0.8589
fold 4: TabPFN 3.5 0.8964 | XGBoost (Kaggle configuration) 0.8362 | XGBoost (library defaults) 0.8190 | Random forest (100 trees) 0.8450
fold 5: TabPFN 3.5 0.8642 | XGBoost (Kaggle configuration) 0.7575 | XGBoost (library defaults) 0.7553 | Random forest (100 trees) 0.8223

                                mean AUPRC    std  folds won
TabPFN 3.5                          0.8911 0.0248          5
Random forest (100 trees)           0.8453 0.0286          0
XGBoost (Kaggle configuration)      0.8110 0.0386          0
XGBoost (library defaults)          0.8025 0.0453          0

Takeaways

No imbalance handling needed. TabPFN sees 394 frauds among 227,845 rows and ranks them well without undersampling, SMOTE, or class weights. The default probabilities can be thresholded directly for a review queue, and that queue is the shortest of the four models for the same recall.
No feature engineering, no tuning. The reference models were configured by experienced Kaggle authors; TabPFN was not configured at all and still comes out ahead on AUPRC across the folds.
Fair comparison, real caveats. All models saw the same 30 columns and a random stratified split. A production evaluation would split by time and would likely add merchant, card, and velocity features that this anonymized dataset cannot provide. For grouped or temporal fraud data, the Thinking mode of the client is a natural next step.

​Setup

​Load the data

​Train/test split

​Metrics

​Authentication

​TabPFN

​Reference models from Kaggle

​Results

​Precision-recall curves

​Is one split enough?

​Takeaways