Documentation Index
Fetch the complete documentation index at: https://docs.priorlabs.ai/llms.txt
Use this file to discover all available pages before exploring further.
When your dataset has many features (especially beyond 500), feature selection can improve both performance and speed.
Why It Helps
TabPFN uses transformer attention over all features. Irrelevant or noisy features dilute the model’s attention budget and can reduce predictive power, especially as feature count grows.
Approaches
Greedy feature selection - remove features individually and check performance. This works particularly well on smaller data with low computational costs.
Mutual information filtering — rank features by mutual information with the target and keep the top-k:
from sklearn.feature_selection import mutual_info_classif, SelectKBest
selector = SelectKBest(mutual_info_classif, k=50)
X_train_selected = selector.fit_transform(X_train, y_train)
X_test_selected = selector.transform(X_test)
from sklearn.decomposition import PCA
pca = PCA(n_components=50)
X_train_reduced = pca.fit_transform(X_train)
X_test_reduced = pca.transform(X_test)
