A survey on self-supervised learning for non-sequential tabular data

Real-world analytics hinges on tabular data, yet prevailing learners face a triple bind: tree ensembles excel on fixed schemas but cannot generalize to new columns, neural nets learn rich features yet overfit small tables, and recent transfer approaches falter when schemas diverge. We tackle these limitations with GTab, a Gradient-Boosting Bipartite Graph Neural Network that marries decision-tree residual refinement with self-supervised graph representation learning. GTab maps each table to an instance-feature bipartite graph, where a GNN, optimised jointly with contrastive, clustering, and reconstruction objectives, captures feature-feature, instance-instance, and cross-type relations. Boosted trees ingest the GNN's gradients, correcting residual errors and injecting the strong inductive bias of split-based models. Building on this backbone, we introduce three variants: E-GTab ensembles multiple overlapping feature sub-graphs for robust classic prediction; I-GTab inductively attaches unseen feature nodes, enabling feature-incremental inference without retraining; and T-GTab pre-trains on a source schema and lightly fine-tunes on a target schema to achieve zero-shot and transfer learning across heterogeneous tables. Across 20 public benchmarks and five clinical trials, GTab consistently ranks first: it outperforms tree, neural, and graph baselines on static tasks, surpasses prior art (and an oracle) when half the test-time columns are unseen, and delivers higher AUC than the leading transformer baseline in both cross-dataset and zero-shot transfer – all with a unified architecture. GTab thus offers a principled, scalable, and adaptable solution to holistic tabular data prediction, bridging the gap between classic ensembles and modern self-supervised representation learning.

Triple Mixup is an augmentation policy in the hidden latent space we introduced in the Contrastive Mixup Self-Semi Supervised learning framework, to address the imbalanced data problem, for Cardiovascular Heart Diseases tabular dataset. Medical tabular datasets are known to present challenges as high imbalanced class, limited annotated quality samples due to the domain nature. Recent literature in Self and Semi supervised learning, has shown tremendous progress in learning useful representations, and leveraging unlabeled dataset and labeled dataset to train a learning model. Most existing methods are not feasible for tabular data due to the data augmentation scheme. In addition, the high imbalanced problem can show lower performance on machine learning algorithms. For this work, we propose the triple data augmentation method in hidden space to attack the unbalanced challenge in self-supervised and semi-supervised learning, from the possible applications of Contrastive Mixup, thus we will study the influence of it.