Understanding Transformers
Foundation Models in Robotics
Decision Transformer: Reinforcement Learning via Sequence Modeling
arxiv.org →GAIA-1: A Generative World Model for Autonomous Driving
arxiv.org →Offline Reinforcement Learning as One Big Sequence Modeling Problem
arxiv.org →Humanoid Locomotion as Next Token Prediction
arxiv.org →Behavior Transformers: Cloning k modes with one stone
arxiv.org →
Foundation Models in Finance
Foundation Models in Consumer Behaviour
Recommender Systems with Generative Retrieval
arxiv.org →OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment
arxiv.org →UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces
arxiv.org →Transferable and Forecastable User Targeting Foundation Model
arxiv.org →
Foundation Models in (Behavioural) Biology
The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics
biorxiv.org →TweetyBERT: Automated parsing of birdsong through self-supervised machine learning
biorxiv.org →scGPT: Towards Building a Foundation Model for Single-Cell Multi-omics Using Generative AI
biorxiv.org →
Foundation Models in Neuroscience
BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals
arxiv.org →POCO: Scalable Neural Forecasting through Population Conditioning
arxiv.org →Generalizable, real-time neural decoding with hybrid state-space models
arxiv.org →Decoding Dynamic Visual Experience from Calcium Imaging via Cell-Pattern-Aware SSL
arxiv.org →