Ekdeep Singh Lubana

I am a member of technical staff at Goodfire AI. I am generally interested in designing (faithful) abstractions of phenomena relevant to controlling or aligning neural networks and in better understanding training dynamics.

Before this, I was a Postdoctoral Research Fellow at the CBS-NTT Program in Physics of Intelligence at Harvard University, where I worked with Hidenori Tanaka and Demba Ba. I did my PhD co-affiliated with EECS, University of Michigan and CBS, Harvard, and was advised by Robert Dick and Hidenori Tanaka. I graduated with a Bachelor's degree in ECE from Indian Institute of Technology (IIT), Roorkee in 2019. My research in undergraduate was primarily focused on embedded systems, such as energy-efficient machine vision systems.

Email  /  CV  /  Google Scholar

We argue a broad set of persistent LLM failures -- sycophancy, logical inconsistency, self-contradiction -- are best understood as failures of self-consistency, and that many existing fixes are instances of one common consistency optimization procedure.

Concepts are often realized as geometric structures spanning low-dimensional regions, rather than isolated directions. We design block-sparse featurizers to capture such concept manifolds, validating across InceptionV1, DINOv3, and SDXL.

We use interpretability to characterize what preference data actually teaches a model, surfacing latent concepts that separate preferred from dispreferred responses. Feature and data interventions then let us suppress problematic signals and amplify desirable ones, e.g., safety and personality traits.

We cast model scale as a competition over resources: larger models suffer less gradient interference, hence can simultaneously allocate capacity to frequent and rare tasks. We validate this account in synthetic settings and in OLMo pretraining.

We propose LLMs update their beliefs in-context by moving along trajectories in a low-dimensional geometric space, showing these updates form structured manifolds that linear probes decode and that interventions steer in predictable ways.

Steering along a manifold fit to representations yields behavioral trajectories that follow the corresponding manifold in output space, while linear steering produces unnatural outputs -- suggesting representational geometry is causally tied to behavior.

Llama-3.1-8B solves cyclic reasoning tasks using task-agnostic Fourier features and a sparse set of reused neurons that compute base-10 sums, rather than modular arithmetic aligned to each concept's period.

SAEs capture concept manifolds either by allocating a global subspace or by tiling them with local features, and do so suboptimally -- arguing future interpretability methods should treat geometric objects, not isolated directions, as the unit of analysis.

We turn interpretable model features into reward functions (RLFR), making Gemma-3-12B-IT substantially less prone to hallucination while retaining performance on standard benchmarks.

Token-level uncertainty during chain-of-thought predicts how easily a model can be steered, and hidden activations predict its future outcome distribution -- implying models implicitly represent the paths they did not take.

Through a Bayesian lens, SAEs impose a prior that concepts are independent across time, i.e., stationary. Language model representations violate this badly -- dimensionality grows, correlations are context-dependent, and statistics are non-stationary. We thus propose Temporal Feature Analysis, an objective with a temporal inductive bias that splits a representation into predictable and novel components.

A unified Bayesian account of prompting and activation steering: steering operates by changing concept priors, while in-context learning accumulates evidence. This yields predictive models of behavioral shifts and of how the two interventions interact.

A large-scale dictionary learning analysis of DINOv2 (32k+ concepts) motivating the Minkowski Representation Hypothesis: tokens are formed as sums of convex mixtures of archetypes, grounded both in conceptual spaces and in multi-head attention.

Probabilistic dependencies between emotional states in model outputs recover hierarchical emotion trees that align with psychological emotion wheels, with larger models forming richer hierarchies. We also find systematic recognition biases across socioeconomic personas.

We model ICL in a hierarchical Bayesian framework as a posterior-weighted average of two well-defined algorithmic strategies. Taking a rational analysis lens, we show across three popular setttings, that we can analytically predict model behavior and characterize the algorithmic phase-diagram identified in ICL in our prior work.

We build on results from identifiability theory to formalize a measure that assesses fine-grained differences in a natural distribution and a generative model trained upon it.

Probes trained on synthetic data to flag "high-stakes" interactions generalize robustly to out-of-distribution, real-world data, matching prompted or finetuned LLM monitors at six orders of magnitude less compute by reusing the monitored model's activations.

We use recent phenomenology of neural network representations, e.g., their ability to encode hierarchical and nonlinear concepts, to argue for the limitations of Linear representation hypothesis, contextualizing SAEs with respect to such concepts.

We demonstrate a duality between how concepts are organized in model representations and the optimal SAE that will help identify them. This implies there exists no universally optimal SAE that can be used across models and domains.

We use vision models to demonstrate algorithmic instability in SAEs, i.e., different runs of the same pipeline yield different features / model interpretations. We propose a geometrical constraint to substantially address this issue.

We demonstrate models can in-context infer novel semantics of a concept, overriding their original meaning based on pretraining. This process occurs relatively rapidly, yielding an emergent reorganization of individual concepts' representations.

We demonstrate an algorithmic phase diagram that decomposes ICL into a mixture of algorithms, showing that phenomenology of ICL may be specific to setups used for experimentation.

We use Formal languages to analyze the limitations of SAEs, finding, similar to prior work in disentangled representation learning, that SAEs find correlational features; explicit biasing is necessary to induce causality.

We implicate rapid acquisition of structures underlying the data generating process as the source of sudden learning of capabilities, and analogize knowledge centric capabilities to the process of graph percolation that undergoes a formal second-order phase transition.

We create a theoretical abstraction of our prior work on compositional generalization and justify the peculiar learning dynamics observed therein, finding there was in fact a quadruple descent embedded therein!

We instantiate a synthetic knowledge graph domain to study how model editing protocols harm broader capabilities, demonstrating all representational organization of different concepts is destroyed under counterfactul edits.

We analyze a model's learning dynamics in "concept space" and identify sudden transitions where the model, when latently intervened, demonstrates a capability, even if input prompting does not show said capability.

We use formal languages as a model system to identify the mechanistic changes induced by safety fine-tuning, and how jailbreaks bypass said mechanisms, verifying our claims on Llama models.

We show the acquisition of structures underlying a data-generating process is the driving cause for abrupt learning in Transformers.

We identify and discuss 18 foundational challenges in assuring the alignment and safety of large language models (LLMs) and pose 200+ concrete research questions.

We formalize and define a notion of composition of primitive capabilities learned via autoregressive modeling by a Transformer, showing the model's capabilities can "explode", i.e., combinatorially increase if it can compose.

We cast stepwise inference methods in LLMs as a graph navigation task, finding a synthetic model is sufficient to explain and identify novel characteristics of such methods.

We show fine-tuning leads to learning of minimal transformations of a pretrained model's capabilities, like a "wrapper", by using procedural tasks defined using Tracr, PCFGs, and TinyStories.

We analyze different LLMs' abilities to model binary sequences generated via different pseduo-random processes, such as a formal automaton, and find that with scale, LLMs are (almost) able to simulate these processes via mere context conditioning.

We propose and analyze a pipeline for re-casting an LLM as a generic reward function that interacts with an LVM to enable embodied AI tasks.

We analyze compositionality in diffusion models, showing that there is a sudden emergence of this capability if models are allowed sufficient training to learn the relevant primitive capabilities.

We show models that rely on entirely different mechanisms for making their predictions can exhibit mode connectivity, but generally the ones that are mechanistically similar are linearly connected.

We present a highly detailed analysis of the landscape of several self-supervised learning objectives to clarify the role of representational collapse.

We propose a theoretical framework that demonstrates limitations of popular graph augmentation strategies for self-supervised learning.

We propose an unsupervised learning method that exploits client heterogeneity to enable privacy preserving, SOTA performance unsupervised federated learning.

We develop a general theory to understand the role of normalization layers in improving training dynamics of a neural network at initialization.

This work demonstrates how quadratic regularization methods for preventing catastrophic forgetting in deep networks rely on a simple heuristic under-the-hood: Interpolation.

A unified, theoretically-grounded framework for network pruning that helps justify often used heuristics in the field.

An image signal processing pipeline that allows use of out-of-the-box deep neural networks on RAW images directly retrieved from image sensors.

An energy-efficient machine vision framework inspired by the concept of Fovea in biological vision. Also see follow-up work presented at CVPR workshop, 2020.

An efficient imaging system that accurately calculates chlorophyll content in leaves by using RAW images.

Website template source available here.