Ekdeep Singh Lubana

I am a PhD candidate in EECS Department at the University of Michigan, advised by Robert Dick. I am also affiliated with Harvard Center for Brain Science, where I am mentored by Hidenori Tanaka.

I am generally interested in designing theoretically motivated, grounded algorithms for practical applications of DNNs. I am also very interested in better understanding training dynamics of neural networks, especially via a statistical physics perspective.

I graduated with a Bachelor's degree in ECE from Indian Institute of Technology (IIT), Roorkee in 2019. My research in undergraduate was primarily focused on embedded systems, such as energy-efficient machine vision systems.

Email  /  CV  /  Google Scholar  /  Github

profile photo
[04/2023] Our work on a mechanistic understanding of loss landscapes was accepted to ICML, 2023.
[01/2023] Our work analyzing loss landscape of self-supervised objectives was accepted to ICLR, 2023.
[09/2022] Our work on data-centric analysis of graph contrastive learning was accepted to NeurIPS, 2022.
[05/2022] Our work on unsupervised federated learning was accepted as a spotlight at ICML, 2022!
[05/2022] Joining Harvard Center for Brain Science as a research affiliate!
[10/2021] Our work on dynamics of normalization layers was accepted to NeurIPS, 2021.
[03/2021] Our work on theory of pruning was accepted as a spotlight at ICLR, 2021.
ssl_landscape Mechanistic Mode Connectivity
Ekdeep Singh Lubana, Eric J. Bigelow, Robert P. Dick, David Krueger, and Hidenori Tanaka
International Conference on Machine Learning (ICML), 2023
bibtex / arXiv / github

We show models that rely on entirely different mechanisms for making their predictions can exhibit mode connectivity, but generally the ones that are mechanistically similar are linearly connected.

ssl_landscape What Shapes the Landscape of Self-Supervised Learning?
Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, and Hidenori Tanaka
International Conference on Learning Representations (ICLR), 2023
bibtex / arXiv

We present a highly detailed analysis of the landscape of several self-supervised learning objectives to clarify the role of representational collapse.

GraphSSL Analyzing Data-Centric Properties for Contrastive Learning on Graphs
Puja Trivedi, Ekdeep Singh Lubana, Mark Heimann, Danai Koutra, and Jay Jayaraman Thiagarajan
Advances in Neural Information Processing Systems (NeurIPS), 2022
bibtex / arXiv / github

We propose a theoretical framework that demonstrates limitations of popular graph augmentation strategies for self-supervised learning.

Orchestra Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering
Ekdeep Singh Lubana, Chi Ian Tang, Fahim Kawsar, Robert P. Dick, and Akhil Mathur
International Conference on Machine Learning (ICML), 2022 (Spotlight)
bibtex / arXiv / github / video

We propose an unsupervised learning method that exploits client heterogeneity to enable privacy preserving, SOTA performance unsupervised federated learning.

beyondbn Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
Ekdeep Singh Lubana, Hidenori Tanaka, and Robert P. Dick
Advances in Neural Information Processing Systems (NeurIPS), 2021
bibtex / github / arXiv / video

We develop a general theory to understand the role of normalization layers in improving training dynamics of a neural network at initialization.

quadreg How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation
Ekdeep Singh Lubana, Puja Trivedi, Danai Koutra, and Robert P. Dick
Conference on Lifelong Learning Agents (CoLLAs), 2022
bibtex / github / arXiv / video
(Also presented at ICML Workshop on Theory and Foundations of Continual Learning, 2021)

This work demonstrates how quadratic regularization methods for preventing catastrophic forgetting in deep networks rely on a simple heuristic under-the-hood: Interpolation.

gradflow A Gradient Flow Framework For Analyzing Network Pruning
Ekdeep Singh Lubana and Robert P. Dick
International Conference on Learning Representations (ICLR), 2021 (Spotlight)
bibtex / github / arXiv / video

A unified, theoretically-grounded framework for network pruning that helps justify often used heuristics in the field.

Undergraduate Research
minsip Minimalistic Image Signal Processing for Deep Learning Applications
Ekdeep Singh Lubana, Robert P. Dick, Vinayak Aggarwal, Pyari Mohan Pradhan
International Conference on Image Processing (ICIP), 2019
bibtex /

An image signal processing pipeline that allows use of out-of-the-box deep neural networks on RAW images directly retrieved from image sensors.

Digital Foveation Digital Foveation: An Energy-Aware Machine Vision Framework
Ekdeep Singh Lubana and Robert P. Dick
IEEE Transactions on Computer-Aided Design of Integrated Circuits and System (TCAD), 2018
bibtex /

An energy-efficient machine vision framework inspired by the concept of Fovea in biological vision. Also see follow-up work presented at CVPR workshop, 2020.

SNAP Snap: Chlorophyll Concentration Calculator Using RAW Images of Leaves
Ekdeep Singh Lubana, Mangesh Gurav, and Maryam Shojaei Baghini
IEEE Sensors, 2018; Global Winner, Ericsson Innovation Awards 2017
bibtex / news

An efficient imaging system that accurately calculates chlorophyll content in leaves by using RAW images.

Website template source available here.