Ekdeep Singh Lubana

I am a PhD candidate in EECS Department at the University of Michigan, working with Prof. Robert Dick. I often collaborate with Dr. Hidenori Tanaka at Harvard Center for Brain Science as well.

I am generally interested in designing theoretically motivated, grounded algorithms for practical applications of DNNs--e.g., model compression, continual learning, and federated learning. I am also very interested in better understanding training dynamics of neural networks, especially via a statistical physics perspective.

I graduated with a Bachelor's degree in ECE from Indian Institute of Technology (IIT), Roorkee in 2019. My research in undergraduate was primarily focused on embedded systems, such as energy-efficient machine vision systems.

Email  /  CV  /  Google Scholar  /  Twitter  /  Github

profile photo
[01/2022] Our work on data-centric analysis of graph contrastive learning was accepted to NeurIPS, 2022.
[05/2022] Our work on unsupervised federated learning was accepted as a spotlight at ICML, 2022!
[05/2022] Joining Harvard Center for Brain Science as a research affiliate!
[10/2021] Our work on dynamics of normalization layers was accepted to NeurIPS, 2021.
[09/2021] I will be interning with Akhil Mathur at Bell Labs, UK this fall.
[05/2021] I will be interning with Hidenori Tanaka at Physics and Informatics Lab, NTT Research this summer.
[03/2021] Our work on theory of pruning was accepted as a spotlight at ICLR, 2021.
GraphSSL Analyzing Data-Centric Properties for Contrastive Learning on Graphs
Puja Trivedi, Ekdeep Singh Lubana, Mark Heimann, Danai Koutra, and Jay Jayaraman Thiagarajan
Advances in Neural Information Processing Systems (NeurIPS), 2022
bibtex / arXiv / github

We propose a theoretical framework that demonstrates limitations of popular graph augmentation strategies for self-supervised learning.

Orchestra Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering
Ekdeep Singh Lubana, Chi Ian Tang, Fahim Kawsar, Robert P. Dick, and Akhil Mathur
International Conference on Machine Learning (ICML), 2022 (Spotlight)
bibtex / arXiv / github / video

We propose an unsupervised learning method that exploits client heterogeneity to enable privacy preserving, SOTA performance unsupervised federated learning.

beyondbn Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
Ekdeep Singh Lubana, Hidenori Tanaka, and Robert P. Dick
Advances in Neural Information Processing Systems (NeurIPS), 2021
bibtex / github / arXiv / video

We develop a general theory to understand the role of normalization layers in improving training dynamics of a neural network at initialization.

quadreg How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation
Ekdeep Singh Lubana, Puja Trivedi, Danai Koutra, and Robert P. Dick
Conference on Lifelong Learning Agents (CoLLAs), 2022
bibtex / github / arXiv / video
(Also presented at ICML Workshop on Theory and Foundations of Continual Learning, 2021)

This work demonstrates how quadratic regularization methods for preventing catastrophic forgetting in deep networks rely on a simple heuristic under-the-hood: Interpolation.

gradflow A Gradient Flow Framework For Analyzing Network Pruning
Ekdeep Singh Lubana and Robert P. Dick
International Conference on Learning Representations (ICLR), 2021 (Spotlight)
bibtex / github / arXiv / video

A unified, theoretically-grounded framework for network pruning that helps justify often used heuristics in the field.

minsip Minimalistic Image Signal Processing for Deep Learning Applications
Ekdeep Singh Lubana, Robert P. Dick, Vinayak Aggarwal, Pyari Mohan Pradhan
International Conference on Image Processing (ICIP), 2019
bibtex /

An image signal processing pipeline that allows use of out-of-the-box deep neural networks on RAW images directly retrieved from image sensors.

Machine Foveation Machine Foveation: An Application-Aware Compressive Sensing Framework
Ekdeep Singh Lubana, Vinayak Aggarwal, and Robert P. Dick
Data Compression Conference (DCC), 2019
bibtex /

An application-aware sampling framework that helps reduce processing energy by focusing on application-relevant regions only.

Digital Foveation Digital Foveation: An Energy-Aware Machine Vision Framework
Ekdeep Singh Lubana and Robert P. Dick
IEEE Transactions on Computer-Aided Design of Integrated Circuits and System (TCAD), 2018
bibtex /

An energy-efficient machine vision framework inspired by the concept of Fovea in biological vision. Also see follow-up work presented at CVPR workshop, 2020.

SNAP Snap: Chlorophyll Concentration Calculator Using RAW Images of Leaves
Ekdeep Singh Lubana, Mangesh Gurav, and Maryam Shojaei Baghini
IEEE Sensors, 2018; Global Winner, Ericsson Innovation Awards 2017
bibtex / news

An efficient imaging system that accurately calculates chlorophyll content in leaves by using RAW images.

Website template source available here.