Ekdeep Singh Lubana

I am a PhD candidate in EECS Department at the University of Michigan, working with Prof. Robert Dick. I often collaborate with Dr. Hidenori Tanaka at Physics and Informatics Lab, NTT Research as well.

I am generally interested in designing theoretically motivated, grounded algorithms for practical applications of DNNs--e.g., model compression, continual learning, and federated learning. I am also very interested in better understanding training dynamics of neural networks, especially via a statistical physics perspective.

I graduated with a Bachelor's degree in ECE from Indian Institute of Technology (IIT), Roorkee in 2019. My research in undergraduate was primarily focused on embedded systems, such as energy-efficient machine vision systems.

Email  /  CV  /  Google Scholar  /  Twitter  /  Github

profile photo
[05/2022] Our work on unsupervised federated learning was accepted to ICML, 2022 (arXiv link coming soon).
[05/2022] Visiting Harvard Center for Brain Science this Summer!
[01/2022] Our work on unsupervised graph representation learning was accepted to TheWebConf (formerly WWW), 2022.
[10/2021] Our work on dynamics of normalization layers was accepted to NeurIPS, 2021.
[09/2021] I will be interning with Akhil Mathur at Bell Labs, UK this fall.
[05/2021] I will be interning with Hidenori Tanaka at Physics and Informatics Lab, NTT Research this summer.
[03/2021] Our work on theory of pruning was accepted as a spotlight presentation at ICLR, 2021.
Orchestra Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering
Ekdeep Singh Lubana, Chi Ian Tang, Fahim Kawsar Robert P. Dick, and Akhil Mathur
International Conference on Machine Learning (ICML), 2022
bibtex / arXiv (coming soon) / github (coming soon)

We propose an unsupervised learning method that exploits client heterogeneity to enable privacy preserving, SOTA performance unsupervised federated learning.

www Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practices
Puja Trivedi, Ekdeep Singh Lubana, Yujun Yan, Yaoqing Yang, and Danai Koutra
ACM The Web Conference (formerly WWW), 2022
bibtex / arXiv

We contextualize the performance of several unsupervised graph representation learning methods with respect to inductive bias of GNNs and show significant improvements by using structured augmentations defined by task-relevance.

beyondbn Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
Ekdeep Singh Lubana, Hidenori Tanaka, and Robert P. Dick
Advances in Neural Information Processing Systems (NeurIPS), 2021
bibtex / github / arXiv

We develop a general theory to understand the role of normalization layers in improving training dynamics of a neural network at initialization.

gradflow How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation
Ekdeep Singh Lubana, Puja Trivedi, Danai Koutra, and Robert P. Dick
ICML Workshop on Theory and Foundations of Continual Learning, 2021
bibtex / github / arXiv

This work demonstrates how quadratic regularization methods for preventing catastrophic forgetting in deep networks rely on a simple heuristic under-the-hood: Interpolation.

gradflow A Gradient Flow Framework For Analyzing Network Pruning
Ekdeep Singh Lubana and Robert P. Dick
International Conference on Learning Representations (ICLR), 2021 (Spotlight)
bibtex / github / arXiv

A unified, theoretically-grounded framework for network pruning that helps justify often used heuristics in the field.

minsip Minimalistic Image Signal Processing for Deep Learning Applications
Ekdeep Singh Lubana, Robert P. Dick, Vinayak Aggarwal, Pyari Mohan Pradhan
International Conference on Image Processing (ICIP), 2019
bibtex /

An image signal processing pipeline that allows use of out-of-the-box deep neural networks on RAW images directly retrieved from image sensors.

Machine Foveation Machine Foveation: An Application-Aware Compressive Sensing Framework
Ekdeep Singh Lubana, Vinayak Aggarwal, and Robert P. Dick
Data Compression Conference (DCC), 2019
bibtex /

An application-aware sampling framework that helps reduce processing energy by focusing on application-relevant regions only.

Digital Foveation Digital Foveation: An Energy-Aware Machine Vision Framework
Ekdeep Singh Lubana and Robert P. Dick
IEEE Transactions on Computer-Aided Design of Integrated Circuits and System (TCAD), 2018
bibtex /

An energy-efficient machine vision framework inspired by the concept of Fovea in biological vision. Also see follow-up work presented at CVPR workshop, 2020.

SNAP Snap: Chlorophyll Concentration Calculator Using RAW Images of Leaves
Ekdeep Singh Lubana, Mangesh Gurav, and Maryam Shojaei Baghini
IEEE Sensors, 2018; Global Winner, Ericsson Innovation Awards 2017
bibtex / news

An efficient imaging system that accurately calculates chlorophyll content in leaves by using RAW images.

Website template source available here.