Video / Slides

Compressive Finetuning

• Can Yaras, Peng Wang, Laura Balzano, Qing Qu (2024). Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation. International Conference on Machine Learning (ICML’24), 2024. (Oral, top 1.5%)
  Preprint – PDF –  BibTex – Code
• Changwoo Lee, Soo Min Kwon, Qing Qu, Hun-Seok Kim. BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inference. Neural Information Processing Systems (NeurIPS’24), 2024.
  Preprint – PDF –  BibTex – Code

Compressive Training

• Soo Min Kwon*, Zekai Zhang*, Dogyoon Song, Laura Balzano, Qing Qu (2023). Efficient Low-Dimensional Compression of Overparameterized Models. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (AISTATS’24), 2024. Preprint – PDF –  BibTex – Code
• Can Yaras*, Peng Wang*, Wei Hu, Zhihui Zhu, Laura Balzano, Qing Qu (2023). The Law of Parsimony in Gradient Descent for Learning Deep Linear Networks. ArXiv Preprint arXiv:2306.01154, 2023.
  Preprint – PDF –  BibTex – Code – Slides
• Changwoo Lee, Soo Min Kwon, Qing Qu, Hun-Seok Kim. BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inference. Neural Information Processing Systems (NeurIPS’24), 2024.
  Preprint – PDF –  BibTex – Code

Compression at Initialization

• Avrajit Ghosh, Xitong Zhang, Kenneth K. Sun, Qing Qu, Saiprasad Ravishankar, Rongrong Wang (2024). Optimal Eye Surgeon: Finding Image Priors Through Sparse Generators at Initialization. International Conference on Machine Learning (ICML’24), 2024.
  Preprint – PDF –  BibTex – Code

Training with Large Learning Rates

• Avrajit Ghosh, Soo Min Kwon, Rongrong Wang, Saiprasad Ravishankar, Qing Qu. Learning Dynamics of Deep Matrix Factorization Beyond the Edge of Stability. International Conference on Learning Representations (ICLR’25), 2025.
  Preprint – PDF –  BibTex

Efficient Training of Diffusion Models

• Huijie Zhang*, Yifu Lu*, Ismail Alkhouri, Saiprasad Ravishankar, Dogyoon Song, Qing Qu (2023). Improving Efficiency of Diffusion Models via Multi-Stage Framework and Tailored Multi-Decoder Architectures. Conference on Computer Vision and Pattern Recognition (CVPR’24), 2024.
  Preprint – PDF –  BibTex – Code – Project Website