Three literatures, one underlying question
Every model that ships is a compressed model. Weights get pruned to fit a memory budget, capability gets distilled from a large teacher into a servable student, and, at long context, the KV cache gets evicted or factorised to stay within a hardware budget.
Pruning, distillation, and KV cache compression are usually treated as three separate fields, each with its own benchmarks and its own definition of "lossless." In practice, people stack all three and get accuracy collapses that none of the three literatures, read on its own, would have predicted.
Our claim is that all three are the same question, asked in a different currency: which directions of a model's representation are actually load-bearing under a budget? Pruning removes directions in parameter space, distillation removes them in function space, and KV compression removes them in state space. Once that's visible, the design choices that matter in each area — what to score, on what data, what to retrain — turn out to be the same choices wearing different notation.
The tutorial builds up this shared formalism across four parts, and backs every claim with a runnable, open-weight demo — no compression background assumed, working knowledge of transformers is enough.
Four parts
What survives when weights are removed
From magnitude and calibration-based criteria (SparseGPT, Wanda) to structured removal — and the underreported cost of calibration dependence, where a compressed model quietly inherits the quirks of a few hundred calibration sequences. Includes PruneNet, a calibration-free method that treats compression as policy learning over a model's intrinsic spectral structure, connected to downscaling laws as a predictive frame.
What a student actually learns from a teacher
The divergence zoo — forward KL, reverse KL, JSD, α-divergences — and what each objective really optimises; on-policy methods (GKD, MiniLLM's policy-gradient view); student-aware meta-distillation; cross-tokenizer and speculative variants. Covers the generalisation–fidelity paradox: the objectives that best match the teacher are not the ones that produce students who generalise. Reframed, in closing, as compute amortisation.
What the cache needs to keep, not what it stores
A taxonomy of quantisation, eviction, budget allocation, and low-rank methods, arguing that most scoring functions are proxies for one quantity — how much future reachable behaviour a cache entry supports. Includes CurDKV, which selects entries by CUR leverage scores to preserve attention output rather than the attention map, and a routing analysis revealing a hallucination cliff near 90% compression, plus a second failure mode where entries survive but head consensus collapses.
What happens when all three are stacked
Which combinations are benign and which are catastrophic, the effect of application order, and why the honest report is a Pareto surface rather than a single favourable operating point. Closes with open problems: predictive compressibility laws, compression-aware fine-tuning, and a unified budget that trades parameters against cache against teacher compute.
Demonstrations
Every demo runs in a Colab-grade notebook on a single GPU, using open-weight models (0.5B–8B), with pinned dependencies and cached artefacts.
The composition failure
A model pruned, distilled, and cache-compressed at individually "safe" ratios collapses on a task each stage passed alone — the motivating artefact for the whole tutorial.
Calibration sensitivity
One pruning algorithm, one ratio, three calibration sets — accuracy moves. The calibration-free alternative runs alongside for comparison, together with layer-wise singular-value spectra.
The divergence zoo
One teacher, one student, matched compute — swapping forward KL, reverse KL, JSD, and α-divergence. Attendees inspect output distributions directly and see mode-seeking versus mode-covering behaviour.
On-policy vs. off-policy
A minimal GKD implementation against sequence-level distillation, instrumented to show where student samples change the gradient — paired with a case where the best-matching student is the worse out-of-distribution model.
Eviction policies on a common harness
Recency, attention-score, and CUR-based selection at matched budgets, built on kvpress so attendees can plug in their own scorer in about twenty lines.
The cliff, and where benchmarks lie
Global eviction ratio tracked against hallucination rate up to 90% compression, with the same cache scored on synthetic retrieval versus a realistic long-context task.
Hardware accounting
Peak memory, time-to-first-token, and decode throughput for every configuration above — so attendees can see which reported compression ratios actually become wall-clock wins.
Take-home notebook
Accepts a model, a memory budget, and a task, and returns a Pareto frontier across all three compression axes.
Instructors
Tanmoy Chakraborty
Rajiv Khemani Young Faculty Chair in AI · Professor, EE & Yardi School of AI, IIT Delhi
Founds and leads the LCS2 research group. Works across NLP, graph neural networks, and social computing, currently focused on economical, adaptable, and interpretable language models. Author of Social Network Analysis and Introduction to Large Language Models; ACM Distinguished Speaker; Program Chair, EMNLP 2025 and ASONAM 2024; Action Editor, TACL and CL; Editor-in-Chief, ACL Rolling Review.
Ayan Sengupta
Fourth-year PhD candidate, IIT Delhi
Leads the Parmanu project on efficient large language models, advised by Tanmoy Chakraborty. Research spans model pruning, knowledge distillation, and KV cache compression, including PruneNet (ICLR 2025), MPDistil (ICLR 2024), CurDKV (NeurIPS 2025), and downscaling laws (ICML 2025). Contributor to Hugging Face PEFT and NVIDIA kvpress. Recipient, Microsoft Research India PhD Award 2026.
Palaash Goel
First-year PhD student, IIT Delhi
Advised by Tanmoy Chakraborty, with three years of NLP research experience. B.Tech in Computer Science and Artificial Intelligence, IIIT Delhi (2025), including work on multimodal sarcasm analysis published at NAACL. Currently focused on advancing LLM compression techniques for practitioners with limited compute.
LCS2 — Laboratory for Computational Social Systems
IIT Delhi · Department of Electrical Engineering & Yardi School of AI