Knowledge Distillation at a Low Level
Published:
Knowledge Distillation at a Low Level
We’ve all heard about knowledge distillation and how it helps in making models smaller by sacrificing a bit of performance. Essentially, it…
Exploring knowledge distillation techniques from a low-level implementation perspective, understanding how knowledge transfer works under the hood.