Knowledge distillation is an increasingly influential technique in deep learning that involves transferring the knowledge embedded in a large, complex “teacher” network to a smaller, more efficient ...