Knowledge Distillation

Meta-prediction Model for Distillation-Aware NAS on Unseen Datasets
Distillation-aware Neural Architecture Search (DaNAS) aims to search for an optimal student architecture that obtains the best …
Hayeon Lee*, Sohyun An*, Minseon Kim, Sung Ju Hwang
Cite PDF Code
Lightweight Neural Architecture Search with Parameter Remapping and Knowledge Distillation
Designing diverse neural architectures taking into account resource constraints or datasets is one of the main challenges in Neural …
Hayeon Lee*, Sohyun An*, Minseon Kim, Sung Ju Hwang
Cite PDF