KD_Lib

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

Stars

525

Forks

49

Language

Python

Last Updated

Jul 13, 2023

Similar Repos
Repo Language Stars Description Updated At
Python 6 model compression and optimization for deployment for Pytorch, including knowledge distillation, quantization and pruning.(知识蒸馏，量化，剪枝) Mar 07, 2023
Python 2 Knowledge Distillation Extension Library for PyTorch May 05, 2022
Python 2 Knowledge Distillation Papers in Pytorch Apr 13, 2022
Python 16 Knowledge Distillation Algorithms implemented with PyTorch Feb 05, 2023
Python 119 A knowledge distillation toolkit based on PyTorch and PyTorch Lightning. Apr 18, 2023
Python 20 This project attempts to build neural network training and lightweighting cookbook including three kinds of … May 06, 2023
Python 2 Model compression related, Pruning and quantization Jun 30, 2022
None 2 A PyTorch Implementation of Knowledge Distillation on SSD Nov 08, 2020
Python 1266 Pytorch implementation of various Knowledge Distillation (KD) methods. May 11, 2023
Python 2 Pytorch Implementation of Relational Knowledge Distillation, CVPR 2019 Feb 28, 2022
Python 3 A PyTorch Implementation of Knowledge Distillation on SSD Mar 27, 2023
Python 14 PyTorch implementation of (Hinton) Knowledge Distillation and a base class for simple implementation of other … Apr 15, 2023
Python 1227 A PyTorch-based knowledge distillation toolkit for natural language processing Sep 12, 2022
Python 3 A PyTorch-based knowledge distillation toolkit for natural language processing Apr 12, 2022
Python 177 pytorch implementation for Patient Knowledge Distillation for BERT Model Compression Sep 05, 2022
Python 5 Pytorch implement of Zero-shot Knowledge Distillation in Deep Neural Networks Apr 07, 2022
Python 76 [AAAI 2023] Official pytorch implementation for "Curriculum Temperature for Knowledge Distillation" May 18, 2023
Python 1444 A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility Sep 01, 2022
Python 196 An Object Detection Knowledge Distillation framework powered by pytorch, now having SSD and yolov5. May 10, 2023
Jupyter Notebook 131 On-the-fly Structured Pruning for PyTorch models. This library implements several attributions metrics and structured pruning … Mar 31, 2023
Python 5 Compact representations of convolutional neural networks via weight pruning and quantization Mar 03, 2022
Python 12 provide some new architecture, channel pruning and quantization methods for yolov5 Apr 03, 2023
Python 3 Rethinking the Value of Network Pruning (Pytorch) Jan 20, 2022
Jupyter Notebook 44 Official PyTorch implementation of Data-free Knowledge Distillation for Object Detection, WACV 2021. Jul 09, 2022
Python 35 [ICCV 2021] Official pytorch implementation for "Online Knowledge Distillation for Efficient Pose Estimation" Apr 08, 2023
Python 129 [CVPR 2020] APQ: Joint Search for Network Architecture, Pruning and Quantization Policy Jul 05, 2022
Python 1589 [ICLR 2020] Contrastive Representation Distillation (CRD), and benchmark of recent knowledge distillation methods Oct 20, 2022
Python 43 INTERSPEECH 2023: "DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models" Jun 20, 2023
Python 19 DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization Aug 22, 2022
Shell 123 A research library for pytorch-based neural network pruning, compression, and more. Oct 29, 2022
Python 42 This is the official implementation of the paper "Instance-conditional Knowledge Distillation for Object Detection", based … Jul 18, 2022
Python 67 PyTorch code for ECCV 2020 paper: "Robust Re-Identification by Multiple Views Knowledge Distillation" Mar 17, 2023
None 10 Official pytorch implementation for "Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?" May 27, 2023
Python 182 PyTorch implementation for the APoT quantization (ICLR 2020) Aug 07, 2022
Python 3 PyTorch implementation of the paper: CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharing. Mar 25, 2023
Python 8 KDSS is the framework for knowledge distillation from LLMs Oct 25, 2023
Python 2 PyTorch utilities for parameter pruning and multiplies reduction Nov 02, 2020
Python 387 PyTorch library to facilitate development and standardized evaluation of neural network pruning methods. Apr 26, 2023
Python 214 Focal and Global Knowledge Distillation for Detectors (CVPR 2022) Aug 20, 2022
Python 11 This is the official PyTorch implementation for the paper: "Directed Acyclic Graph Factorization Machines for … Apr 19, 2023
Python 47 PyTorch code for training MM-DistillNet for multimodal knowledge distillation. http://rl.uni-freiburg.de/research/multimodal-distill Jul 21, 2022
Python 20 An official implementation of "Decomposed Knowledge Distillation for Class-incremental Semantic Segmentation" (NeurIPS 2022) in PyTorch. Nov 26, 2022
Jupyter Notebook 2 Knowledge Distillation using Teacher-Student and Teacher-Assistant-Student Models Feb 05, 2022
None 9 some paper of Knowledge Distillation and Adversarial Training about NLP May 18, 2023
Python 1285 A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and … Aug 31, 2022
None 2 A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and … Sep 22, 2021
None 8 A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and … Apr 03, 2023
Python 26 Code for the NeurIPS 2022 paper "Optimal Brain Compression: A Framework for Accurate Post-Training Quantization … Apr 30, 2023
Python 3 Distill on the Go: Online knowledge distillation in self-supervised learning Apr 08, 2022
Python 6 The implementation of "Personalized Edge Intelligence via Federated Self- Knowledge Distillation". Apr 24, 2023