InfMoE

Inference framework for MoE layers based on TensorRT with Python binding

Stars

41

Forks

5

Language

C++

Last Updated

Apr 25, 2024

Similar Repos
Repo Language Stars Description Updated At
None 2 Inference framework for MoE layers based on TensorRT with Python binding Mar 07, 2022
None 5 Inference framework for MoE layers based on TensorRT with Python binding Dec 22, 2021
Python 4 MT-Yolov6 TensorRT Inference with Python. Oct 07, 2022
Python 17 tensorRt-inference darknet2onnx pytorch2onnx mxnet2onnx python version Feb 17, 2023
C++ 4 YOLO v5 inference with TensorRT (C++) Apr 28, 2023
Python 13 Train cifar10 networks and inference with tensorrt. Feb 01, 2023
Python 13 A framework for knowledge distillation using TensorRT inference on teacher network Apr 26, 2023
Python 2 Integrate python with TensorRT for high performance deep learning inference mainly for embedded sys Feb 05, 2021
Python 9 Inference speed-up for stable-diffusion (ldm) with TensorRT. Mar 17, 2023
Python 43 ChatBot: sample for TensorRT inference with a TF model Jul 27, 2022
C++ 10 Vectornet with tensorrt deploy, less than 1ms/object for inference. May 29, 2023
Python 2 Paddleocr code for stand alone tensorrt inference Nov 01, 2022
C++ 203 MTCNN C++ implementation with NVIDIA TensorRT Inference accelerator SDK Apr 30, 2023
Python 3 Torch, onnxruntime and TensorRT inference program for RTM3D. Jul 11, 2023
Python 2 Pygmalion is a python-based framework for inference in graphical models May 03, 2023
C++ 2 yolov5s suitable for bdd100k with tensorrt inference, support image folder and video input, and mAP … Aug 22, 2022
Python 268 Convert pointpillars Pytorch Model To ONNX for TensorRT Inference Aug 03, 2022
Python 28 Global Data Binding for Python Qt framework May 07, 2023
C++ 662 Deep learning inference nodes for ROS / ROS2 with support for NVIDIA Jetson and TensorRT Sep 11, 2022
Jupyter Notebook 44 A scalable python-based framework for gene regulatory network inference using tree-based ensemble regressors. Apr 04, 2023
Python 65 Python Tk declarative UI framework with reactive data binding Mar 01, 2023
Java 16 A data binding framework for web based user interfaces with Vaadin Nov 07, 2022
Python 64 Randomization-based inference in Python Aug 03, 2022
Go 14 Golang binding for Menoh DNN inference library Apr 19, 2019
Jupyter Notebook 11 Haskell binding for Menoh DNN inference library Mar 27, 2020
C 24 Ruby binding for Menoh DNN inference library Jul 09, 2020
C# 15 C# binding for Menoh DNN inference library Apr 20, 2020
JavaScript 5 NodeJS binding for Menoh DNN inference library Jul 11, 2019
JavaScript 6 Lambda layers test with serverless framework May 01, 2023
Python 3 Python binding for the G'MIC Image Processing Framework Aug 05, 2022
Go 11 Testing framework for easily testing GORM-based data access layers Apr 20, 2022
C++ 4 Deep learning inference framework based on TensoRT。 May 29, 2022
Python 8 A framework for working with inference rules. Apr 26, 2023
Python 139 Python sample for referencing object detection model with TensorRT Apr 23, 2023
None 2 Deep learning inference nodes for ROS with support for NVIDIA Jetson TX1/TX2/Xavier and TensorRT May 02, 2021
C++ 9 Deep learning inference nodes for ROS with support for NVIDIA Jetson TX1/TX2/Xavier and TensorRT May 04, 2023
Python 2 Python interface for inference with LibDAI Jun 05, 2021
Python 2 Common Framework for Inference May 12, 2022
Cython 7 A Cython based Python binding for modern libpcap Apr 29, 2022
Python 3 CFFI-based Python binding for LZF stream compression Jan 31, 2022
Jupyter Notebook 45 Benchmark inference speed of CNNs with various quantization methods in Pytorch+TensorRT with Jetson Nano/Xavier Mar 13, 2023
Jupyter Notebook 194 Real-time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention. Dec 03, 2022
C++ 5 Python binding of SLAM graph optimization framework g2o Jul 12, 2021
C++ 570 Python binding of SLAM graph optimization framework g2o May 04, 2023
Python 10 Tools for simple inference testing using TensorRT, CUDA and OpenVINO CPU/GPU and CPU providers. Simple … Jun 22, 2022
Python 3 Python Dependency Injector based on interface binding Jan 26, 2022
Scala 20 A framework for PSL inference. Jan 11, 2021
C++ 4385 TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and … Jan 19, 2024
C# 348 .NET/C# binding for Baidu paddle inference library and PaddleOCR Aug 19, 2022
Python 41 Machine learning based causal inference/uplift in Python Jul 19, 2022