EET

Easy and Efficient Transformer : Scalable Inference Solution For Large NLP model

Stars

250

Forks

46

Language

Python

Last Updated

Jan 25, 2024

Similar Repos
Repo Language Stars Description Updated At
Jupyter Notebook 222 Edge Inference in Browser with Transformer NLP model Oct 03, 2022
Jupyter Notebook 6 Scalable NLP model fine-tuning and batch inference with Ray and Anyscale Apr 21, 2023
Python 1338 Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀 Apr 24, 2023
Python 116 Large-scale model inference. Aug 28, 2022
HTML 5 Fast, scalable, parallel and distributed inference of very large networks by Bayesian Model Averaging Mar 09, 2021
Cuda 122 Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity Jan 17, 2024
Python 25 Large Language Model (LLM) Inference API and Chatbot Apr 27, 2023
Python 7 Large Language Model Text Generation Inference May 02, 2023
Python 2 Large Language Model Text Generation Inference Sep 18, 2023
Jupyter Notebook 175 Scalable inference for a generative model of astronomical images Apr 28, 2023
Python 4 [WIP] Efficient and scalable CPU/GPU inference server for OpenAI Whisper models 🚀 Mar 23, 2023
Python 2 Online Inference API for NLP, Transformer models - summarization, text classification, sentiment analysis and more Apr 28, 2023
C++ 6 NLP-Fast: A Fast, Scalable, and Flexible System to Accelerate Large-Scale Heterogeneous NLP Models Apr 20, 2023
Jupyter Notebook 3 Tradeoff between runtime and RAM usage for large language model inference. Apr 20, 2023
None 13 An enterprise ready, resilient and horizontal scalable solution for large video landscapes. Jul 24, 2022
Python 2 Scalable deployment of transformer based paraphrase model using Docker and Kubernetes Apr 07, 2023
Python 3728 An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Language Model for … Apr 09, 2023
Java 32 A universal scalable machine learning model deployment solution Sep 01, 2022
Python 29 Simple, but efficient entity linking model using transformer architecture. Apr 24, 2023
Python 52 An Efficient Pipelined Data Parallel Approach for Training Large Model Jun 01, 2022
Shell 63 Making it easy to build Bitcoin secure, efficient and scalable Bitcoin applications! Nov 24, 2022
Python 9 Efficient methods for LLMs finetuning and inference. May 22, 2023
Jupyter Notebook 7 An FPGA Accelerator for Transformer Inference Jun 13, 2022
C++ 781 Fast inference engine for Transformer models Apr 25, 2023
Python 2 A transformer-based model for extensive multi-tasking of various clinical NLP tasks. Nov 12, 2021
Python 265 Efficient Inference for Big Models Jun 24, 2022
C++ 30 Scalable inference for Correlated Topic Models Oct 12, 2020
JavaScript 2 A simple and efficient solution for dealing with includes in a large YAML file structure Mar 27, 2023
Go 2976 An open, easy, fast, reliable and battery-efficient solution for real-time communications Aug 21, 2022
Python 25 A graph learning library for PyTorch that makes distributed GNN training and inference easy and … Apr 22, 2023
Python 944 High performance, easy-to-use, and scalable package for learning large-scale knowledge graph embeddings. Sep 01, 2022
None 2 High performance, easy-to-use, and scalable package for learning large-scale knowledge graph embeddings. Apr 10, 2022
Python 1376 LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its … Jan 19, 2024
C++ 2 Fast and scalable inference for various Latent Variable Models Mar 21, 2022
Jupyter Notebook 3 Task Complexity Classifier using Transformer-based NLP model based on Bloom's Taxonomy Jan 30, 2023
R 5 Functions for model fitting and inference Feb 16, 2023
None 2 Windows - C++ Visual Studio solution for Image Classification using Caffe Model and TensorRT inference … Feb 17, 2023
Jupyter Notebook 225 1st Solution for 2019-CIKM-Analyticup: Efficient and Novel Item Retrieval for Large-scale Online Shopping Recommendation May 11, 2023
Jupyter Notebook 7 An experimental first-stage model used for quick and efficient inference on part of the data. Sep 03, 2022
Python 3 A scalable, efficient, cross-platform and easy-to-use workflow engine in pure Python Oct 06, 2021
Jupyter Notebook 20 Scalable solution for ML Observability Aug 01, 2022
Python 15 Optimizing scalable ML inference workloads with Amazon Elastic Inference and Amazon EKS Jul 21, 2022
Python 15 Latency and Memory Analysis of Transformer Models for Training and Inference May 04, 2023
Python 7 Inference Model for BertSum Nov 17, 2021
Python 2 Converts a Transformer model, tokenizer, and config to be compatible with Mighty Inference Server (https://max.io/) May 08, 2022
JavaScript 9 Efficient rendering for large lists Apr 06, 2019
Go 13 Simple and efficient autoscalling solution for K8S Apr 28, 2022
Python 4 real-time inference and async inference interface for Stable Diffusion Model Apr 19, 2023
None 2 Implement: EfficientDet: Scalable and Efficient Object Detection Sep 16, 2021
JavaScript 231 Simple and scalable folder management for large Nuxt projects Apr 17, 2023