Megatron-LM

Ongoing research training transformer models at scale

Stars

362

Forks

50

Language

Python

Last Updated

May 08, 2024

Similar Repos
Repo Language Stars Description Updated At
Python 3149 Ongoing research training transformer models at scale Aug 10, 2022
None 2 Ongoing research training transformer models at scale Jun 17, 2022
Python 4 Ongoing research training transformer models at scale Apr 18, 2023
Python 14 Ongoing research training transformer models at scale Dec 08, 2023
Python 2 Ongoing research training transformer models at scale May 09, 2024
Python 14 Ongoing research training transformer language models at scale, including: BERT Apr 16, 2021
None 2 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Jun 12, 2021
Python 133 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Aug 07, 2022
Python 174 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Jul 13, 2022
None 2 Ongoing research training transformer language models at scale, including: BERT & GPT-2 May 22, 2023
Python 4 Ongoing research training transformer language models at scale, including: BERT & GPT-2 May 30, 2023
Python 4 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Jun 02, 2023
Jupyter Notebook 10 Staged Training for Transformer Language Models Jul 21, 2022
Python 4 PyTorch DistributedDataParallel training for Transformer models. Apr 04, 2023
Python 2 Efficiently training Transformer-based models in single line. May 06, 2023
HTML 4 Data Management in large-scale education research training series Feb 04, 2023
Python 96 Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale, TACL (2022) Mar 19, 2023
Scala 4 Training Large Scale Statistical Machine Translation Models on Spark Aug 18, 2019
Python 3 Training Large-scale Text Embedding Models with 🤗 Transformers Aug 26, 2023
Python 15 Latency and Memory Analysis of Transformer Models for Training and Inference May 04, 2023
Python 4 Training transformer models (e.g. RoBERTa, GPT2 and GPT-J) from scratch. Oct 09, 2023
Jupyter Notebook 4 Transformers Workshop on behalf of ML India. Contains resource notebook for training/inferring large scale transformer … Apr 14, 2023
Python 42 A library of transformer models for computer vision and multi-modality research Aug 22, 2022
C++ 8 Ongoing attempts at gesture models Apr 02, 2022
Jupyter Notebook 38 Input pipelines for large scale, sharded training of deep learning models. Sep 10, 2021
Python 69 Pytorch library for end-to-end transformer models training, inference and serving Dec 12, 2022
Jupyter Notebook 6 Ongoing and published research, papers and code. Jul 28, 2021
Go 2 Trayne is a distributed machine learning platform for training models at scale Jan 17, 2023
HTML 2 Experimental material for ongoing research, or stuff outside a research project. Jan 13, 2019
Python 4 Hierarchical Multi-Scale Gaussian Transformer Implementation May 06, 2023
Python 4 Norwegian Speech Transformer Models Aug 29, 2022
Python 3 Modyn is a research-platform for training ML models on dynamic datasets. May 04, 2023
Python 9 Testing DQN training directly in the real world through small-scale cars models. Apr 18, 2023
Python 39 PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021 May 30, 2022
None 2 Markdown notes for all ongoing research / infra projects Apr 22, 2024
Python 25 Research publication code for "Forward Compatible Training for Large-Scale Embedding Retrieval Systems", CVPR 2022. Aug 01, 2022
Python 27 DeepGNN is a framework for training machine learning models on large scale graph data. Aug 05, 2022
Python 1060 A Large-scale Chinese Short-Text Conversation Dataset and Chinese pre-training dialog models Oct 16, 2022
Python 3 A Large-scale Chinese Short-Text Conversation Dataset and Chinese pre-training dialog models Apr 20, 2021
None 2 A Large-scale Chinese Short-Text Conversation Dataset and Chinese pre-training dialog models Nov 05, 2021
None 2 A Large-scale Chinese Short-Text Conversation Dataset and Chinese pre-training dialog models Nov 15, 2023
Python 4 Training a chatbot using a transformer. Aug 01, 2022
Python 572 Transformer training code for sequential tasks Jul 13, 2022
Python 131 Training Transformer-XL on 128 GPUs Apr 13, 2022
Jupyter Notebook 53 Chinese Transformer Generative Pre-Training Model Apr 02, 2023
None 6 Scale models of WIkihouse Jun 24, 2022
Jupyter Notebook 2 An ongoing MSc research project comparing deep learning models on their capabilities and interpretabilities in … Jun 08, 2023
Julia 415 Julia Implementation of Transformer models Apr 30, 2023
Python 2 Study on 'Transformer'-based models Jun 30, 2023
Jupyter Notebook 4 Learning and Testing transformer models Sep 28, 2023