kirt_bert_on_abci

Training BERT on ABCI

Stars

19

Forks

5

Language

Python

Last Updated

Jan 04, 2024

Similar Repos
Repo Language Stars Description Updated At
None 4 BERT-related projects on ABCI by AIST-AIRC Nov 09, 2021
Jupyter Notebook 4 Yelp Bert Training and Prediction Apr 04, 2023
C 14 C ABCI libraries Aug 23, 2021
JavaScript 79 Javascript ABCI libraries May 24, 2022
Python 10 BERT for Distributed PyTorch + AMP Training Mar 15, 2023
Jupyter Notebook 7 Training a BERT model from scratch. Apr 18, 2023
Rust 26 Tower-based ABCI interface. Jun 30, 2022
Elixir 25 elixir ABCI spec implementation Aug 04, 2022
Go 31 NDID tendermint ABCI app Mar 04, 2023
Go 2 Forum application demonstrating ABCI++ Sep 15, 2023
Jupyter Notebook 4 pre-training your own BERT modelfrom scratch Mar 05, 2023
Python 48 HeBERT: Pre-training BERT for modern Hebrew Aug 06, 2022
Python 2 Training BERT with relatively few compute resources. Sep 27, 2023
Elixir 11 Tendermint Elixir ABCI Application Boilerplate Nov 24, 2021
Makefile 44 ABCI User Guide & Portal Guide Jul 14, 2022
Go 2 An ABCI cryptocurrency in Golang Aug 31, 2022
Go 4 Proof-of-Concept ABCI Application Mar 15, 2023
Python 12 BERT model training impelmentation using 1024 A100 GPUs for MLPerf Training v1.1 Apr 27, 2022
None 180 PERT: Pre-training BERT with Permuted Language Model Aug 13, 2022
Python 12 Expanded KR-BERT by adding more training data Feb 18, 2023
Go 102 ABCI-client implementation for Optimistic Rollups. Aug 11, 2022
Python 71 PyTorch implementation of BERT in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" Sep 02, 2022
Python 5223 Chinese version of GPT2 training code, using BERT tokenizer. Sep 12, 2022
Python 17 Pre-training BERT masked language models with custom vocabulary Feb 19, 2023
None 23 Chinese version of GPT2 training code, using BERT tokenizer. Apr 10, 2023
Python 6 multi gpu training in one machine for BERT from scratch Aug 27, 2020
Python 14 Ongoing research training transformer language models at scale, including: BERT Apr 16, 2021
Python 237 (CVPR2021) Kaleido-BERT: Vision-Language Pre-training on Fashion Domain. Sep 10, 2022
Python 101 Source code for "Efficient Training of BERT by Progressively Stacking" Jun 19, 2022
Python 203 Enhancing the BERT training with Semi-supervised Generative Adversarial Networks Sep 09, 2022
Python 2 About BERT model training implementation using more than 1000 A100 GPUs for MLPerf Training v2.0 Sep 05, 2022
Python 5 A hyperparameter optimization library for the ABCI. Aug 09, 2022
Python 7297 Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型） Aug 15, 2022
Jupyter Notebook 392 iBOT :robot:: Image BERT Pre-Training with Online Tokenizer (ICLR 2022) Aug 11, 2022
Python 3 CLI for training multiclass and multilabel text classification models with BERT. Jan 21, 2022
Python 27 Code for pre-training CharacterBERT models (as well as BERT models). May 09, 2022
Python 18 Chinese version of GPT2 training code, using BERT or BPE tokenizer. Nov 29, 2022
Jupyter Notebook 2 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding review Jul 21, 2020
Go 17 ABCI-client for Dymension RollApps forked from Optimint Apr 17, 2023
Haskell 32 A Haskell framework that facilitates writing ABCI applications Mar 09, 2023
Go 3 A demo of an oracle implementation using ABCI++ May 16, 2023
Python 13 ☯️ AllenNLP training configurations for promising models on Named Entity Recognition. (BiLSTM-CRF, BiLSTM-CNN-CRF, BERT, BERT-CRF) Jun 02, 2022
Python 987 Cramming the training of a (BERT-type) language model into limited compute. Mar 19, 2023
Python 20 [ICLR 2023] "Mole-BERT: Rethinking Pre-training Graph Neural Networks for Molecules" Apr 23, 2023
None 2 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Jun 12, 2021
Jupyter Notebook 17 This repository contains code for the O'Reilly Live Online Training for BERT May 03, 2022
Python 133 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Aug 07, 2022
Python 174 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Jul 13, 2022
None 2 Drug-BERT: Pre-training Drug Sub-structure Representation for Molecular Property Prediction Sep 14, 2020
Python 85 [NAACL'22] TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning Mar 30, 2023