PretrainingBERT

Pre-training BERT masked language models with custom vocabulary

Stars

17

Forks

5

Language

Python

Last Updated

Feb 19, 2023

Similar Repos
Repo Language Stars Description Updated At
None 180 PERT: Pre-training BERT with Permuted Language Model Aug 13, 2022
Python 1072 MASS: Masked Sequence to Sequence Pre-training for Language Generation Aug 08, 2022
Python 16 Masked Structural Growth for 2x Faster Language Model Pre-training Jan 21, 2024
Python 27 Code for pre-training CharacterBERT models (as well as BERT models). May 09, 2022
Python 140 Masked Visual Pre-training for Robotics Apr 28, 2023
Python 4 Pre-training Machine-Reader (Instead of Masked Language Model) at Scale Jun 03, 2023
Python 237 (CVPR2021) Kaleido-BERT: Vision-Language Pre-training on Fashion Domain. Sep 10, 2022
Python 600 Portuguese pre-trained BERT models Sep 10, 2022
Jupyter Notebook 29 Pre-trained Mongolian BERT models Jan 02, 2022
Jupyter Notebook 2 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding review Jul 21, 2020
None 119 📝 A list of pre-trained BERT models for Japanese with word/subword tokenization + vocabulary construction … Aug 19, 2022
Python 71 PyTorch implementation of BERT in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" Sep 02, 2022
Python 14 Ongoing research training transformer language models at scale, including: BERT Apr 16, 2021
Python 218 MPNet: Masked and Permuted Pre-training for Language Understanding https://arxiv.org/pdf/2004.09297.pdf Aug 08, 2022
HTML 6 Fill-in-the-BERT uses pre-trained BERT Masked Language Model for Infering the task of fill in the … Apr 11, 2023
Python 258 Pre-Trained Models for ToD-BERT Sep 08, 2022
None 131 Pre-trained Nordic models for BERT Sep 01, 2022
Python 230 Transformer with Untied Positional Encoding (TUPE). Code of paper "Rethinking Positional Encoding in Language Pre-training". … Aug 30, 2022
Python 203 Implementation of DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models Mar 15, 2023
Jupyter Notebook 4 pre-training your own BERT modelfrom scratch Mar 05, 2023
Python 48 HeBERT: Pre-training BERT for modern Hebrew Aug 06, 2022
Python 227 [CVPR 2022] Pre-Training 3D Point Cloud Transformers with Masked Point Modeling Jun 27, 2022
Python 208 Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" Sep 12, 2022
Python 36 Implement Question Generator with SOTA pre-trained Language Models (RoBERTa, BERT, GPT, BART, T5, etc.) May 05, 2023
Python 46 Author: Wenhao Yu ([email protected]). ACL 2022 Dict-BERT: Enhancing Language Model Pre-training with Dictionary May 10, 2023
None 7 Object Detection with Vision-Language Pre-training Apr 10, 2023
Python 5 Masked language models of biosynthetic gene clusters Jul 25, 2022
None 2 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Jun 12, 2021
Python 133 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Aug 07, 2022
Python 174 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Jul 13, 2022
None 2 Ongoing research training transformer language models at scale, including: BERT & GPT-2 May 22, 2023
Python 4 Ongoing research training transformer language models at scale, including: BERT & GPT-2 May 30, 2023
Python 4 Ongoing research training transformer language models at scale, including: BERT & GPT-2 Jun 02, 2023
Jupyter Notebook 392 iBOT :robot:: Image BERT Pre-Training with Online Tokenizer (ICLR 2022) Aug 11, 2022
Python 4 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 中文 汉语 Apr 16, 2022
None 2 pre-trained Language Models Mar 04, 2023
Jupyter Notebook 159 pre-trained Language Models May 10, 2023
Python 5 Annotating Columns with Pre-trained Language Models Aug 02, 2022
Python 574 Grounded Language-Image Pre-training Sep 01, 2022
Python 5 Grounded Language-Image Pre-training Mar 31, 2023
Python 11 BERT Tokenizer with vocabulary tailored for Cantonese Apr 18, 2023
Python 9 [ICLR 2023] Masked Frequency Modeling for Self-Supervised Visual Pre-Training Apr 29, 2023
Python 111 Voxel-MAE: Masked Autoencoders for Pre-training Large-scale Point Clouds Oct 19, 2022
Python 54 Financial Domain Question Answering with pre-trained BERT Language Model May 25, 2022
None 2 Pre-Training of German T5 models Aug 29, 2023
Python 5 TensorFlow code and pre-trained models for BERT Dec 11, 2019
Python 6 TensorFlow code and pre-trained models for BERT Apr 19, 2021
Python 6 TensorFlow code and pre-trained models for BERT Jan 14, 2022
Python 12 TensorFlow code and pre-trained models for BERT Apr 27, 2022
Python 32026 TensorFlow code and pre-trained models for BERT Sep 02, 2022