KDD19-tutorial

KDD19 Tutorial: From Shallow to Deep Language Representations: Pre-training, Fine-tuning, and Beyond

Stars

58

Forks

32

Language

Jupyter Notebook

Last Updated

Mar 14, 2022

Similar Repos
Repo Language Stars Description Updated At
Python 88 Efficient Training (including pre-training and fine-tuning) for Big Models Aug 12, 2022
None 7 UniLM - Unified Language Model Pre-training / Pre-training for NLP and Beyond Jun 03, 2022
Python 80 [CVPR 2020] Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning Jun 29, 2022
Python 11 General Fine-Tuning: A little language for Deep Nets (ACL-2022 Tutorial) Apr 21, 2023
Python 26 An Implementation of ERNIE For Language Understanding (including Pre-training models and Fine-tuning tools) Oct 18, 2022
Python 173 Chinese GPT2: pre-training and fine-tuning framework for text generation Jul 29, 2022
Python 451 Fast & Simple repository for pre-training and fine-tuning T5-style models Apr 30, 2023
Python 24 PyTorch – SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models May 24, 2022
Python 35 Training LLaMA language model with MMEngine! It supports LoRA fine-tuning! Apr 09, 2023
Python 13 CSS-LM: Contrastive Semi-supervised Fine-tuning of Pre-trained Language Models Aug 08, 2022
Python 40 use google pre-training model bert to fine-tuning for the chinese multiclass classification Aug 26, 2022
Jupyter Notebook 365 End-to-End recipes for pre-training and fine-tuning BERT using Azure Machine Learning Service Aug 05, 2022
Python 3 End-to-End recipes for pre-training and fine-tuning BERT using Azure Machine Learning Service Mar 21, 2022
Python 17 PyTorch code for "FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization" (NAACL 2022) Jun 24, 2022
Python 12 A unified framework for Low-resource Audio Processing and Evaluation (SSL Pre-training and Downstream Fine-tuning) Jun 27, 2023
None 2 Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning[NAACL 2022] Jan 27, 2023
Python 2 Pre-training of Deep Bidirectional Transformers for Language Understanding Oct 30, 2018
Python 1639 Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, quantization, LoRA fine-tuning, … Apr 01, 2023
Python 2 Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, quantization, LoRA fine-tuning, … May 31, 2023
Python 953 Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN Aug 03, 2022
Python 42 Language Model Fine-tuning for Moby Dick Jun 20, 2022
Python 61 Fine Tuning Language Models for Multilabel Prediction Aug 12, 2022
Python 101 The PyTorch implementation of fine-tuning the GPT-2(Generative Pre-Training 2) for dialogue generation. Sep 05, 2022
Python 214 Code for the paper Fine-Tuning Language Models from Human Preferences Aug 12, 2022
Python 413 An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks Aug 08, 2022
Jupyter Notebook 4 GPT3 fine tuning from Excel sheet Jan 02, 2023
Python 51 official implementation for our paper Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning Dec 11, 2023
Python 335 Fine-Tuning your VITS model using a pre-trained model May 10, 2023
Jupyter Notebook 2 Notebooks for teaching transformers and fine-tuning a pre-trained BERT model May 19, 2023
Python 4 Fine-tuning large language models with huggingface transformers and deepspeed Jul 03, 2023
Python 8 The ability of GPT-2 to create abstractive summaries with fine-tuning using only keywords, without pre-training. Jan 04, 2022
TypeScript 312 LLM fine-tuning and eval Jan 17, 2024
Jupyter Notebook 2 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding review Jul 21, 2020
Python 7 Recipes from fine-tuning a pre-trained wav2vec 2.0 model using the espresso tool kit May 16, 2023
Python 9 Code for our SIGIR 2022 accepted paper : P3 Ranker: Mitigating the Gaps between Pre-training … May 13, 2022
Python 2 A primer of transformer models and fine-tuning on language tasks Jan 10, 2022
Python 574 Grounded Language-Image Pre-training Sep 01, 2022
Python 5 Grounded Language-Image Pre-training Mar 31, 2023
Python 13 Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models Aug 06, 2022
Jupyter Notebook 7 Fine Tuning and Feature Extraction Examples May 20, 2022
None 2 Implementation of the StableLM/Pythia language models based on nanoGPT. Supports flash attention, Int8 and GPTQ … May 04, 2023
JavaScript 4 Script for fine-tuning GPT-3 from Airtable Apr 15, 2023
Python 3 A Toolkit for Fine-Tuning Large Language Models with LoRA and DeepSpeed Apr 14, 2023
Python 9 Speech synthesis (TTS) in low-resource languages by training from scratch with Fastpitch and fine-tuning with … Apr 15, 2023
Python 10 Final training script from HuggingFace Whisper Fine tuning event - to get best results on … Feb 15, 2023
Python 208 Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" Sep 12, 2022
Jupyter Notebook 10 This repository provides code and resources for Parameter Efficient Fine-Tuning (PEFT), a technique for improving … Oct 13, 2023
Python 7 Trainer and Evaluation scripts for fine-tuning Whisper models for the Ukrainian language Apr 20, 2023
Python 264 A short tutorial on performing fine tuning or transfer learning in PyTorch. Apr 25, 2023
Jupyter Notebook 3 Fine-tuning Transformers from HuggingFace for financials news sentiment Aug 30, 2023