llama.mmengine

Training LLaMA language model with MMEngine! It supports LoRA fine-tuning!

Stars

39

Forks

0

Language

Python

Last Updated

Nov 24, 2023

Similar Repos
Repo Language Stars Description Updated At
Python 45 llama fine-tuning with lora Jul 18, 2023
None 3 LLaMA-TRL: Fine-tuning LLaMA with PPO and LoRA Apr 06, 2023
Python 1639 Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, quantization, LoRA fine-tuning, … Apr 01, 2023
Python 2 Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, quantization, LoRA fine-tuning, … May 31, 2023
Python 4 Fine-tuning LLaMA with PEFT (SFT+RLHF) May 28, 2023
Python 5 Tools for testing and fine-tuning LLaMA models using LoRA, based on Alpaca-LoRA (https://github.com/tloen/alpaca-lora). Apr 09, 2023
None 2 Implementation of the StableLM/Pythia language models based on nanoGPT. Supports flash attention, Int8 and GPTQ … May 04, 2023
Python 3 A Toolkit for Fine-Tuning Large Language Models with LoRA and DeepSpeed Apr 14, 2023
Python 42 Language Model Fine-tuning for Moby Dick Jun 20, 2022
Python 221 Guide for fine-tuning Llama/CodeLlama models Jan 18, 2024
Dockerfile 4 fine-tuning LLaMA to generate SCAD to make STLs Apr 20, 2023
Jupyter Notebook 3 Efficient Fine-tuning LLaMA Using DiffFit within 0.7M Parameters May 03, 2023
Python 48 LoRA training model packaged with Cog Apr 23, 2023
Python 610 Code for fine-tuning Platypus fam LLMs using LoRA Jan 17, 2024
Python 8 Minimal implementation of multiple PEFT methods for LLaMA fine-tuning Apr 26, 2023
None 2 Minimal implementation of multiple PEFT methods for LLaMA fine-tuning Apr 20, 2023
Python 210 1.4B latent diffusion model fine tuning Sep 11, 2022
Python 18 Fine-tuning "ImageBind One Embedding Space to Bind Them All" with LoRA May 17, 2023
Python 5 用RLHF可选LoRA对LLaMA和MOSS进行训练|Training LLaMA or MOSS with RLHF [LoRA] Jun 12, 2023
Jupyter Notebook 21 Notebooks for fine-tuning a BERT model and training a LSTM model for financial QA Mar 09, 2023
Python 74 Repository for training a LoRA for the LLaMA model on HuggingFace with 8-bit quantization. Research … Apr 07, 2023
Python 342 Preprint: Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning Jan 17, 2024
Python 6 Text classification with Foundation Language Model LLaMA Apr 06, 2023
Python 40 use google pre-training model bert to fine-tuning for the chinese multiclass classification Aug 26, 2022
Python 88 Efficient Training (including pre-training and fine-tuning) for Big Models Aug 12, 2022
Python 61 Fine Tuning Language Models for Multilabel Prediction Aug 12, 2022
Jupyter Notebook 3 Fine Tuning for Image Recognition with a model trained with Reptile Feb 08, 2020
Python 2068 Fine-tuning LLaMA to follow instructions within 1 Hour and 1.2M Parameters Apr 10, 2023
None 2 Fine-tuning LLaMA to follow instructions within 1 Hour and 1.2M Parameters Apr 12, 2023
None 6 Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters Jun 19, 2023
Jupyter Notebook 58 KDD19 Tutorial: From Shallow to Deep Language Representations: Pre-training, Fine-tuning, and Beyond Mar 14, 2022
Python 38 A Kernel-Based View of Language Model Fine-Tuning https://arxiv.org/abs/2210.05643 Oct 19, 2022
Jupyter Notebook 33 Fine-tuning 6-Billion GPT-J (& other models) with LoRA and 8-bit compression Mar 21, 2023
Python 4 Fine-tuning large language models with huggingface transformers and deepspeed Jul 03, 2023
Python 335 Fine-Tuning your VITS model using a pre-trained model May 10, 2023
Jupyter Notebook 27 BERT Model Fine-tuning on Quora Questions Pairs Aug 26, 2022
None 4 fine-tuned LLaMA model on both English and Chinese Mar 29, 2023
Python 87 TorToiSe fine-tuning with DLAS May 03, 2023
Python 2 TorToiSe fine-tuning with DLAS Jul 04, 2023
Python 26 An Implementation of ERNIE For Language Understanding (including Pre-training models and Fine-tuning tools) Oct 18, 2022
Python 2 Tuning Llama Models > 7b Jun 19, 2023
Python 10 Final training script from HuggingFace Whisper Fine tuning event - to get best results on … Feb 15, 2023
Python 13 Example of Alpaca-LoRA with llama index. Apr 08, 2023
Python 3 Tools for the LLaMA language model Apr 09, 2023
Python 18 Finetune Bloom big language model with Lora method Apr 19, 2023
Python 2 Discord bot for interacting with the LLaMA language model Apr 02, 2023
Python 3 housing our model example of fine tuning an 11B t5 with FSDP Apr 15, 2023
Jupyter Notebook 6 Scalable NLP model fine-tuning and batch inference with Ray and Anyscale Apr 21, 2023
Python 2 BERT fine-tuning with Chinese corpus Sep 21, 2021
Python 32 Code for Paper: Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data Jul 20, 2022