lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, quantization, LoRA fine-tuning, pre-training. Apache 2.0-licensed.

Stars

3

Forks

1

Language

Python

Last Updated

Jan 18, 2024

Similar Repos
Repo Language Stars Description Updated At
Python 1639 Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, quantization, LoRA fine-tuning, … Apr 01, 2023
None 2 Implementation of the StableLM/Pythia language models based on nanoGPT. Supports flash attention, Int8 and GPTQ … May 04, 2023
Python 45 llama fine-tuning with lora Jul 18, 2023
Python 35 Training LLaMA language model with MMEngine! It supports LoRA fine-tuning! Apr 09, 2023
None 3 LLaMA-TRL: Fine-tuning LLaMA with PPO and LoRA Apr 06, 2023
Python 5 Tools for testing and fine-tuning LLaMA models using LoRA, based on Alpaca-LoRA (https://github.com/tloen/alpaca-lora). Apr 09, 2023
Python 4 Fine-tuning LLaMA with PEFT (SFT+RLHF) May 28, 2023
Python 221 Guide for fine-tuning Llama/CodeLlama models Jan 18, 2024
Dockerfile 4 fine-tuning LLaMA to generate SCAD to make STLs Apr 20, 2023
Jupyter Notebook 3 Efficient Fine-tuning LLaMA Using DiffFit within 0.7M Parameters May 03, 2023
Python 610 Code for fine-tuning Platypus fam LLMs using LoRA Jan 17, 2024
Python 8 Minimal implementation of multiple PEFT methods for LLaMA fine-tuning Apr 26, 2023
None 2 Minimal implementation of multiple PEFT methods for LLaMA fine-tuning Apr 20, 2023
Python 7 Recipes from fine-tuning a pre-trained wav2vec 2.0 model using the espresso tool kit May 16, 2023
Jupyter Notebook 2 Tensorflow 2.0 kullanarak transfer öğrenme ve fine tuning gerçekleştirme Oct 29, 2021
Jupyter Notebook 32 Resources for running, fine-tuning, and inferencing Dolly 2.0 Apr 15, 2023
Python 2068 Fine-tuning LLaMA to follow instructions within 1 Hour and 1.2M Parameters Apr 10, 2023
None 2 Fine-tuning LLaMA to follow instructions within 1 Hour and 1.2M Parameters Apr 12, 2023
None 6 Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters Jun 19, 2023
Python 335 Fine-Tuning your VITS model using a pre-trained model May 10, 2023
Jupyter Notebook 14 ❓Fine-Tuning BERT for extractive QA on SQuAD 2.0 May 09, 2023
Python 3 A Toolkit for Fine-Tuning Large Language Models with LoRA and DeepSpeed Apr 14, 2023
Python 18 Fine-tuning "ImageBind One Embedding Space to Bind Them All" with LoRA May 17, 2023
Python 88 Efficient Training (including pre-training and fine-tuning) for Big Models Aug 12, 2022
Python 173 Chinese GPT2: pre-training and fine-tuning framework for text generation Jul 29, 2022
Jupyter Notebook 33 Fine-tuning 6-Billion GPT-J (& other models) with LoRA and 8-bit compression Mar 21, 2023
Python 74 Repository for training a LoRA for the LLaMA model on HuggingFace with 8-bit quantization. Research … Apr 07, 2023
Python 13 CSS-LM: Contrastive Semi-supervised Fine-tuning of Pre-trained Language Models Aug 08, 2022
Python 80 [CVPR 2020] Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning Jun 29, 2022
Python 451 Fast & Simple repository for pre-training and fine-tuning T5-style models Apr 30, 2023
Jupyter Notebook 2 Notebooks for teaching transformers and fine-tuning a pre-trained BERT model May 19, 2023
Python 24 PyTorch – SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models May 24, 2022
Python 13 Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models Aug 06, 2022
Python 40 use google pre-training model bert to fine-tuning for the chinese multiclass classification Aug 26, 2022
Jupyter Notebook 58 KDD19 Tutorial: From Shallow to Deep Language Representations: Pre-training, Fine-tuning, and Beyond Mar 14, 2022
Python 125 Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques … Jan 08, 2024
Jupyter Notebook 2 Easily customizable implementation of Transfer Learning with Fine-Tuning using Keras pre-trained CNN models. Apr 21, 2023
Python 26 An Implementation of ERNIE For Language Understanding (including Pre-training models and Fine-tuning tools) Oct 18, 2022
Jupyter Notebook 365 End-to-End recipes for pre-training and fine-tuning BERT using Azure Machine Learning Service Aug 05, 2022
Python 3 End-to-End recipes for pre-training and fine-tuning BERT using Azure Machine Learning Service Mar 21, 2022
Python 17 PyTorch code for "FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization" (NAACL 2022) Jun 24, 2022
Python 101 The PyTorch implementation of fine-tuning the GPT-2(Generative Pre-Training 2) for dialogue generation. Sep 05, 2022
TypeScript 2 Llamallamallama is a chat solution that allows users to chat with "Llama" fine tuned with … Apr 15, 2024
Python 37 Instruct LLMs for flat and nested NER. Fine-tuning Llama and Mistral models for instruction named … Feb 26, 2024
Jupyter Notebook 12 Here is a Google Colab Notebook for fine-tuning Alpaca Lora (within 3 hours with a … Apr 16, 2023
Python 12 A fork of textgen that still supports V1 GPTQ, 4-bit lora and other GPTQ models … May 04, 2023
None 2 Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning[NAACL 2022] Jan 27, 2023
Python 12 A unified framework for Low-resource Audio Processing and Evaluation (SSL Pre-training and Downstream Fine-tuning) Jun 27, 2023
Python 51 official implementation for our paper Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning Dec 11, 2023
Python 8 The ability of GPT-2 to create abstractive summaries with fine-tuning using only keywords, without pre-training. Jan 04, 2022