|
|
None |
13 |
Implementation of a simple BPE tokenizer, but in Nim |
Jan 03, 2023 |
|
|
Go |
68 |
Go BPE tokenizer (Encoder+Decoder) for GPT2 and GPT3 |
Jun 19, 2023 |
|
|
Python |
24 |
Russian version of GPT2 Bert и BPE tokenizer |
Mar 28, 2023 |
|
|
JavaScript |
152 |
JS tokenizer for LLaMA based LLMs |
Jun 19, 2023 |
|
|
Python |
13 |
Sentencepiece based BPE tokenizer for English and Japanese language text. |
Apr 30, 2023 |
|
|
TypeScript |
6 |
Isomorphic Tokenizer for GPT3 algorithm for OpenAI. |
May 14, 2022 |
|
|
Python |
18 |
Chinese version of GPT2 training code, using BERT or BPE tokenizer. |
Nov 29, 2022 |
|
|
Python |
5 |
OpenAI compatible API for open source LLMs |
Jun 18, 2023 |
|
|
C# |
2 |
.NET BPE Encoder / Decoder for GPT-2 / GPT-3 |
Jan 07, 2023 |
|
|
Python |
4 |
Simple documentation generation using OpenAI LLMs |
Mar 27, 2023 |
|
|
Swift |
30 |
Swift BPE Encoder/Decoder for OpenAI GPT Models. A programmatic interface for tokenizing text for OpenAI … |
Apr 24, 2023 |
|
|
JavaScript |
25 |
A command line interface for the OpenAI LLMs. |
Mar 21, 2023 |
|
|
Python |
21 |
A tiny implementation of an autonomous agent powered by LLMs (OpenAI GPT-4) |
Mar 30, 2023 |
|
|
Go |
2 |
Golang BPE (Bytes Pair Encoding) algorithm implementation. |
Aug 01, 2022 |
|
|
C# |
27 |
OpenAI library for .NET |
Jan 21, 2023 |
|
|
Java |
9 |
Java implementation of GPT2 tokenizer. |
Jul 28, 2022 |
|
|
Java |
2 |
Java implementation of GPT2 tokenizer. |
Apr 23, 2023 |
|
|
None |
14 |
Hack OpenAI LLMs' System Prompts By Reverse Prompt Engineering |
Apr 29, 2023 |
|
|
TypeScript |
13 |
Use self-hosted LLMs with an OpenAI compatible API |
Jun 21, 2023 |
|
|
Java |
83 |
JTokkit is a Java tokenizer library designed for use with OpenAI models. |
Apr 26, 2023 |
|
|
TypeScript |
2 |
Isomorphic JavaScript/TypeScript Tokenizer for GPT-4 and Codex Models by OpenAI. |
Apr 30, 2023 |
|
|
Rust |
138 |
Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigram … |
Oct 19, 2022 |
|
|
Python |
4 |
openai style api for open large language models, using LLMs just as chatgpt! |
May 24, 2023 |
|
|
Jupyter Notebook |
2 |
OpenAI and ChatGPT meet .Net |
Feb 24, 2024 |
|
|
None |
3 |
Benchmarking BPE embeddings for Text Classification |
Feb 08, 2019 |
|
|
Go |
95 |
Pure Go implementation of OpenAI's tiktoken tokenizer |
Apr 12, 2023 |
|
|
Java |
20 |
Java implementation of a GPT3/4 tokenizer. |
Oct 19, 2023 |
|
|
JavaScript |
2 |
An implementation of Keras Tokenizer in JavaScript. |
Jan 12, 2024 |
|
|
Python |
9 |
Subword and BPE Tokenizers for Turkish language |
Feb 18, 2022 |
|
|
Python |
3 |
Burmese version of GPT equivalent to GPT2 from openai with secret sauce(special tokenizer). |
May 01, 2024 |
|
|
Python |
30 |
A simple coding assistant app powered by Microsoft guidance, OpenAI LLMs, and Gradio. |
Jun 29, 2023 |
|
|
JavaScript |
104 |
WhatsApp client with LLM integration. It support OpenAI API and also local LLMs |
Jan 19, 2024 |
|
|
Python |
2 |
A dashboard to query PDFs using LLMs such as using OpenAI and GPT4ALL |
May 24, 2024 |
|
|
HTML |
17 |
Blazor Server playground for OpenAI using Cledev.OpenAI .NET library |
Apr 03, 2023 |
|
|
Python |
2168 |
Call all LLM APIs using the OpenAI format. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, … |
Oct 21, 2023 |
|
|
Java |
10 |
Rest client for manipulating bpm-bpe related resources |
May 31, 2022 |
|
|
Python |
3 |
🚧 WIP 🚧 OpenAI compatible API for open source LLMs (LLaMA, ChatGLM, MOSS and many … |
May 22, 2023 |
|
|
Python |
72 |
PyTorch Implementation of OpenAI GPT |
Mar 21, 2023 |
|
|
Python |
5 |
OpenAI clip implementation in PaddlePaddle |
Sep 20, 2022 |
|
|
Python |
2 |
PyTorch Implementation of OpenAI GPT |
May 08, 2023 |
|
|
C# |
2 |
.NET Implementation for HypeRate.IO |
Jan 13, 2023 |
|
|
Python |
54 |
Japanese tokenizer for Transformers |
Oct 15, 2022 |
|
|
Rust |
35 |
Lindera tokenizer for Tantivy. |
Sep 20, 2022 |
|
|
JavaScript |
11 |
A tokenizer for French |
Mar 31, 2021 |
|
|
Go |
41 |
Tokenizer (lexer) for golang |
Apr 21, 2023 |
|
|
JavaScript |
2 |
RWKV tokenizer for node.js |
Jun 08, 2023 |
|
|
Python |
2 |
Tokenizer For Indian Languages |
Mar 29, 2023 |
|
|
C# |
38 |
.NET 7 SDK for OpenAI with a Blazor Server playground |
Apr 10, 2023 |
|
|
Python |
49 |
Pytorch implementation of OpenAI-GPT for ROC stories |
Feb 09, 2023 |
|
|
Jupyter Notebook |
3 |
Implementation of an AI Instructor using LLMs and Langchain |
Sep 16, 2023 |