transformer-blocks

Multi-Head Attention, Transformer, Perceiver, Linear Attention.

Stars

6

Forks

2

Language

Python

Last Updated

Oct 19, 2023

Similar Repos
Repo Language Stars Description Updated At
Python 2 ResNet for License Plate Detection & Multi-Head Self Attention Transformer for OCR Jun 12, 2023
Python 159 Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation May 02, 2023
Python 110 Multi-head attention in PyTorch Nov 30, 2022
Python 2 Visualize DETR Multi-Head Self-Attention weight May 08, 2023
Python 135 Code for Multi-Head Attention: Collaborate Instead of Concatenate Apr 29, 2023
Python 16 A PyTorch implementation of the Compact Multi-Head Self-Attention Mechanism from the paper: "Low Rank Factorization … Aug 11, 2022
Python 11 Neural News Recommendation with Multi-Head Self-Attention using BERT Sep 12, 2022
Jupyter Notebook 3 Multi-Head Bi-Directional Attention Flow For Machine Reading Comprehension Aug 07, 2022
Python 853 An implementation of Performer, a linear attention-based transformer, in Pytorch Aug 11, 2022
Python 37 Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch Aug 10, 2022
Python 48 SOTA Google's Perceiver-AR Music Transformer Implementations and Models Apr 14, 2023
Python 3 A sliding window based multi-head attention using longformer cuda kernels Oct 22, 2022
None 5 Nested Deformable Multi-head Attention for Facial Image Inpainting [WACV-23] Apr 26, 2023
Python 9 Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch Feb 24, 2022
Python 815 Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch Aug 11, 2022
Python 82 Implementation of Perceiver, General Perception with Iterative Attention in TensorFlow Jul 23, 2022
Python 4 A pytorch version handwritten recognition with CNN and Multi head self attention Jan 11, 2022
Python 65 Local windowed attention multi-instrumental music transformer for supervised music generation May 01, 2023
Python 8 text classification with various advanced modules and latest models such as leam, hard attention, multi-head … Apr 30, 2022
Python 2 PyTorch Attention Transformer wrapper Sep 03, 2022
Python 784 [CVPR 2023] Neighborhood Attention Transformer and [arXiv] Dilated Neighborhood Attention Transformer repository. Apr 26, 2023
Python 45 Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012 May 05, 2022
Python 42 Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a … Jul 26, 2022
Python 88 Implementation of CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification Aug 18, 2022
Python 2 [Experimental] Full-attention multi-instrumental music transformer with single token MIDI encoding May 12, 2023
Python 15 Source code of the paper "Attention as Relation: Learning Supervised Multi-head Self-Attention for Relation Extraction, … Dec 22, 2021
Python 6 Code for ACL 2022 findings paper "Gaussian Multi-head Attention for Simultaneous Machine Translation" Apr 14, 2023
JavaScript 19 Visualizing the Music Transformer attention Apr 20, 2023
Python 4 [Preprint] Neighborhood Attention Transformer, 2022 Mar 30, 2023
None 2 [Preprint] Neighborhood Attention Transformer, 2022 Apr 18, 2023
Python 2 [Preprint] Neighborhood Attention Transformer, 2022 Nov 14, 2022
Jupyter Notebook 18 Multi-Headed Self-Attention via Vision Transformer for Zero-Shot Learning (ViT-ZSL) May 29, 2022
Python 9 Sparse Attention with Linear Units May 29, 2022
Python 403 Transformer based on a variant of attention that is linear complexity in respect to sequence … Aug 10, 2022
Python 7 Experiments using fast linear transformer Nov 28, 2022
Python 8 MULTI HEAD SELF-ATTENTION BASED SPATIAL-TEMPORAL INFORMATION GRAPH CONVOLUTIONAL NETWORKS FOR TRAFFIC FLOW FORECASTING May 18, 2022
Python 20 The PyTorch implementation of ReCoSa(the Relevant Contexts with Self-attention) for dialogue generation using the multi-head … Mar 14, 2022
Python 115 Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not … Aug 12, 2022
Python 2 Attention-Aligned Transformer for Image Captioning Mar 03, 2024
Python 60 Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity) Apr 26, 2023
Python 205 A PyTorch implementation of Perceiver, Perceiver IO and Perceiver AR with PyTorch Lightning scripts for … Oct 06, 2022
Python 8 The code for Kernel attention transformer (KAT) Sep 22, 2022
Python 2 NatIR: Image Restoration Using Neighborhood-Attention-Transformer Mar 20, 2023
Python 269 Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image … Aug 10, 2022
None 2 Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image … Mar 07, 2021
Jupyter Notebook 2 Abstractive summarisation on financial news articles with Chinese-English translation via multi-headed attention transformer architecture Feb 13, 2022
Python 7 Local windowed attention multi-instrumental music transformer tailored for music orchestration/instrumentation and stable music generation Apr 06, 2023
Python 32 PyTorch code for "Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention" (WACV 2023) May 23, 2023
R 2 Multi-Dimensional Linear Model Jul 05, 2022
Python 161 Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time" Aug 30, 2022