compact-multi-head-self-attention-pytorch

A PyTorch implementation of the Compact Multi-Head Self-Attention Mechanism from the paper: "Low Rank Factorization for Compact Multi-Head Self-Attention"

Stars

16

Forks

3

Language

Python

Last Updated

Aug 11, 2022

Similar Repos
Repo Language Stars Description Updated At
Python 110 Multi-head attention in PyTorch Nov 30, 2022
Python 2 Visualize DETR Multi-Head Self-Attention weight May 08, 2023
Python 4 A pytorch version handwritten recognition with CNN and Multi head self attention Jan 11, 2022
Python 15 Source code of the paper "Attention as Relation: Learning Supervised Multi-head Self-Attention for Relation Extraction, … Dec 22, 2021
Python 20 The PyTorch implementation of ReCoSa(the Relevant Contexts with Self-attention) for dialogue generation using the multi-head … Mar 14, 2022
Jupyter Notebook 74 Low-Rank Matrix Factorization for Recommender Systems Apr 11, 2023
Python 11 Neural News Recommendation with Multi-Head Self-Attention using BERT Sep 12, 2022
Python 115 Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not … Aug 12, 2022
Python 6 Multi-Head Attention, Transformer, Perceiver, Linear Attention. Oct 19, 2023
Python 145 pytorch neural network attention mechanism Jan 05, 2024
Python 34 Gated Recurrent Unit with Low-rank matrix factorization Apr 02, 2020
Jupyter Notebook 2 PyTorch subspace low rank pruning Oct 11, 2022
Python 2 ResNet for License Plate Detection & Multi-Head Self Attention Transformer for OCR Jun 12, 2023
Python 5 Official code of ICASSP 2023 paper "Frame-Level Multi-Label Playing Technique Detection Using Multi-Scale Network and … Oct 29, 2023
Jupyter Notebook 19 Pytorch implementation of the paper Stand-Alone Self-Attention in Vision Models Mar 18, 2023
None 13 Code for our SIGIR 2021 short paper "Lighter and Better: Low-Rank Decomposed Self-Attention Networks for … Aug 05, 2022
None 2 华为Non-invasive self-Attention mechanism Jun 05, 2023
None 2 Implementation of the paper "Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism" Oct 06, 2022
Python 3 Low-rank tensor learning with PyTorch. Oct 21, 2021
Python 2 Low rank implementations of PyTorch layers May 18, 2022
Python 5 Recommender System using Low Rank Matrix Factorization (Cross Validation Added) Jun 06, 2019
Python 12 This is (official) pytorch implementation of the paper "Reference based sketch extraction via attention mechanism" Apr 25, 2023
Jupyter Notebook 5 Code for "Low Rank Forecasting" paper. Mar 10, 2022
Python 6 Code for ACL 2022 findings paper "Gaussian Multi-head Attention for Simultaneous Machine Translation" Apr 14, 2023
C 35 Multi-scale Low Rank Matrix Decomposition Code Oct 24, 2022
Python 4 Repository for the Multi-Head Multi-Loss Model Calibration paper Feb 28, 2024
Python 8 MULTI HEAD SELF-ATTENTION BASED SPATIAL-TEMPORAL INFORMATION GRAPH CONVOLUTIONAL NETWORKS FOR TRAFFIC FLOW FORECASTING May 18, 2022
Python 16 Pytorch implementation of Self-Attention ConvLSTM May 08, 2023
Python 2 A PyTorch implementation of MCA based on PRICAI 2022 paper "Weakly-supervised Temporal Action Localization with … Jul 29, 2023
MATLAB 3 code of "Hyperspectral restoration via L0 Gradient Regularized Low-Rank Tensor factorization" Apr 05, 2023
Python 29 The Implementation of Attention Mechanism Enhanced Kernel Prediction Networks in PyTorch. Jan 20, 2023
Python 74 Implementation of the paper "Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism" [AAAI-MLPS 2021] May 02, 2023
Python 42 Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a … Jul 26, 2022
Python 2 PyTorch implementation of image captioning based on attention mechanism May 17, 2022
Python 2 Transformer and Attention Mechanism written in PyTorch Lightning ⚡️ Jun 10, 2021
MATLAB 24 multi-focus image fusion using low-rank representation Apr 12, 2023
Python 135 Code for Multi-Head Attention: Collaborate Instead of Concatenate Apr 29, 2023
Python 2 A seq2seq neural network model based on multi-head self-attention mechanism at encoding stage and adopting … Sep 24, 2022
Python 2 PyTorch Implementation of Self-Attention GAN (SAGAN) Oct 27, 2023
Python 85 Implementation of Nyström Self-attention, from the paper Nyströmformer Aug 04, 2022
Jupyter Notebook 146 Python codes for low-rank tensor factorization, tensor completion, and tensor regression techniques. Oct 18, 2022
MATLAB 5 The code for the paper "Matrix recovery with implicitly low-rank data" Feb 12, 2022
Jupyter Notebook 4 Pytorch implementation of the paper Attention Augmented Convolutional Networks Jan 01, 2023
Python 15 Some Rank/Multi-task model implemented by Pytorch Aug 04, 2022
MATLAB 4 Multi-slice MR Reconstruction with Low-Rank Tensor Completion Apr 27, 2022
Jupyter Notebook 3 Multi-Head Bi-Directional Attention Flow For Machine Reading Comprehension Aug 07, 2022
Matlab 2 A simple implementation of low-rank matrix factorization using MATLAB's built-in Levenberg Marquardt. Feb 28, 2017
Python 65 Self-Attention Generative Adversarial Networks Implementation in PyTorch Aug 28, 2022
Python 65 BERT+Self-attention Encoder ; Biaffine Decoder ; Pytorch Implement Sep 01, 2022
Python 3 Visualizing RNNs using the attention mechanism Jun 13, 2022