Nystromformer

An implementation of the Nyströmformer, using Nystrom method to approximate standard self attention

Stars

55

Forks

4

Language

Python

Last Updated

Jan 21, 2024

Similar Repos
Repo Language Stars Description Updated At
MATLAB 2 An implementation of nystrom method for approximating infinite-dimensional sample in the feature space Oct 26, 2022
MATLAB 2 materials for nystrom method applying in kernel methods Sep 10, 2022
Python 16 Pytorch implementation of Self-Attention ConvLSTM May 08, 2023
Python 5 TensorFlow implementation of Self Attention GAN Nov 15, 2020
Python 85 Implementation of Nyström Self-attention, from the paper Nyströmformer Aug 04, 2022
Python 2 PyTorch Implementation of Self-Attention GAN (SAGAN) Oct 27, 2023
Python 14 The implementation of the paper "Improving Sample Quality of Diffusion Models Using Self-Attention Guidance". Oct 12, 2022
Python 20 The PyTorch implementation of ReCoSa(the Relevant Contexts with Self-attention) for dialogue generation using the multi-head … Mar 14, 2022
Python 65 Self-Attention Generative Adversarial Networks Implementation in PyTorch Aug 28, 2022
None 15 Official implementation of the paper "Improving Sample Quality of Diffusion Model Using the Self-Attention Guidance" Oct 05, 2022
None 2 Implementation of the paper "Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism" Oct 06, 2022
Python 26 Electricity theft detection using Self-Attention mechanisms Mar 28, 2023
Python 69 A 🤗-style implementation of BERT using lambda layers instead of self-attention Aug 04, 2022
Python 16 A PyTorch implementation of the Compact Multi-Head Self-Attention Mechanism from the paper: "Low Rank Factorization … Aug 11, 2022
Python 76 Implementation of Lie Transformer, Equivariant Self-Attention, in Pytorch Aug 01, 2022
Python 5 A PyTorch implementation for Self-Attention Generative Adversarial Networks Mar 25, 2023
Python 2353 Pytorch implementation of Self-Attention Generative Adversarial Networks (SAGAN) May 14, 2023
Python 183 Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient … Jul 26, 2022
Python 80 The official implementation of ELSA: Enhanced Local Self-Attention for Vision Transformer May 24, 2022
Jupyter Notebook 19 Pytorch implementation of the paper Stand-Alone Self-Attention in Vision Models Mar 18, 2023
Scala 5 Spark implementation of kernel k-means with Nystrom approximation Oct 07, 2020
Python 3 Reinforcement learning agent using subnetworks and self-attention May 12, 2022
Julia 15 A Julia package to approximate multivariate continuous functions using Smolyak's method. May 16, 2023
Python 532 Simple Tensorflow implementation of "Self-Attention Generative Adversarial Networks" (SAGAN) Aug 03, 2022
Python 33 Authors implementation of LieTransformer: Equivariant Self-Attention for Lie Groups Dec 23, 2021
None 2 a pytorch implementation of self-attention with relative position representations May 03, 2023
Python 81 A Pytorch implementation of Global Self-Attention Network, a fully-attention backbone for vision tasks May 19, 2022
Python 11 Code search model based the self-attention Sep 21, 2022
Python 74 Implementation of the paper "Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism" [AAAI-MLPS 2021] May 02, 2023
Python 7 An Interpretable Self-Attention Network with block-attention and attention-attribution. Mar 24, 2023
Jupyter Notebook 7 Cacultating Self-Attention with numpy Mar 20, 2023
C++ 20 C++ implementation of Lox interpreter (based on the book Crafting Interpreters by Bob Nystrom) Apr 21, 2023
Python 288 Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories … Aug 03, 2022
Python 33 A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual … May 06, 2022
Python 11 Neural News Recommendation with Multi-Head Self-Attention using BERT Sep 12, 2022
Python 2 Zero-Permutation Jet-Parton Assignment using a Self-Attention Network Oct 01, 2021
Python 438 Implementing Stand-Alone Self-Attention in Vision Models using Pytorch Apr 06, 2023
Python 67 Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch Apr 10, 2023
Python 2 🔢 A program that will approximate the value of x in a function using Bolzano … Apr 03, 2023
Python 5 This is a Tensorflow implementation of the paper Homogeneous Learning: Self-Attention Decentralized Deep Learning. Mar 30, 2023
Python 94 TensorFlow implementation for paper Time Interval Aware Self-Attention for Sequential Recommendation. Apr 23, 2023
Python 1035 Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository. May 22, 2023
Python 115 Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not … Aug 12, 2022
Fortran 3 Godunov method with approximate Riemann solvers for Euler equations. Apr 14, 2023
Python 3 Video Compression Standard Prediction using Attention-based Bidirectional LSTM (JBE 2019) Aug 22, 2022
None 6 Pyramid Self-Attention for Semantic Segmentation Dec 22, 2021
Python 20 Clickbait Language Model with Self-Attention Apr 05, 2023
Python 11 Self-Attention Generative Adversarial Network for Speech Enhancement using Tensorflow 2 Oct 26, 2022
Jupyter Notebook 8 Classifying questions into class and subclass using Self Attention Transformer model Nov 19, 2022
Python 10 The official implementation of Triplet Attention. Aug 10, 2022