lite-transformer

[ICLR 2020] Lite Transformer with Long-Short Range Attention

Stars

587

Forks

78

Language

Python

Last Updated

Jan 15, 2024

Similar Repos
Repo Language Stars Description Updated At
Python 2 [ICLR 2020] Lite Transformer with Long-Short Range Attention Jun 02, 2023
Python 103 Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, … Jul 26, 2022
Python 83 Representing Long-Range Context for Graph Neural Networks with Global Attention May 16, 2023
Python 63 Long Short-Term Transformer for Online Action Detection Aug 23, 2022
Jupyter Notebook 2 Image Descriptor with Visual Attention Mechanism Using Long Short-term Memory Jul 06, 2022
Python 14 Code for the ICLR 2020 paper "Attention-based interpretability with Concept Transformers" Jul 26, 2022
Python 186 Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021). Jul 30, 2022
Python 90 [ECCV2022] Efficient Long-Range Attention Network for Image Super-resolution Aug 12, 2022
Python 83 [ICLR'23 Spotlight] Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs May 15, 2023
Python 2 Explore the performance of Transformer models with different attention mechanisms on long-sequence prediction tasks. Jul 16, 2022
Python 4 PyTorch implementation of Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Jul 27, 2022
C 527 Long Range Zip May 17, 2023
Python 121 Source code of paper "BP-Transformer: Modelling Long-Range Context via Binary Partitioning" Mar 27, 2023
Python 53 Implementation of Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification. Jul 25, 2022
Python 6 Multi-Head Attention, Transformer, Perceiver, Linear Attention. Oct 19, 2023
Python 2 PyTorch Attention Transformer wrapper Sep 03, 2022
C++ 3 long story short Dec 09, 2019
Python 784 [CVPR 2023] Neighborhood Attention Transformer and [arXiv] Dilated Neighborhood Attention Transformer repository. Apr 26, 2023
Python 12 Companion website for the article Learning Long-Range Perception Using Self-Supervision from Short-Range Sensors and Odometry Dec 20, 2022
Python 24 Tensorflow implementation of the Long-Short Term Memory Network with Deep Attention Fusion for the SNLI … Dec 11, 2019
C 146 Raven Long Range System Mar 20, 2023
Python 296 Repository of Vision Transformer with Deformable Attention (CVPR2022) Aug 18, 2022
Python 24 Jittor implementation of Vision Transformer with Deformable Attention May 18, 2022
Python 269 Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image … Aug 10, 2022
None 2 Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image … Mar 07, 2021
Python 2 ICLR 2020 and 2019 reviews Nov 11, 2020
Python 7 ICLR 2020 and 2019 reviews Sep 02, 2021
JavaScript 19 Visualizing the Music Transformer attention Apr 20, 2023
Python 4 [Preprint] Neighborhood Attention Transformer, 2022 Mar 30, 2023
None 2 [Preprint] Neighborhood Attention Transformer, 2022 Apr 18, 2023
Python 2 [Preprint] Neighborhood Attention Transformer, 2022 Nov 14, 2022
Python 27 Code for SegTree Transformer (ICLR-RLGM 2019). Jan 28, 2023
Python 255 Code for Federated Learning with Matched Averaging, ICLR 2020. Aug 02, 2022
None 2 Neural Machine Translation with universal Visual Representation (ICLR 2020) May 27, 2021
Python 2 Code for Federated Learning with Matched Averaging, ICLR 2020. Jun 16, 2022
Python 249 "Few-shot Text Classification with Distributional Signatures" ICLR 2020 Apr 28, 2023
Python 85 Reference-based Image Super-Resolution with Deformable Attention Transformer Apr 24, 2023
Python 4 Image Captioning with Transformer (Attention is All You Need) Apr 07, 2020
Python 6 implementation of "Leveraging Transformer Self Attention Encoder for Crisis Event Detection in Short Texts" (ECIR … May 23, 2022
Python 1540 Longformer: The Long-Document Transformer Aug 11, 2022
Python 10 Transformer-based Long Document Classification Feb 27, 2023
Python 2 UFS Short-Range Weather Application Apr 26, 2022
Python 3 UFS Short-Range Weather Application Jun 28, 2022
Python 39 UFS Short-Range Weather Application Dec 02, 2022
Python 104 Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention" Aug 11, 2022
Python 10 Implementation of Mega, the Single-head Attention with Multi-headed EMA architecture that currently holds SOTA on … Sep 23, 2022
Python 2 Attention-Aligned Transformer for Image Captioning Mar 03, 2024
Jupyter Notebook 1390 Improving Convolutional Networks via Attention Transfer (ICLR 2017) Apr 26, 2023
Python 27 [ICLR'20] [PyTorch] Inverted Attention Routing for Capsules Apr 21, 2022
Python 22 Sentic Long Short Term Memory Sep 06, 2021