Implementation of abstractive summarization using LSTM in the encoder-decoder architecture with local attention.
[CVPR 2023] Efficient Semantic Segmentation by Altering Resolutions for Compressed Videos
Implementation of LA_MIL, Local Attention Graph-based Transformer for WSIs, PyTorch
Neural Machine Tranlation using Local Attention
LEAP: Linear Explainable Attention in Parallel for causal language modeling with O(1) path length, and O(1) inference
🚀 Self-implemented Vision Transformer (ViT) with Local Attention! Unlike standard ViT, this version integrates local attention for improved efficiency. Fully customizable with configurable patch embeddings, attention mechanisms, transformer layers as well as mixing global and local attention.
From-scratch attention mechanisms in NumPy: scaled dot-product, self/cross, multi-head, causal/local, MQA/GQA, additive attention, and verified gradients.
Investigating inductive biases in CNNs vs Transformers. Code and report for the Deep Learning Course Project, ETH Zurich, HS 2021.
To associate your repository with the local-attention topic, visit your repo's landing page and select "manage topics."