Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block from the ITTR paper for Image to Image Translation using Transformers

Overview

ITTR - Pytorch

Implementation of the Hybrid Perception Block (HPB) and Dual-Pruned Self-Attention (DPSA) block from the ITTR paper for Image to Image Translation using Transformers.

Install

$ pip install ITTR-pytorch

Usage

They had 9 blocks of Hybrid Perception Block (HPB) in the paper

import torch
from ITTR_pytorch import HPB

block = HPB(
    dim = 512,              # dimension
    dim_head = 32,          # dimension per attention head
    heads = 8,              # number of attention heads
    attn_height_top_k = 16, # number of top indices to select along height, for the attention pruning
    attn_width_top_k = 16,  # number of top indices to select along width, for the attention pruning
    attn_dropout = 0.,      # attn dropout
    ff_mult = 4,            # expansion factor of feedforward
    ff_dropout = 0.         # feedforward dropout
)

fmap = torch.randn(1, 512, 32, 32)

out = block(fmap) # (1, 512, 32, 32)

You can also use the dual-pruned self-attention as so

import torch
from ITTR_pytorch import DPSA

attn = DPSA(
    dim = 512,         # dimension
    dim_head = 32,     # dimension per attention head
    heads = 8,         # number of attention heads
    height_top_k = 48, # number of top indices to select along height, for the attention pruning
    width_top_k = 48,  # number of top indices to select along width, for the attention pruning
    dropout = 0.       # attn dropout
)

fmap = torch.randn(1, 512, 32, 32)

out = attn(fmap) # (1, 512, 32, 32)

Citations

@inproceedings{Zheng2022ITTRUI,
  title   = {ITTR: Unpaired Image-to-Image Translation with Transformers},
  author  = {Wanfeng Zheng and Qiang Li and Guoxin Zhang and Pengfei Wan and Zhongyuan Wang},
  year    = {2022}
}
You might also like...

Dual-Path Attention and Recurrent Network for speech separation

DPARNet Light-weight speech separation based on dual-path attention and recurrent neural network 基于双路注意力循环网络的轻量化语音分离 [1] This paper has been submitted

Nov 4, 2022

Distributed linear attention and self-attention for long-sequence language models

This repository extends from the original Sequence Parallelism code. Sequence Parallelism Sequence Parallelism is a method to train long sequences in

Oct 4, 2022

Official Pytorch implementation of "Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis", ECCV 2022

Official Pytorch implementation of

Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis Project page | Paper "Injecting 3D Perception of

Nov 26, 2022

This repo presents you the official code of "VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention"

This repo presents you the official code of

VISTA VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention Shengheng Deng, Zhihao Liang, Lin Sun and Kui Jia* (*) Corresponding a

Nov 22, 2022

Dual-scale Doppler Attention for Human Identification

Dual-scale Doppler Attention for Human Identification. Pytorch code for the 2022 Sensor Journal paper DSDA: Dual-scale Doppler Attention for Human Ide

Sep 1, 2022

Code and Data repository for the paper titled "Enhancing PM2.5 Prediction Using NARX-Based Combined CNN and LSTM Hybrid Model"

This is the code and data associated with the paper titled: "Enhancing PM2.5 Prediction Using NARX-Based Combined CNN and LSTM Hybrid Model" The paper

Nov 18, 2022

Unofficial PyTorch implementation of the paper Transformer-based Dual Relation Graph for Multi-label Image Recognition. ICCV 2021

Unofficial PyTorch implementation of the paper Transformer-based Dual Relation Graph for Multi-label Image Recognition. ICCV 2021

TDRG Unofficial PyTorch implementation of the paper Transformer-based Dual Relation Graph for Multi-label Image Recognition. ICCV 2021 Architecture Th

Sep 29, 2022

This is the official repository for IEEE JSTSP paper "Surface Material Perception Through Multimodal Learning".

This is the official repository for IEEE JSTSP paper

Surface Material Perception This is the official repository for IEEE JSTSP paper "Surface Material Perception Through Multimodal Learning". Start up c

Sep 12, 2022

[ECCV 2022] Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations

[ECCV 2022] Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations

SelfDZSR (ECCV 2022) PyTorch implementation of Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations 1. Framework Ove

Nov 23, 2022
Comments
Releases(0.0.4)
Owner
Phil Wang
Working with Attention. It's all we need
Phil Wang
Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process with disentangled search and retrieval head aggregation, in Pytorch

Compositional Attention - Pytorch Implementation of Compositional Attention from MILA. They reframe the "heads" of multi-head attention as "searches",

Phil Wang 45 Nov 28, 2022
This is the official implementation code of the paper: HoW-3D: Holistic 3D Wireframe Perception from a Single Image

HoW-3D: Holistic 3D Wireframe Perception from a Single Image This is the official implementation code of our 3DV 2022 paper. Getting Started Clone the

Wenchao Ma 25 Nov 23, 2022
Code for paper "DaViT: Dual Attention Vision Transformer"

DaViT: Dual Attention Vision Transformer This repo contains the official detection and segmentation implementation of paper "DaViT: Dual Attention Vis

Mingyu Ding 173 Nov 10, 2022
Attention-based CNN-LSTM and XGBoost hybrid model for stock prediction

Attention-based CNN-LSTM and XGBoost hybrid model for stock prediction Requirements The code has been tested running under Python 3.7.4, with the foll

zshicode 73 Nov 21, 2022
Repository of our paper "Dual-distribution discrepancy with self-supervised refinement for anomaly detection in medical images"

DDAD-ASR This is the repository of our paper: Dual-distribution discrepancy with self-supervised refinement for anomaly detection in medical images Yu

Yu Cai 18 Nov 12, 2022
This is the code for the paper ''Person Re-Identification Baseline Based on Attention Block Neural Architecture Search''.

Person-Re-Identification-Baseline-Based-on-Attention-Block-Neural-Architecture-Search This is the Pytorch code of the paper ''Person Re-Identification

null 1 May 31, 2022
[ECCV2022] Official Implementation of paper "V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer"

V2X-ViT This is the official implementation of ECCV2022 paper "V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer". Install

Runsheng Xu 143 Nov 24, 2022
Pytorch implementation of "Block Recurrent Transformers" (Hutchins & Schlag et al., 2022)

Block Recurrent Transformer A PyTorch implementation of Hutchins & Schlag et al.. Owes very much to Phil Wang's x-transformers. Very much in-progress.

Dashiell Stander 56 Oct 29, 2022
Pytorch code for Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistency

PyTorch Implementation of Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistency. Viraj Prabhu*, Sriram Yen

Viraj Prabhu 12 Nov 22, 2022