[BMVC 2021] AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposal Generation

Overview

AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation

PWC

A pytorch-version implementation codes of paper: "AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation", which is accepted in BMVC 2021.

Installation Guide

  1. conda create -n aei python=3.8
  2. conda activate aei
  3. conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch
  4. pip install tqdm pandas tensorboard matplotlib fvcore scipy
  5. cd into tapg-aei (root)
  6. git clone https://github.com/frostinassiky/align1d
  7. cd into tapg-aei/align1d
  8. pip install -e .
  9. cd ..

Download Features

3D Resnet-50 features extracted from rescaled videos of ActivityNet-1.3 can be downloaded below:

Training and Testing of AEI

Default configurations of AEI are stored in config/defaults.py. The modified configurations are stored in config/*.yaml for training and testing of AEI on different datasets (ActivityNet-1.3 and THUMOS-14). We can also modify configurations through commandline arguments.

  1. To train AEI on TAPG task of ActivityNet-1.3 with 1 GPU:
python main.py --cfg-file config/anet_proposals.yaml MODE 'training' GPU_IDS [0]
  1. To evaluate AEI on validation set of ActivityNet-1.3 with 1 GPU:
python main.py --cfg-file config/anet_proposals.yaml MODE 'validation' GPU_IDS [0]

Reference

This implementation is partly based on this pytorch-implementation of BMN for the boundary matching module.

paper: AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation

Q&A

1q. "UserWarning: This DataLoader will create 12 worker processes in total. Our suggested max number of worker in current system is #, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary."

1a. Change num_workers to # in line 171 of root>main.py>inference function

Citation

If you find AEI useful for your research, please consider citing:

@article{khoavoAEI2021,
  author    = {Khoa Vo and
               Hyekang Joo and
               Kashu Yamazaki and
               Sang Truong and
               Kris Kitani and
               Minh{-}Triet Tran and
               Ngan Le},
  title     = {{AEI:} Actors-Environment Interaction with Adaptive Attention for
               Temporal Action Proposals Generation},
  journal   = {CoRR},
  volume    = {abs/2110.11474},
  year      = {2021},
  url       = {https://arxiv.org/abs/2110.11474},
  eprinttype = {arXiv},
  eprint    = {2110.11474},
}

Contact

Khoa Vo:

You might also like...

Image Denoising Using Adaptive and Overlapped Average Filtering and Mixed-Pooling Attention Refinement Networks paper

Image Denoising Using Adaptive and Overlapped Average Filtering and Mixed-Pooling Attention Refinement Networks paper

AOAF-MARNs-paper Image Denoising Using Adaptive and Overlapped Average Filtering and Mixed-Pooling Attention Refinement Networks paper Network Architec

Jun 12, 2022

SDA-xNet: Selective Depth Attention Networks for Adaptive Multi-scale Feature Representation

SDA-xNet: Selective Depth Attention Networks for Adaptive Multi-scale Feature Representation Introduction Existing multi-scale solutions lead to a ris

Dec 23, 2022

code for our BMVC 2022 paper "Positive Pair Distillation Considered Harmful: Continual Meta Metric Learning for Lifelong Object Re-Identification"

Positive Pair Distillation Considered Harmful: Continual Meta Metric Learning for Lifelong Object Re-Identification (BMVC 2022) Here we afford the cod

Oct 18, 2022

MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks [BMVC 2022]

MUAD: Multiple Uncertainties for Autonomous Driving Dataset This repository contains some development kits (in PyTorch) that we used for our BMVC 2022

Nov 25, 2022

Source code for "Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors." (Spotlight at the BMVC 2022)

Source code for

Audio-visual Synchronisation with Trainable Selectors This is the official implementation of Iashin, V., Xie, W., Rahtu, E. and Zisserman, A. "Sparse

Dec 23, 2022

Multi-Scale Temporal Frequency Convolutional Network With Axial Attention for Speech Enhancement

MTFAA-Net Unofficial PyTorch implementation of Baidu's MTFAA-Net: "Multi-Scale Temporal Frequency Convolutional Network With Axial Attention for Speec

Jan 3, 2023

public repo for TANGO (Target Adaptive No-code neural network Generation and Operation framework)

public repo for TANGO (Target Adaptive No-code neural network Generation and Operation framework)

TANGO Table of Contents Introduction to TANGO Source Tree Structure How to build images and run containers Docker and Docker-compose Installation TANG

Dec 15, 2022

Designed a containerized pipeline to detect fake-face submissions using a CNN model including generation of fake datasets for machine learning in a Test Driven Development environment.

Designed a containerized pipeline to detect fake-face submissions using a CNN model including generation of fake datasets for machine learning in a Test Driven Development environment.

Sep 12, 2022

An extension library for interactions.py allowing files in interaction responses.

interactions-files An external for interactions.py to add files into CommandContext send. How to use It is pretty straight forward. Here is an example

Dec 3, 2022
Official implementation of "An Action Is Worth Multiple Words: Handling Ambiguity in Action Recognition", BMVC 2022

An Action Is Worth Multiple Words: Handling Ambiguity in Action Recognition Kiyoon Kim, Davide Moltisanti, Oisin Mac Aodha, Laura Sevilla-Lara In BMVC

Kiyoon Kim 9 Dec 23, 2022
Official repository of "Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control" (SIGGRAPH Asia 2021)

Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control Project Page | Video | Paper | Data Abstract: We propose Neural Actor (NA),

lingjie0206 67 Dec 20, 2022
Official Implementation for "Robust Action Segmentation from Timestamp Supervision" - BMVC 2022

Code for "Robust Action Segmentation from Timestamp Supervision" - BMVC 2022 Official implementation of Robust Action Segmentation from Timestamp Supe

Yasser Souri 4 Oct 25, 2022
An adversarial channel-temporal attention network on UDA for first-person action recognition.

First-Person Video Domain Adaptation with Multi-Scene Cross-Site Datasets and Attention-Based Methods Introduction This repository contains the PyTorc

Xianyuan Liu 3 Dec 16, 2022
code for our paper "Attention Distillation: self-supervised vision transformer students need more guidance" in BMVC 2022

Attention Distillation: self-supervised vision transformer students need more guidance (BMVC 2022) Kai Wang, Fei Yang and Joost van de Weijer Requirem

kai wang 8 Nov 25, 2022
Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process with disentangled search and retrieval head aggregation, in Pytorch

Compositional Attention - Pytorch Implementation of Compositional Attention from MILA. They reframe the "heads" of multi-head attention as "searches",

Phil Wang 45 Nov 28, 2022
Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation

The official implementation of 'Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation' (CVPR 2022)

null 31 Dec 4, 2022
A proposal for modular, upgradeable StarkNet contracts. Inspired by the Diamond Standard.

StarkNet Modular Contracts (SMC) Standard This repository presents a way to build and deploy modular StarkNet contracts. It is heavily inspired by the

Apibara 32 Oct 18, 2022
A proposal for modular, upgradeable StarkNet contracts. Inspired by the Diamond Standard.

StarkNet Modular Contracts (SMC) Standard This repository presents a way to build and deploy modular StarkNet contracts. It is heavily inspired by the

Apibara 31 Jun 24, 2022
Lightweight Image Super-Resolution with Multi-Scale Feature Interaction Network (ICME 2021)

MSFIN:Lightweight Image Super-Resolution with Multi-Scale Feature Interaction Network This repository is an official PyTorch implementation of the pap

Zhengxue Wang 8 Dec 28, 2022