Official Implementation of ECCV2022 paper "OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers"

Overview

OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers (ECCV 2022)

Official Implementation of "OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers"

OSFormer

Jialun Pei, Tianyang Cheng, Deng-Ping Fan, He Tang, Chuanbo Chen, and Luc Van Gool

The first two authors have equal contribution to this work

Image 1 Image 2 Image 3 Image 4

Environment preparation

The code is tested on CUDA 11.1 and pytorch 1.9.0, change the versions below to your desired ones.

git clone https://github.com/PJLallen/OSFormer.git
cd OSFormer
conda create -n osformer python=3.8 -y
conda activate osformer
conda install pytorch==1.9.0 torchvision cudatoolkit=11.1 -c pytorch -c nvidia -y
python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html
python setup.py build develop

Dataset preparation

Download the datasets

Register datasets

  1. generate coco annotation files, you may refer to the tutorial of mmdetection for some help
  2. change the path of the datasets as well as annotations in adet/data/datasets/cis.py, please refer to the docs of detectron2 for more help
# adet/data/datasets/cis.py
# change the paths 
DATASET_ROOT = 'COD10K-v3'
ANN_ROOT = os.path.join(DATASET_ROOT, 'annotations')
TRAIN_PATH = os.path.join(DATASET_ROOT, 'Train/Image')
TEST_PATH = os.path.join(DATASET_ROOT, 'Test/Image')
TRAIN_JSON = os.path.join(ANN_ROOT, 'train_instance.json')
TEST_JSON = os.path.join(ANN_ROOT, 'test2026.json')

NC4K_ROOT = 'NC4K'
NC4K_PATH = os.path.join(NC4K_ROOT, 'Imgs')
NC4K_JSON = os.path.join(NC4K_ROOT, 'nc4k_test.json')

Pre-trained models

Weights: Baidu (password:l6vn) / Google / Quark

Model Config COD10K-test AP NC4K-test AP
R50-550 configs/CIS_RT.yaml 36.0 41.4
R50 configs/CIS_R50.yaml 41.0 42.5
R101 configs/CIS_R101.yaml 42.0 44.4
PVTv2-B2-Li configs/CIS_PVTv2B2Li 47.2 50.5
SWIN-T configs/CIS_SWINT.yaml 47.7 50.2

Usage

Train

python tools/train_net.py --config-file configs/CIS_R50.yaml --num-gpus 1 \
  OUTPUT_DIR {PATH_TO_OUTPUT_DIR}

Please replace {PATH_TO_OUTPUT_DIR} to your own output dir

Eval

python tools/train_net.py --config-file configs/CIS_R50.yaml --eval-only \
  MODEL.WEIGHTS {PATH_TO_PRE_TRAINED_WEIGHTS}

Please replace {PATH_TO_PRE_TRAINED_WEIGHTS} to the pre-trained weights

Inference

python demo/demo.py --config-file configs/CIS_R50.yaml \
  --input {PATH_TO_THE_IMG_DIR_OR_FIRE} \
  --output {PATH_TO_SAVE_DIR_OR_IMAGE_FILE} \
  --opts MODEL.WEIGHTS {PATH_TO_PRE_TRAINED_WEIGHTS}
  • {PATH_TO_THE_IMG_DIR_OR_FIRE}: you can put image dir or image paths here
  • {PATH_TO_SAVE_DIR_OR_IMAGE_FILE}: the place where the visualizations will be saved
  • {PATH_TO_PRE_TRAINED_WEIGHTS}: please put the pre-trained weights here

Acknowledgement

This work is based on:

We also get help from mmdetection. Thanks them for their great work!

Citation

If this helps you, please cite this work:

@inproceedings{pei2022osformer,
  title={OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers},
  author={Pei, Jialun and Cheng, Tianyang and Fan, Deng-Ping and Tang, He and Chen, Chuanbo and Van Gool, Luc},
  booktitle={European conference on computer vision},
  year={2022},
  organization={Springer}
}
You might also like...

[ECCV2022] Official implementation of 'Learning Depth from Focus in the Wild'

Learning Depth from Focus in the Wild [ECCV2022] Official pytorch implementation of "Learning Depth from Focus in the Wild" Paper Contact email : cywo

Aug 26, 2022

Official MegEngine implementation of ECCV2022 "D2C-SR: A Divergence to Convergence Approach for Real-World Image Super-Resolution".

Official MegEngine implementation of ECCV2022

[ECCV 2022] D2C-SR: A Divergence to Convergence Approach for Real-World Image Super-Resolution Youwei Li$^1$, Haibin Huang$^2$, Lanpeng Jia$^1$, Haoqi

Sep 24, 2022

An official implementation of the PseCo (ECCV2022)

An official implementation of the PseCo (ECCV2022)

PseCo (ECCV 2022) PseCo: Pseudo Labeling and Consistency Training for Semi-Supervised Object Detection Gang Li, Xiang Li, Yujie Wang, Yichao Wu, Ding

Sep 26, 2022

The Official PyTorch Implementation for Face2Face^ρ (ECCV2022)

Face2Faceρ: Official Pytorch Implementation Environment CUDA 10.2 or above Python 3.8.5 pip install -r requirements.txt For visdom, some dependencies

Sep 16, 2022

Official Pytorch implementation of CCPL and SCTNet (ECCV2022, Oral)

Official Pytorch implementation of CCPL and SCTNet (ECCV2022, Oral)

CCPL: Contrastive Coherence Preserving Loss for Versatile Style Transfer (ECCV 2022 Oral) Paper | Video Demo | Web Demo @inproceedings{wu2022ccpl, t

Sep 21, 2022

[ECCV2022] The PyTorch implementation of paper "Equivariance and Invariance Inductive Bias for Learning from Insufficient Data"

[ECCV2022] The PyTorch implementation of paper

[ECCV2022] EqInv This repository contains the official PyTorch implementation of paper "Equivariance and Invariance Inductive Bias for Learning from I

Sep 21, 2022

Official Code for ECCV2022: Learning Semantic Correspondence with Sparse Annotations

Official Code for ECCV2022: Learning Semantic Correspondence with Sparse Annotations

Learning Semantic Correspondence with Sparse Annotations (To be appeared at ECCV'22) For more information, check out the paper on [arXiv] (TODO). Pret

Aug 25, 2022

The official code of WaveGAN: Frequency-aware GAN for High-Fidelity Few-shot Image Generation (ECCV2022)

Official codes for WaveGAN: An Frequency-aware GAN for High-Fidelity Few-shot Image Generation (ECCV2022) [Paper] Requirements imageio==2.9.0 lmdb==1.

Sep 29, 2022

Code of ECCV2022 paper "Inverted Pyramid Multi-task Transformer for Dense Scene Understanding"

Code of ECCV2022 paper

🔥 ECCV2022 InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understanding 📜 Introduction This repository implements our ECCV2022 paper

Sep 20, 2022
Owner
Pei Jialun
PHD. in school of computer science and technology at Huazhong university of science and technology.
Pei Jialun
[ECCV2022] Official Implementation of paper "V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer"

V2X-ViT This is the official implementation of ECCV2022 paper "V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer". Install

Runsheng Xu 115 Sep 22, 2022
[ECCV2022] Official PyTorch implementation of the paper "Outpainting by Queries"

QueryOTR Outpainting by Queries, ECCV 2022. ArXiv we propose a novel hybrid vision-transformer-based encoder-decoder framework, named Query Outpaintin

Kai Yao 16 Sep 20, 2022
This repo is the official megengine implementation of the ECCV2022 paper: Efficient One Pass Self-distillation with Zipf's Label Smoothing.

This repo is the official megengine implementation of the ECCV2022 paper: Efficient One Pass Self-distillation with Zipf's Label Smoothing. The pytorc

MEGVII Research 13 Sep 1, 2022
Official implementation of the ECCV2022 paper: Audio-Visual Segmentation

Audio-Visual Segmentation [Project Page] [Arxiv] This repository provides the PyTorch implementation for the ECCV2022 paper "Audio-Visual Segmentation

null 345 Sep 23, 2022
[ECCV2022] Official implementation of the paper "View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrums"

Codes for View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrums Abstract Understanding and forecasting future trajec

conghaowong 20 Sep 24, 2022
This repository is the official code of the ECCV2022 paper "Global Spectral Filter Memory Network for Video Object Segmentation"

Global Spectral Filter Memory Network for Video Object Segmentation ECCV 2022 Abstract This paper studies semi-supervised video object segmentation th

liuyong 17 Aug 27, 2022
Official code for ECCV2022 paper: Learning Series-Parallel Lookup Tables for Efficient Image Super-Resolution

SPLUT Official code for ECCV2022 paper: Learning Series-Parallel Lookup Tables for Efficient Image Super-Resolution The folder training_testing_code c

null 23 Sep 24, 2022
[ECCV2022] Official Pytorch Implementation of Object Discovery via Contrastive Learning for Weakly Supervised Object Detection

Object Discovery via Contrastive Learning for Weakly Supervised Object Detection Jinhwan Seo, Wonho Bae, Danica J. Sutherland, Junhyug Noh, and Daijin

Jinhwan Seo 9 Sep 22, 2022
Official Implementation of "Towards Sequence-Level Training for Visual Tracking" (ECCV2022)

SLTtrack Official implementation of the ECCV 2022 paper Towards Sequence-Level Training for Visual Tracking Minji Kim*, Seungkwan Lee*, Jungseul Ok, B

Minji Kim 29 Sep 21, 2022
Official implementation of "Self-slimmed Vision Transformer" (ECCV2022)

Self-slimmed Vision Transformer (ECCV2022) This repo is the official implementation of "Self-slimmed Vision Transformer". Updates 07/20/2022 [Initial

SenseTime X-Lab 15 Sep 23, 2022