[ECCV2022 Oral] Registration based Few-Shot Anomaly Detection

Related tags

Admin Panels RegAD
Overview

Registration based Few-Shot Anomaly Detection

This is an official implementation of “Registration based Few-Shot Anomaly Detection” (RegAD) with PyTorch, accepted by ECCV 2022 (Oral).

Paper Link

@inproceedings{huang2022regad,
  title={Registration based Few-Shot Anomaly Detection}
  author={Huang, Chaoqin and Guan, Haoyan and Jiang, Aofan and Zhang, Ya and Spratlin, Michael and Wang, Yanfeng},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2022}
}

Abstract: This paper considers few-shot anomaly detection (FSAD), a practical yet under-studied setting for anomaly detection (AD), where only a limited number of normal images are provided for each category at training. So far, existing FSAD studies follow the one-model-per-category learning paradigm used for standard AD, and the inter-category commonality has not been explored. Inspired by how humans detect anomalies, i.e., comparing an image in question to normal images, we here leverage registration, an image alignment task that is inherently generalizable across categories, as the proxy task, to train a category-agnostic anomaly detection model. During testing, the anomalies are identified by comparing the registered features of the test image and its corresponding support (normal) images. As far as we know, this is the first FSAD method that trains a single generalizable model and requires no re-training or parameter fine-tuning for new categories.

Keywords: Anomaly Detection, Few-Shot Learning, Registration

Get Started

Environment

  • python >= 3.7.11
  • pytorch >= 1.11.0
  • torchvision >= 0.12.0
  • numpy >= 1.19.5
  • scipy >= 1.7.3
  • skimage >= 0.19.2
  • matplotlib >= 3.5.2
  • kornia >= 0.6.5
  • tqdm

Files Preparation

  1. Download the MVTec dataset here.
  2. Download the support dataset for few-shot anomaly detection here and unzip the dataset.
    tar -xvf support_set.tar
    
    We hope the followers could use these support datasets to make a fair comparison between different methods.
  3. Download the pre-train models here and unzip the checkpoint files.
    tar -xvf save_checkpoints.tar
    

After the preparation work, the whole project should have the following structure:

./RegAD
├── README.md
├── train.py                                  # training code
├── test.py                                   # testing code
├── MVTec                                     # MVTec dataset files
│   ├── bottle
│   ├── cable
│   ├── ...                  
│   └── zippper
├── support_set                               # MVTec support dataset files
│   ├── 2
│   ├── 4                 
│   └── 8
├── models                                    # models and backbones
│   ├── stn.py  
│   └── siamese.py
├── losses                                    # losses
│   └── norm_loss.py  
├── datasets                                  # dataset                      
│   └── mvtec.py
├── save_checkpoints                          # model checkpoint files                  
└── utils                                     # utils
    ├── utils.py
    └── funcs.py

Quick Start

python test.py --obj $target-object --shot $few-shot-number --stn_mode rotation_scale

For example, if run on the category bottle with k=2:

python test.py --obj bottle --shot 2 --stn_mode rotation_scale

Training

python train.py --obj $target-object --shot $few-shot-number --data_type mvtec --data_path ./MVTec/ --epochs 50 --batch_size 32 --lr 0.0001 --momentum 0.9 --inferences 10 --stn_mode rotation_scale 

For example, to train a RegAD model on the MVTec dataset on bottle with k=2, simply run:

python train.py --obj bottle --shot 2 --data_type mvtec --data_path ./MVTec/ --epochs 50 --batch_size 32 --lr 0.0001 --momentum 0.9 --inferences 10 --stn_mode rotation_scale 

Then you can run the evaluation using:

python test.py --obj bottle --shot 2 --stn_mode rotation_scale

Results

Results of few-shot anomaly detection and localization with k=2:

AUC (%) Detection Localization
K=2 RegAD Inplementation RegAD Inplementation
bottle 99.4 99.7 98.0 98.6
cable 65.1 69.8 91.7 94.2
capsule 67.5 68.6 97.3 97.6
carpet 96.5 96.7 98.9 98.9
grid 84.0 79.1 77.4 77.5
hazelnut 96.0 96.3 98.1 98.2
leather 99.4 100 98.0 99.2
metal_nut 91.4 94.2 96.9 98.0
pill 81.3 66.1 93.6 97.0
screw 52.5 53.9 94.4 94.1
tile 94.3 98.9 94.3 95.1
toothbrush 86.6 86.8 98.2 98.2
transistor 86.0 82.2 93.4 93.3
wood 99.2 99.8 93.5 96.5
zipper 86.3 90.9 95.1 98.3
average 85.7 85.5 94.6 95.6

Results of few-shot anomaly detection and localization with k=4:

AUC (%) Detection Localization
K=4 RegAD Inplementation RegAD Inplementation
bottle 99.4 99.3 98.4 98.5
cable 76.1 82.9 92.7 95.5
capsule 72.4 77.3 97.6 98.3
carpet 97.9 97.9 98.9 98.9
grid 91.2 87 85.7 85.7
hazelnut 95.8 95.9 98.0 98.4
leather 100 99.9 99.1 99
metal_nut 94.6 94.3 97.8 96.5
pill 80.8 74.0 97.4 97.4
screw 56.6 59.3 95.0 96.0
tile 95.5 98.2 94.9 92.6
toothbrush 90.9 91.1 98.5 98.5
transistor 85.2 85.5 93.8 93.5
wood 98.6 98.9 94.7 96.3
zipper 88.5 95.8 94.0 98.6
average 88.2 89.2 95.8 96.2

Results of few-shot anomaly detection and localization with k=8:

AUC (%) Detection Localization
K=8 RegAD Inplementation RegAD Inplementation
bottle 99.8 99.8 97.5 98.5
cable 80.6 81.5 94.9 95.8
capsule 76.3 78.4 98.2 98.4
carpet 98.5 98.6 98.9 98.9
grid 91.5 91.5 88.7 88.7
hazelnut 96.5 97.3 98.5 98.5
leather 100 100 98.9 99.3
metal_nut 98.3 98.6 96.9 98.3
pill 80.6 77.8 97.8 97.7
screw 63.4 65.8 97.1 97.3
tile 97.4 99.6 95.2 96.1
toothbrush 98.5 96.6 98.7 99.0
transistor 93.4 90.3 96.8 95.9
wood 99.4 99.5 94.6 96.5
zipper 94.0 93.4 97.4 97.4
average 91.2 91.2 96.8 97.1

Visualization

Acknowledgement

We borrow some codes from SimSiam, STN and PaDiM

Contact

If you have any problem with this code, please feel free to contact [email protected].

You might also like...

[IJCAI2022 Oral] Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration

[IJCAI2022 Oral] Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration

UMFusion Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration [IJCAI2022 Oral Presentation]

Sep 14, 2022

Unsupervised Face Morphing Attack Detection via Self-paced Anomaly Detection

SPL-MAD: Unsupervised Face Morphing Attack Detection via Self-paced Anomaly Detection Note This is the official repository of the paper: Unsupervised

Aug 15, 2022

The official PyTorch implementation of the "AcroFOD: An Adaptive Method for Cross-domain Few-shot Object Detection".

AcroFOD The official PyTorch implementation of AcroFOD. The paper is accepted on the ECCV2022 and will be public soon. Things to do. The commiter is f

Sep 23, 2022

The source code of paper "Self-Supervised Task Augmentation for Few-Shot Intent Detection"

STAM The source code of paper "Self-Supervised Task Augmentation for Few-Shot Intent Detection" in JCST 2022. Reference If the code is used in your re

Jul 24, 2022

System that ranks 1st in DCASE 2022 Challenge Task 5: Few-shot Bioacoustic Event Detection

System that ranks 1st in DCASE 2022 Challenge Task 5: Few-shot Bioacoustic Event Detection

Few-shot bio-acoustic detection Few-shot learning is a highly promising paradigm for sound event detection. It is also an extremely good fit to the ne

Jul 26, 2022

ECCV2022 Towards Hard-Positive Query Mining for DETR-based Human-Object Interaction Detection

ECCV2022 Towards Hard-Positive Query Mining for DETR-based Human-Object Interaction Detection

Towards Hard-Positive Query Mining for DETR-based Human-Object Interaction Detection by Xubin Zhong, Changxing Ding, Zijian Li and Shaoli Huang. This

Sep 30, 2022

"Rethinking IoU-based Optimization for Single-stage 3D Object Detection", ECCV2022 accept!

"Rethinking IoU-based Optimization for Single-stage 3D Object Detection" Thanks for the OpenPCDet, this implementation of the RDIoU is mainly based on

Sep 8, 2022

Next-generation Video instance recognition framework on top of Detectron2 which supports SeqFormer(ECCV Oral) and IDOL(ECCV Oral))

Next-generation Video instance recognition framework on top of Detectron2 which supports SeqFormer(ECCV Oral) and IDOL(ECCV Oral))

VNext: VNext is a Next-generation Video instance recognition framework on top of Detectron2. Currently it provides advanced online and offline video i

Sep 29, 2022

Code for NAACL2022 Long Paper "An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling"

Code for NAACL2022 Long Paper

ESD Code For NAACL2022 Paper "An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling" 🔥 Introduction Few-Shot Sequence Labeling (

Sep 26, 2022
Comments
  • I can't understand the training way

    I can't understand the training way

    Dear author

    From your paper and source code, it is known that the target category is chosen to be tested, and other categories in the datasets are used for training. I am extremely confused about it. In my opinion, It should be that the target category is chosen to be tested, and all categories in the datasets are used for training.

    I will appreciate it if you can help me.

    opened by tanpengjie 8
  • i met this error when i preparing data

    i met this error when i preparing data

    support_set/metal_nut/
    support_set/metal_nut/8_10.pt
    tar: Unexpected EOF in archive
    tar: Unexpected EOF in archive
    tar: Error is not recoverable: exiting now
    
    opened by wzr0108 4
  • I don't understand the function of

    I don't understand the function of "fixtheta" in the STN Module.

    Dear author

    Thank you very much for sharing your source code. But I don't understand the below code in the STN Module:

    tmp = np.tile(np.array([0, 0, 1]), (x.shape[0], 1, 1)).astype(np.float32) fixtheta1 = torch.from_numpy(np.linalg.inv(np.concatenate((theta1.detach().cpu().numpy(), tmp), axis=1))[:,:-1,:]).cuda() self.stn1_output = self._fixstn(x.detach(), fixtheta1)

    I will appreciate it if you can help me.

    opened by tanpengjie 3
  • 不能运行python test.py --obj bottle --shot 2 --stn_mode rotation_scale

    不能运行python test.py --obj bottle --shot 2 --stn_mode rotation_scale

    win11编程环境。

    按照要求下载了所有tar文件,想办法将其解压(添加.gz后缀),所有的checkpoints预训练权重都是pca字样的(如bottle_2_pca_rotation_scale_model.pt),但是代码中是不带pca字样的,CKPT_name = f'./save_checkpoints/rotation_scale/{args.shot}/{args.obj}/{args.obj}_{args.shot}_rotation_scale_model.pt',

    故将其改为CKPT_name = f'./save_checkpoints/{args.shot}/{args.obj}/{args.obj}_{args.shot}_pca_rotation_scale_model.pt'之后,运行到STN.load_state_dict(model_CKPT['STN'])报错: KeyError: 'STN'

    请问如何解决,谢谢!

    opened by xqyd 2
Owner
null
The official code of WaveGAN: Frequency-aware GAN for High-Fidelity Few-shot Image Generation (ECCV2022)

Official codes for WaveGAN: An Frequency-aware GAN for High-Fidelity Few-shot Image Generation (ECCV2022) [Paper] Requirements imageio==2.9.0 lmdb==1.

Master of Science (Just call me 科大侠) 44 Sep 29, 2022
[ECCV2022] The implementation for "Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis".

DFRF The pytorch implementation for our ECCV2022 paper "Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis". Requirements Pyt

Shuai Shen 92 Oct 1, 2022
(ECCV2022) This is the official PyTorch implementation of ECCV2022 paper: Towards Efficient and Scale-Robust Ultra-High-Definition Image Demoireing

Towards Efficient and Scale-Robust Ultra-High-Definition Image Demoiréing Project Page | Dataset | Paper Towards Efficient and Scale-Robust Ultra-High

CVMI Lab 60 Sep 19, 2022
Official PyTorch Implementation of Learning What Not to Segment: A New Perspective on Few-Shot Segmentation (CVPR 2022 Oral).

Learning What Not to Segment: A New Perspective on Few-Shot Segmentation This repo contains the code for our CVPR 2022 Oral paper "Learning What Not t

null 118 Sep 29, 2022
Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

T-Few This repository contains the official code for the paper: "Few-Shot Parameter-Efficient Fine-Tuning Outperforms In-Context Learning". This metho

null 176 Sep 27, 2022
PyTorch implementation of SSQL (Accepted to ECCV2022 oral presentation)

SSQL-ECCV2022 Official code for Synergistic Self-supervised and Quantization Learning (Accepted to ECCV 2022 oral presentation). Paper is now availabl

MEGVII Research 53 Sep 26, 2022
[ECCV2022,oral] Identifying Hard Noise in Long-Tailed Sample Distribution

Identifying Hard Noise in Long-Tailed Sample Distribution This project introduces a new challenge as Noisy Long-Tailed Classification (NLT) and three

null 16 Sep 20, 2022
Adaptive Patch Exiting for Scalable Single Image Super-Resolution" (ECCV2022 Oral)

Adaptive Patch Exiting for Scalable Single Image Super-Resolution (ECCV2022 Oral) This repository is an official PyTorch implementation of the paper "

null 13 Sep 24, 2022
Official Pytorch implementation of CCPL and SCTNet (ECCV2022, Oral)

CCPL: Contrastive Coherence Preserving Loss for Versatile Style Transfer (ECCV 2022 Oral) Paper | Video Demo | Web Demo @inproceedings{wu2022ccpl, t

Jarrent Wu 80 Sep 21, 2022
PyTorch implementation of SSQL (Accepted to ECCV2022 oral presentation)

SSQL-ECCV2022 Official code for Synergistic Self-supervised and Quantization Learning (Accepted to ECCV 2022 oral presentation). Paper is now availabl

MEGVII Research 44 Aug 18, 2022