POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples (NeurIPS 2021)

Overview

Poodle

This repository contains the implementation of the paper POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples.

Duong H. Le*, Khoi D. Nguyen*, Khoi Nguyen, Quoc-Huy Tran, Rang Nguyen, Binh-Son Hua (NeurIPS 2021)

TLDR: We leverage samples from distractor classes or randomly generated noise to improve the generalization of few-shot learner.

Citation

If you find our paper/code helpful, please cite our paper:

@inproceedings{
le2021poodle,
title={{POODLE}: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples},
author={Duong Hoang Le and Khoi Duc Nguyen and Khoi Nguyen and Quoc-Huy Tran and Rang Nguyen and Binh-Son Hua},
booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
year={2021},
url={https://openreview.net/forum?id=wEvO8BCqZcm}
}

Introduction

In this work, we propose to use out-of-distribution samples, i.e., unlabeled samples coming from outside the target classes, to improve few-shot learning. Specifically, we exploit the easily available out-of-distribution samples to drive the classifier to avoid irrelevant features by maximizing the distance from prototypes to out-of-distribution samples while minimizing that of in-distribution samples (i.e., support, query data). Our approach is simple to implement, agnostic to feature extractors, lightweight without any additional cost for pre-training, and applicable to both inductive and transductive settings. Extensive experiments on various standard benchmarks demonstrate that the proposed method consistently improves the performance of pretrained networks with different architectures.

Usage

1. Download datasets

  1. Download these zipped files and put them into ./data

  2. (Optional) Download pretrained checkpoints here and extract to ./results.

  3. Run init.sh to preprocess all data.

After these steps, the your folder should be organized as follow:

results/
├── cub/
    ├── resnet12/
    ├────── student_0/
    ├────── student_1/
    ├────── checkpoint.pth.tar
    ├────── model_best.pth.tar
    ├── resnet12_ssl/
├── mini/
    ├── resnet12/
    ├── resnet12_ssl/
    ├── ...
├── tiered/
    ├── resnet12/
    ├── resnet12_ssl/
    ├── ...
data/
├── images/
    ├── n0153282900000005.jpg
    ├── n0153282900000006.jpg
    ├── ...
├── tiered-imagenet/
    ├── data/
    ├── class_names.txt
    ├── ...
├── CUB_200_100/
    ├── attributes/
    ├── images/
    ├── ...
├── split/
    ├── mini/
    ├────── train.csv
    ├────── val.csv
    ├────── test.csv
    ├── tiered/
    ├────── train.csv
    ├────── val.csv
    ├────── test.csv
    ├── cub/
    ├────── train.csv
    ├────── val.csv
    ├────── test.csv
assets/
configs/
src/
...

2. How to run

To run the code:

  1. Reconfigurate argument in run.sh (please read the comment to adjust the dataset/architecture). Quick guideline:

    • To train the model, remove the option --evaluate.
    • To train the model with rotation loss, add --do-ssl.
    • Note that, knowledge distillation is done after finishing training automatically.
    • Set save_path in [resnet12 | mobilenet | wideres | ...].config to different checkpoints for simple, rot, and rot+kd baselines for example:
      • simple: set save_path to ./results/mini/resnet12.
      • rot: set save_path to ./results/mini/resnet12_ssl.
      • rot+kd: set save_path to ./results/mini/resnet12_ssl/student_1.
  2. Run bash run.sh

Acknowledgement

Our implementation is based on the the official implementation of Simple Shot and TIM.

You might also like...

Multi-Step Deductive Reasoning Over Natural Language: An Empirical Study on Out-of-Distribution Generalisation

Multi-step-Deductive-Reasoning-over-Natural-Language This repository contains the code for the paper "Multi-Step Deductive Reasoning Over Natural Lang

Nov 17, 2022

ECCV 2022 Workshop: A Benchmark for Robustness to Individual Nuisances in Real-World Out-of-Distribution Shifts

ROBIN-dataset Changes The Phase-1 of the competition will not be a code submission challenge, we have released all the test data and labels in this re

Dec 6, 2022

Train your model from scratch or fine-tune a pretrained model using the losses provided in this library to improve out-of-distribution detection and uncertainty estimation performances

Train your model from scratch or fine-tune a pretrained model using the losses provided in this library to improve out-of-distribution detection and uncertainty estimation performances

Train your model from scratch or fine-tune a pretrained model using the losses provided in this library to improve out-of-distribution detection and uncertainty estimation performances. Calibrate your model to produce enhanced uncertainty estimations. Detect out-of-distribution data using the defined score type and threshold.

Nov 29, 2022

GOOD: A Graph Out-of-Distribution Benchmark

GOOD: A Graph Out-of-Distribution Benchmark

✨ GOOD: A Graph Out-of-Distribution Benchmark ✨ Documentation | Paper (and Leaderboard) 🔥 New! The GOOD is now also parts of the software library DIG

Jan 4, 2023

NeurIPS22 "RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection"

NeurIPS22

RankFeat NeurIPS22 paper "RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection" ID/OOD Dataset Preparation In-Distribution (ID) dataset.

Nov 19, 2022

pytorch implementation of a WACV 2021 Paper "Class-agnostic Few-shot-Object-Counting"

pytorch implementation of a WACV 2021 Paper

Class agnostic Few shot Object Counting This repository is the non-official pytorch implementation of a WACV 2021 Paper "Class-agnostic Few-shot-Objec

Dec 30, 2022

Meta-learning with Self-Improving Momentum Target (NeurIPS 2022)

Meta-learning with Self-Improving Momentum Target (NeurIPS 2022)

SiMT: Self-improving Momentum Target Official PyTorch implementation of "Meta-learning with Self-Improving Momentum Target" (NeurIPS 2022) by Jihoon T

Dec 7, 2022

Official PyTorch Implementation of Learning What Not to Segment: A New Perspective on Few-Shot Segmentation (CVPR 2022 Oral).

Official PyTorch Implementation of Learning What Not to Segment: A New Perspective on Few-Shot Segmentation (CVPR 2022 Oral).

Learning What Not to Segment: A New Perspective on Few-Shot Segmentation This repo contains the code for our CVPR 2022 Oral paper "Learning What Not t

Dec 31, 2022

Repository for the paper : ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning

Repository for the paper : ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning

ME-D2N_for_CDFSL Repository for the paper : ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning (to appear in ACM M

Dec 11, 2022
Comments
  • some question about loss function

    some question about loss function

    Hi~ thank you for your code ^^ I found that loss function below is only used in evaluate, and in training use cross-entropy to train base class. image maybe i misunderstood, but in this way, it doesn't do anything for training. can you help me get over :)

    opened by ruixuan-gao 4
Owner
Duong H. Le
...
Duong H. Le
Efficiently Improving Classification Accuracy, Uncertainty Estimation, and Out-of-Distribution Detection Simply Replacing the Loss and Calibrating

Improve classification accuracy, uncertainty estimation, and out-of-distribution detection (open set recognition) by changing a few lines of code in your project! Perform efficient inferences (i.e., do not increase inference time) without repetitive model training, hyperparameter tuning, or collecting additional data.

David Macêdo 39 Dec 27, 2022
[NeurIPS 2022] Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs

Causality Inspired Invariant Graph LeArning (CIGA) This repo contains the sample code for reproducing the results of NeurIPS 2022 paper: Learning Caus

Andrew Chan 45 Jan 7, 2023
Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

T-Few This repository contains the official code for the paper: "Few-Shot Parameter-Efficient Fine-Tuning Outperforms In-Context Learning". This metho

null 220 Dec 31, 2022
code for our paper "Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition" in CVPR 2022 3rd CLVISION continual learning workshop

ERD for IML code for our CVPRW 2022 paper Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition by Kai Wang, Xiale

kai wang 2 Oct 3, 2022
Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

?? Flamingo - Pytorch Implementation of Flamingo, state-of-the-art few-shot visual question answering attention net, in Pytorch. It will include the p

Phil Wang 627 Dec 27, 2022
This repository contains the code of the distribution shift framework presented in A Fine-Grained Analysis on Distribution Shift (Wiles et al., 2022).

Distribution Shift Framework This repository contains the code of the distribution shift framework presented in A Fine-Grained Analysis on Distributio

DeepMind 60 Nov 30, 2022
[3DV 2022] Visual Localization via Few-Shot Scene Region Classification.

[3DV 2022] Visual Localization via Few-Shot Scene Region Classification *Siyan Dong, *Shuzhe Wang, Yixin Zhuang, Juho Kannala, Marc Pollefeys, Baoquan

Siyan Dong 22 Nov 26, 2022
CaCo: Both Positive and Negative Samples are Directly Learnable via Cooperative-adversarial Contrastive Learning

CaCo CaCo is a contrastive-learning based self-supervised learning methods, which is submitted to IEEE-T-PAMI. Copyright (C) 2020 Xiao Wang, Yuhang Hu

Lab for MAchine Perception and LEarning (MAPLE) 11 Sep 14, 2022
Code for our Findings of EMNLP-2022 paper: "Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning"

MMBS (Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning) Here is the implementation of our Findings

Qingyi Si 6 Oct 18, 2022
The repository provides scripts to extract road user behavior distribution out of datasets.

Quantification of Actual Road User Behavior on the Basis of Given Traffic Rules Code and supplementary materials Repository of the paper Quantificatio

FZI Forschungszentrum Informatik 3 Aug 22, 2022