BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations

Overview

BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations

This is the official code for:

BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations

Daiqing Li, Huan Ling, Seung Wook Kim, Karsten Kreis, Adela Barriuso, Sanja Fidler, Antonio Torralba
[Paper] [Bibtex] [Project Page]

Requirements

  • Python 3.7
  • Cuda v11.0+
  • gcc v7.3.0
  • Pytorch 1.9.0+

Pretrained BigGAN weights

Our annotation and model are based on BigGAN-512, please download the model from https://tfhub.dev/deepmind/biggan-512/2, store it in ./pretrain folder. Since the original model is trained using TensorFlow, you need to convert the model weights back to Pytorch, following the instructions here https://github.com/ajbrock/BigGAN-PyTorch/tree/master/TFHub. Notice the model is Licensed under Apache-2.0 issued by DeepMind.

Dataset preparation

We only release our annotations on sampled BigGAN images and images from ImageNet along with its latents used to get the sampled images. For their licenses, please refer to their websites. Notice our dataset release is under the Creative Commons BY-NC 4.0 license by NVIDIA Corporation. Please see License session for details.

  1. Download ImageNet from here.
  2. Download our annotations annotations.zip and latent codes latents.zip from gdrive. Unzip them into data folder under ./data/.
  3. Process images from ImageNet into our dataset format. Run the following script
python prepare_imagenet_images.py --imagenet_dir [path to imagenet dir] --dataset_dir ./data/
  1. Prepare images generated from BigGAN. Please download the pretrained weights following this session. And run
python prepare_biggan_images.py --biggan_ckpt ./pretrain/biggan-512.pth --dataset_dir ./data/

After the processing steps, you should have data folder structure like this

data
|
└───annotations
│   |
│   └───biggan512
│   |   │   n01440764
│   |   │   ...
|   └───real-random-list.txt
└───images
│   |
│   └───biggan512
│   |   │   n01440764
│   |   │   ...
|   └───real-random
│       │   n01440764
│       │   ...
└───latents
│   |
│   └───biggan512
│   |   │   n01440764
│   |   │   ...

Training

After the dataset preparation, we now can train BigDatasetGAN to synthesize dataset.

Run the following

python train.py --gan_ckpt ./pretrain/biggan-512.pth \
                --dataset_dir ./data/ \
                --save_dir ./logs/

You can monitor the training progress in tensorboard, as well as the training predictions in logs dir.

By default, the training runs 5k iteration with a batch size of 4, you can adjust it for the best capacity.

Sampling dataset

After the training is done, we can synthesize ImageNet with pixel-wise labels.

Run the following

python sample_dataset.py --ckpt [path to your pretrained BigDatasetGAN weights] \
                         --save_dir ./dataset_viz/ \
                         --class_idx 225, 200, [you can give it more with ImagetNet class idx] \
                         --samples_per_class 10
                         --z_var 0.9

As an example, here we sample class 225 and 200 with 10 samples each.

License

For any code dependency related to BigGAN, the license is under the MIT License, see https://github.com/ajbrock/BigGAN-PyTorch/blob/master/LICENSE.

The work BigDatasetGAN code is released under Creative Commons BY-NC 4.0 license, full text at http://creativecommons.org/licenses/by-nc/4.0/legalcode.

The dataset of BigDatasetGAN is released under the Creative Commons BY-NC 4.0 license by NVIDIA Corporation. You can use, redistribute, and adapt the material for non-commercial purposes, as long as you give appropriate credit by citing our paper and indicating any changes that you've made.

Citation

Please cite the following paper if you used the code in this repository.

@misc{li2022bigdatasetgan,
      title={BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations}, 
      author={Daiqing Li and Huan Ling and Seung Wook Kim and Karsten Kreis and Adela Barriuso and Sanja Fidler and Antonio Torralba},
      year={2022},
      eprint={2201.04684},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
You might also like...

Classic CV backbones for imagenet without pretrained based on pytorch.

CVBackbones-imagenet-no-pretrained classic CV backbones for imagenet without pretrained based on pytorch. Models The supported net args are: alexnet s

Aug 24, 2022

PyTorch repository for ICLR 2022 paper (GSAM) which improves generalization (e.g. +3.8% top-1 accuracy on ImageNet with ViT-B/32)

PyTorch repository for ICLR 2022 paper (GSAM) which improves generalization (e.g. +3.8% top-1 accuracy on ImageNet with ViT-B/32)

GSAM Optimizer Surrogate Gap Guided Sharpness-Aware Minimization [Paper, ICLR 2022] [Webpate] Disclaimer: original code (during internship at Google)

Sep 18, 2022

A large-scale repository of images for software-specific lexicons database called 'SE-ImageNet' to complement software engineering communities and computer vision researchers

SE-ImageNet Deenu summer research 2021 work - Preparing a large-scale repository of images for software-specific lexicons called 'SE-ImageNet' constru

May 25, 2022

ImageNet Annotation Tool - Frontend (FE)

 ImageNet Annotation Tool - Frontend (FE)

ImageNet Annotation Tool - Frontend (FE) ImageNet is an iconic training and benchmarking dataset in computer vision. Not only did it set a standard as

Aug 31, 2022

Bamboo: 4 times larger than ImageNet; 2 time larger than Object365; Built by active learning.

Bamboo: 4 times larger than ImageNet; 2 time larger than Object365; Built by active learning.

Zhang Yuanhan1  Sun Qinghong2  Zhou Yichun3  He Zexin3  Yin Zhenfei4  Wang Kun4  Sheng Lv3  Qiao Yu5  Shao Jing4  Liu Ziwei1 1S-Lab, Nanyang Technolog

Sep 23, 2022

"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (CVPRW 2022) Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Z

Sep 23, 2022

📦 PyTorch based visualization package for generating layer-wise explanations for CNNs.

📦 PyTorch based visualization package for generating layer-wise explanations for CNNs.

Explainable CNNs 📦 Flexible visualization package for generating layer-wise explanations for CNNs. It is a common notion that a Deep Learning model i

Sep 22, 2022

DIS18a/b Projektarbeit I - Linked Open Data und Knowledge Graphs (WiSe 2021/2022 und SoSe 2022) Group 1

DIS18a/b Projektarbeit I - Linked Open Data und Knowledge Graphs (WiSe 2021/2022 und SoSe 2022) Group 1 / Datenkrake In this repository you will find

May 4, 2022

Task-Aware Sampling Layer for Point-Wise Analysis, TVCG 2022

Task-Aware Sampling Layer for Point-Wise Analysis Yiqun Lin, Lichang Chen, Haibin Huang, Chongyang Ma, Xiaoguang Han, Shuguang Cui, "Task-Aware Sampli

Jul 4, 2022
Owner
null
Pixel-Wise Classification of Urban Hyperspectral Images with Convolutional Neural Networks

CNN_Hyperspectral_Classification Pixel-Wise Classification of Urban Hyperspectral Images with Convolutional Neural Networks This repo contains the cod

Farid Qamar 1 Sep 6, 2022
Research code for pixel-based encoders of language (PIXEL)

PIXEL This repository contains code for PIXEL, the Pixel-based Encoder of Language. PIXEL is a language model that operates on text rendered as images

Phillip Rust 170 Sep 19, 2022
A pixel art style filter tool made with python. It can help you turn the picture style into pixel art.

Pixel-Art-transform-in-python English | 繁體中文 A pixel art style filter tool made with python. It can help you turn the picture style into pixel art. Yo

JingShing 1 Sep 14, 2022
Maua is a Python library (and command line interface) for synthesizing images, video, and audio using deep learning.

Maua ?? ⛏️ WIP ??️ ?? Maua is a Python library (and command line interface) for synthesizing images, video, and audio using deep learning. While many

Maua 91 Sep 22, 2022
Tool from 'Synthesizing History and Prophecy Variables for Symbolic Model Checking'

Setup Create a virtual environment python3 -m venv ch Activate environment source ch/bin/activate Install requirements python -m pip install -r requir

Cole Vick 2 Sep 26, 2022
Computer Vision to the Rescue: Infant Postural Symmetry Estimation from Incongruent Annotations (2022))

Infant Postural Symmetry Measurement Codes and experiments for the following paper: Xiaofei Huang, Michael Wan, Lingfei Luan, Bethany Tunik, Sarah Ost

Sarah Ostadabbas 1 Aug 25, 2022
Official Code for ECCV2022: Learning Semantic Correspondence with Sparse Annotations

Learning Semantic Correspondence with Sparse Annotations (To be appeared at ECCV'22) For more information, check out the paper on [arXiv] (TODO). Pret

Shuaiyi Huang 14 Aug 25, 2022
Type annotations and runtime checking for shape and dtype of JAX arrays, and PyTrees.

jaxtyping Type annotations and runtime checking for: shape and dtype of JAX arrays; PyTrees. For example: from jaxtyping import f32, PyTree def matri

Google 269 Sep 21, 2022
Planning annotated Reinforcement Learning (PaRL) Agent annotations

parl_annotations defines abstract class for planning annotation reinforcement learning. Install conda create -n parl_minigrid python=3.7 conda activat

International Business Machines 1 Sep 14, 2022
Training ImageNet / CIFAR models with sota strategies and fancy techniques such as ViT, KD, NAS, Rep, etc.

Image Classification SOTA Image Classification SOTA is an image classification toolbox based on PyTorch. Updates March 24, 2022 Support training strat

Tao Huang 30 Sep 29, 2022