[ECCV'22] This repository is the implementation of "Autoregressive 3D Shape Generation via Canonical Mapping".

Overview

Autoregressive 3D Shape Generation via Canonical Mapping

[arxiv]

Code release WIP:

  • Training/inference code
  • Installation and running instructions
  • Pre-trained checkpoints & generated samples
  • Evaluation using Jupyter notebook
  • Cleanup

Requirements

The code is developed and tested on Ubuntu 18.04, with python==3.8, CUDA==10.2.89, pytorch==1.10.0, pytorch_lightning==1.5.2.

By default, the Chamfer Loss module should work properly. If you failed to run the chamfer loss module, please see the following link and follow their instruction.

https://github.com/ThibaultGROUEIX/ChamferDistancePytorch

To install EMD Loss, please follow the instruction in here.

cd external/emd
python setup.py install

The installed build folder should be under external/emd.

Dataset

We use the processed version of ShapeNet provided PointFlow. Please download the dataset here. Don't forget to change the root path in src/data/core.py to your dataset location.

Train

To train the entire model from scratch, please follow these steps:

  1. Pre-train the canonical mapping:
python main.py --base configs/base/airplane.yaml -t True --gpus 0,1,2,3 -n WHATEVER_NAME
  1. Pre-train the grouping:
python main.py --base configs/stage1/128/airplane.yaml -t True --gpus 0,1,2,3 -n WHATEVER_NAME
  1. Train the VQVAE:
python main.py --base configs/stage2/128/airplane.yaml -t True --gpus 0,1,2,3 -n WHATEVER_NAME
  1. Train the Transformer:
python main.py --base configs/stage3/128/airplane.yaml -t True --gpus 0,1,2,3 -n WHATEVER_NAME

The checkpoint will be stored in the log directory, together with the tensorboard logger file and the generated samples. For each stage, please change the ckpt_path in the config file to the path of the checkpoints from last stage.

If you would like to start training from a certain stage, you can use the pre-trained checkpoints from this link and use the ignore_keys in the config file to determine which module(s) to be trained from scratch.

Evaluation

The pre-trained checkpoints and generated samples can be downloaded from this link. Please change the ckpt_path in configs/stage3/128/*CATEGORY*_test.yaml to either your trained checkpoints or or pre-trained checkpoints. Currently, the evaluation code only supports single GPU. If you use multiple GPUs, the code would still be runnable, but the calculated metrics may be incorrect.

python main.py --base configs/stage3/128/airplane_test.yaml -t False --gpus 0, -n airplan_test
python main.py --base configs/stage3/128/car_test.yaml -t False --gpus 0, -n car_test
python main.py --base configs/stage3/128/chair_test.yaml -t False --gpus 0, -n chair_test

The script will evaluate both auto-encoding performance and generation performance.

You can also directly evaluate the performance with our generated samples. Please download the samples from this link, and refer to benchmark.ipynb for details.

Reference

Please cite our paper (link) in your publications if this repo helps your research:

@inproceedings{cheng2022autoregressive,
    title     = {Autoregressive 3D Shape Generation via Canonical Mapping},
    author    = {Cheng, An-Chieh and Li, Xueting and Liu, Sifei and Sun, Min and Yang, Ming-Hsuan},
    booktitle = {ECCV},
    year      = {2022}
}

Acknowledgement

This repo is built on top of VQ-GAN. We thank the authors for sharing the codebase!

You might also like...

Generate and link local repository with remote repository with gitmate CLI.

Generate and link local repository with remote repository with gitmate CLI.

Gitmate 🐈‍⬛ Automating the process of creating and linking local repository with remote repository. Why to use Gitmate ? The general process of creat

Jul 31, 2022

This repository consists of a drowsiness detection based on YOLOv5 implementation.

This repository consists of a drowsiness detection based on YOLOv5 implementation.

Drowsiness-Detection-with-YoloV5 This repository consists of a drowsiness detection based on YOLOv5 implementation. You can reach the base repo here 1

Jul 17, 2022

Contribution to an open source repository which implements the Bayesian Optimization algorithm - Knowledge Gradient implementation

Contribution to an open source repository which implements the Bayesian Optimization algorithm - Knowledge Gradient implementation

Authors: Raffaella D'Anna Michele Di Sabato Martina Garavaglia Anna Iob Veronica Mazzola Open source project: original repository: https://github.com/

May 13, 2022

This repository includes the official implementation our paper "In Defense of Image Pre-Training for Spatiotemporal Recognition".

This repository includes the official implementation our paper

In Defense of Image Pre-Training for Spatiotemporal Recognition [arXiv] [NEW!] 2022/5/5 - We have released the code and models. Overview This is a PyT

Sep 13, 2022

This repository hosts an implementation of a classifier, that is based on the Commuting Quantum Query Language CQQL.

CQQL-Classifier This repository hosts an implementation of a classifier, that is based on the Commuting Quantum Query Language CQQL. It is based on th

May 26, 2022

This repository contains the official PyTorch implementation for Visual Prompt Tuning.

This repository contains the official PyTorch implementation for Visual Prompt Tuning.

This repository contains the official PyTorch implementation for Visual Prompt Tuning.

Sep 19, 2022

This repository is the implementation of "Don't Forget Me: Accurate Background Recovery for Text Removal via Modeling Local-Global Context".

CTRNet This repository is the implementation of "Don't Forget Me: Accurate Background Recovery for Text Removal via Modeling Local-Global Context". No

Sep 20, 2022

This repository contains the implementation for our work "Learning Topological Interactions for Multi-Class Medical Image Segmentation", accepted to ECCV 2022 (Oral)

This repository contains the implementation for our work

TopoInteraction This repository contains the implementation for our work "Learning Topological Interactions for Multi-Class Medical Image Segmentation

Sep 16, 2022

This repository is the official implementation of DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras.

This repository is the official implementation of DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras.

News 31/07/2022 We plan to release the THUman5.0 dataset for acadamic use. The dataset contains 10 dynamic human sequences which are captured by 32 RG

Sep 19, 2022
Comments
  • What should I put in for the `WHATEVER_NAME`?

    What should I put in for the `WHATEVER_NAME`?

    Here: https://github.com/AnjieCheng/CanonicalVAE/blob/e6d1401d20bc59f33689cb7031e598363c009549/README.md?plain=1#L53

    Your codes are too cool and too shiny, I cannot read them. Plz, help me.

    opened by hubert0527 2
Owner
An-Chieh Cheng
An-Chieh Cheng
The official implementation for ECCV22 paper: "FOSTER: Feature Boosting and Compression for Class-Incremental Learning" in PyTorch.

FOSTER: Feature Boosting and Compression for Class-Incremental Learning The code repository for "Feature Boosting and Compression for Class-Incrementa

Fu-Yun Wang 14 Sep 16, 2022
ECCV22 paper "Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality"

OrthoImproveCond ECCV22 paper "Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality" We propose nearest-orthogonal gradient (nog)

YueSong 3 Jul 6, 2022
The code for ECCV22 paper "Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis"

Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis Project | YouTube | arXiv Abstract: Video-to-Video synthesis (Vid2Vid) has ach

fast-vid2vid 87 Sep 23, 2022
Code release for Concurrent Subsidiary Supervision for Unsupervised Source-Free Domain Adaptation (ECCV22)

Concurrent Subsidiary Supervision for Source-Free DA (ECCV22) Code for our ECCV 2022 paper 'Concurrent Subsidiary Supervision for Unsupervised Source-

Video Analytics Lab -- IISc 12 Sep 21, 2022
Deep Semantic Statistics Matching (D2SM) Denoising Network (ECCV22)

Deep Semantic Statistics Matching (D2SM) Denoising Network Official PyTorch Implementation of the ECCV 2022 Paper Project | arXiv This repo contains t

Kangfu Mei 18 Aug 22, 2022
The official code for our ECCV22 oral paper: tracking objects as pixel-wise distributions.

[ECCV22 Oral] P3AFormer: Tracking Objects as Pixel-wise Distributions This is the official code for our ECCV22 oral paper: tracking objects as pixel-w

DV Lab 123 Sep 14, 2022
[ECCV22] BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering

BungeeNeRF This repository contains the code release for BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering, aka Ci

CitySuper 261 Sep 19, 2022
[ECCV22] BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering (Jittor)

BungeeNeRF This repository contains the code release (Jittor) for BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Renderin

CitySuper 3 Sep 17, 2022
A Sphinx extension to insert code snippets from a git repository, helping you to keep a repository of versions working code examples.

sphinxcontrib-gitinclude https://sphinxcontrib-gitinclude.readthedocs.org A Sphinx extension to insert code snippets from a git repository, helping yo

Shift Computing 1 May 8, 2022
Pooya Mohammadi Kazaj 2 Jun 30, 2022