Pytorch implementation of diffusion models on Lie Groups for 6D grasp pose generation https://sites.google.com/view/se3dif/home

Overview

Pytorch implementation of Diffusion models in SE(3) for grasp and motion generation

This library provides the tools for training and sampling diffusion models in SE(3), implemented in PyTorch. We apply them to learn 6D grasp distributions. We use the learned distribution as cost function for grasp and motion optimization problems. See reference [1] for additional details.

[Website] [Preprint]

diffusiondiffusiondiffusiondiffusion

Installation

Create a conda environment

conda env create -f environment.yml

Activate the environment and install the library

conda activate se3dif_env && pip install -e .

Preproccessed Acronym [2] and Shapenet dataset [3]

Download the preprocessed dataset and unzip the folder in the before the root directory

  1. Mugs (Grasps, Meshes, SDF, Pointcloud) data

Trained Models

Download the trained models and unzip the folder in the before the root directory

  1. Pointcloud conditioned SE(3) GraspDiffusion model PointGraspDif

Sample examples

Generate SE(3) grasp poses

python scripts/sample/generate_pointcloud_6d_grasp_poses.py --n_poses 10 --n_obj 0

References

[1] Julen Urain*, Niklas Funk*, Georgia Chalvatzaki, Jan Peters. "SE(3)-DiffusionFields: Learning cost functions for joint grasp and motion optimization through diffusion" Under review 2022. [arxiv]

@article{urain2022se3dif,
  title={Learning cost functions for joint grasp and motion optimization through diffusion},
  author={Urain, Julen and Funk, Niklas and Chalvatzaki, Georgia and Peters, Jan},
  journal={https://arxiv.org/pdf/2209.03855.pdf},
  year={2022}

[2] Eppner Clemens, Arsalan Mousavian, Dieter Fox. "Acronym: A large-scale grasp dataset based on simulation." IEEE International Conference on Robotics and Automation (ICRA). 2021 [arxiv]

[3] Chang Angel X., et al. "Shapenet: An information-rich 3d model repository." arXiv preprint arXiv:1512.03012. 2015 [arxiv]

Contact

If you have any questions or find any bugs, please let me know: Julen Urain julen[at]robot-learning[dot]de

You might also like...

Long-form text-to-images generation, using a pipeline of deep generative models (GPT-3 and Stable Diffusion)

Long-form text-to-images generation, using a pipeline of deep generative models (GPT-3 and Stable Diffusion)

Long Stable Diffusion: Long-form text to images e.g. story - Stable Diffusion - illustrations Right now, Stable Diffusion can only take in a short p

Sep 29, 2022

Minimal diffusion model for generating MNIST, from 'Classifier-Free Diffusion Guidance'

Minimal diffusion model for generating MNIST, from 'Classifier-Free Diffusion Guidance'

Conditional Diffusion MNIST script.py is a minimal, self-contained implementation of a conditional diffusion model. It learns to generate MNIST digits

Sep 28, 2022

Stable Diffusion web UI - A browser interface based on Gradio library for Stable Diffusion

Stable Diffusion web UI - A browser interface based on Gradio library for Stable Diffusion

Stable Diffusion web UI A browser interface based on Gradio library for Stable Diffusion. Features Detailed feature showcase with images: Original txt

Sep 27, 2022

Self-contained, minimalistic implementation of diffusion models with Pytorch.

minDiffusion Goal of this educational repository is to provide a self-contained, minimalistic implementation of diffusion models using Pytorch. Many i

Sep 22, 2022

Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in Pytorch

Retrieval-Augmented Denoising Diffusion Probabilistic Models (wip) Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in P

Sep 20, 2022

An implementation of Elucidating the Design Space of Diffusion-Based Generative Models (Karras et al., 2022) for PyTorch.

k-diffusion An implementation of Elucidating the Design Space of Diffusion-Based Generative Models (Karras et al., 2022) for PyTorch. This repo is a w

Sep 27, 2022

A smart home SDK uses the intranet loopback address socket to input the scene. Through the yolov5 and Media Pose network analysis contained in the SDK, it gives whether the scene is effectively triggered.

A smart home SDK uses the intranet loopback address socket to input the scene. Through the yolov5 and Media Pose network analysis contained in the SDK, it gives whether the scene is effectively triggered.

Introduction A smart home SDK uses the intranet loopback address socket to input the scene. Through the yolov5 and Media Pose network analysis contain

May 2, 2022

Score-based Generative Models (Diffusion Models) for Speech Enhancement and Dereverberation

Score-based Generative Models (Diffusion Models) for Speech Enhancement and Dereverberation

Speech Enhancement and Dereverberation with Diffusion-based Generative Models This repository contains the official PyTorch implementations for the 20

Sep 29, 2022

Implementation of AudioLM, a Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

Implementation of AudioLM, a Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

AudioLM - Pytorch (wip) Implementation of AudioLM, a Language Modeling Approach to Audio Generation out of Google Research, in Pytorch Citations @inpr

Sep 23, 2022
Comments
  • Update README for running `scripts/sample/generate_pointcloud_6d_grasp_poses.py`

    Update README for running `scripts/sample/generate_pointcloud_6d_grasp_poses.py`

    Hi @TheCamusean

    Thanks for sharing the code for this exciting work. I was able to get sample examples to run smoothly. There is a minor issue that might help others save some time.

    There is a mismatch in the names of the flag here in the README. There are no flags n_poses or n_obj in generate_pointcloud_6d_grasp_poses.py. https://github.com/TheCamusean/grasp_diffusion/blob/f7493dbb60f623e033ebe79852df182986fc00d4/README.md?plain=1#L41

    The flags should be updated to --n_grasps and --obj_id instead?

    python scripts/sample/generate_pointcloud_6d_grasp_poses.py --n_grasps 10 --obj_id 0
    

    Also, it might help to clarify in the README, the directory structure imposed by se3dif/utils/torch_utils.py and how ACRONYM data and pretrained models should be organized.

    Cheers!

    opened by kuldeepbrd1 2
Owner
Julen Urain
PhD. in Robot Learning at IAS under the supervision of Jan Peters
Julen Urain
Implementation for ICRA 2021 paper "RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Images"

rgbd_graspnet Implementation for ICRA 2021 paper RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Images. Project Website https://graspnet.ne

Gou Minghao 17 Sep 24, 2022
Stable Diffusion Video to Video, Image to Image, Template Prompt Generation system and more, for use with any stable diffusion model

SDUtils: Stable Diffusion Utility Wrapper Stable Diffusion General utilities wrapper including: Video to Video, Image to Image, Template Prompt Genera

null 12 Sep 24, 2022
Official implementation of MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation (https://arxiv.org/abs/2205.09853)

MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation Vikram Voleti*, Alexia Jolicoeur-Martineau*, Christopher Pal We

Vikram Voleti 83 Sep 23, 2022
Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch

these fireworks do not exist Video Diffusion - Pytorch (wip) Text to video, it is happening! Official Project Page Implementation of Video Diffusion M

Phil Wang 392 Sep 27, 2022
(CVPR 2022 - oral) Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry

Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry Official implementation of the paper Multi-View Depth Est

Bae, Gwangbin 101 Sep 25, 2022
Official PyTorch implementation for paper: Diffusion-GAN: Training GANs with Diffusion

Diffusion-GAN — Official PyTorch implementation Diffusion-GAN: Training GANs with Diffusion Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen a

Daniel 86 Sep 27, 2022
Implementation of Bit Diffusion, Hinton's group's attempt at discrete denoising diffusion, in Pytorch

Bit Diffusion - Pytorch Implementation of Bit Diffusion, Hinton's group's attempt at discrete denoising diffusion, in Pytorch It seems like they misse

Phil Wang 98 Sep 30, 2022
Unconditional audio generation using diffusion models, in PyTorch.

Audio Diffusion - PyTorch Unconditional audio generation using diffusion models, in PyTorch. The goal of this repository is to explore different archi

archinet 368 Sep 22, 2022
Small library containing basic functions for lie algebra (SO(3) and SE(3)) matrix manipulation

LibLie LibLie is a small lib containing utils functions for matrix operations on the special orthogonal group SO(3), the special euclidian group SE(3)

Northern Robotics Laboratory 4 May 13, 2022
Simple google search parser, based on https://rapidapi.com/apigeek/api/google-search3 api

Google Search Parser Simple google search parser, based on https://rapidapi.com/apigeek/api/google-search3 api Setup Install all modules with pip inst

Shitcoder 1 May 27, 2022